2024-04-20T10:50:11.224 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T10:50:11.224 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497 --name rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --verbose -- /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/orig.config.yaml 2024-04-20T10:50:11.278 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T10:50:11.364 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497 branch: wip-rishabh-testing-20240416.193735 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} email: null first_in_suite: false job_id: '7664497' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - Replacing daemon mds sha1: a8c84f5997950fd7d008204839253291d52e6eda ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH6OQS2TZtB4oGflvkl/p/zQruUyfhs+3zLMeL+zbjuRQSdVNAPdlMUtZ77HbtVvr3qonxSOCRcvDwShsLOc6mQ= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKRllXXUQ26mcfrH6E+Cd9sxM9XhjEJMmvuw/QCC0oGUM1jPNGGqrhbC9SoT/2O/4ygDa9Ejot2zF49ezENpeVM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - mds_thrash: null - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_09:50:15 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T10:50:11.365 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T10:50:11.365 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T10:50:11.365 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T10:50:11.367 INFO:teuthology.task.internal:Checking packages... 2024-04-20T10:50:11.390 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T10:50:11.390 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:50:11.390 INFO:teuthology.packaging:ref: None 2024-04-20T10:50:11.391 INFO:teuthology.packaging:tag: None 2024-04-20T10:50:11.391 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T10:50:11.391 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:50:11.391 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T10:50:11.607 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T10:50:11.608 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T10:50:11.615 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T10:50:11.615 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T10:50:11.634 INFO:teuthology.task.internal:Saving configuration 2024-04-20T10:50:11.655 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T10:50:11.661 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T10:50:11.682 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:44:02.526732', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH6OQS2TZtB4oGflvkl/p/zQruUyfhs+3zLMeL+zbjuRQSdVNAPdlMUtZ77HbtVvr3qonxSOCRcvDwShsLOc6mQ='} 2024-04-20T10:50:11.702 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:44:02.525210', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKRllXXUQ26mcfrH6E+Cd9sxM9XhjEJMmvuw/QCC0oGUM1jPNGGqrhbC9SoT/2O/4ygDa9Ejot2zF49ezENpeVM='} 2024-04-20T10:50:11.703 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T10:50:11.708 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-20T10:50:11.709 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-20T10:50:11.709 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T10:50:11.767 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5e1a106160>, signals=[15]) 2024-04-20T10:50:11.767 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T10:50:11.773 INFO:teuthology.task.internal:Opening connections... 2024-04-20T10:50:11.773 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T10:50:11.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:50:11.855 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T10:50:11.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:50:11.930 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T10:50:11.936 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-04-20T10:50:11.955 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-04-20T10:50:11.955 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-04-20T10:50:12.015 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:50:12.016 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:50:12.017 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:50:12.017 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-04-20T10:50:12.042 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-04-20T10:50:12.060 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-04-20T10:50:12.060 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:50:12.117 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:50:12.118 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:50:12.118 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:50:12.118 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:50:12.118 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:50:12.118 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-04-20T10:50:12.133 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T10:50:12.143 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T10:50:12.148 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T10:50:12.148 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:50:12.151 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:50:12.172 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T10:50:12.179 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T10:50:12.179 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:50:12.209 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:50:12.316 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T10:50:12.360 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T10:50:12.374 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T10:50:12.375 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-20T10:50:12.375 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-20T10:50:12.375 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:50:12.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:50:12.393 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-04-20T10:50:12.451 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-437.el9.x86_64 2024-04-20T10:50:12.451 INFO:teuthology.task.kernel:Running kernel on smithi027: 5.14.0-437.el9.x86_64 2024-04-20T10:50:12.451 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2024-04-20T10:50:15.205 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-20T10:50:17.056 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-20T10:50:21.890 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-04-20T10:50:23.357 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-20T10:50:25.934 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-04-20T10:50:31.084 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-20T10:50:33.065 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-20T10:50:33.065 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-20T10:50:33.158 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-20T10:50:33.159 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2024-04-20T10:50:33.159 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-20T10:50:33.262 DEBUG:teuthology.orchestra.run.smithi027:> echo no | sudo yum reinstall kernel || true 2024-04-20T10:50:33.786 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:02 ago on Sat 20 Apr 2024 10:50:31 AM UTC. 2024-04-20T10:50:33.921 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-20T10:50:33.935 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-20T10:50:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-04-20T10:50:33.935 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-20T10:50:33.935 INFO:teuthology.orchestra.run.smithi027.stdout:Reinstalling: 2024-04-20T10:50:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-20T10:50:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:50:33.935 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-20T10:50:33.935 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-20T10:50:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:50:33.936 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 255 k 2024-04-20T10:50:33.936 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 0 2024-04-20T10:50:33.936 INFO:teuthology.orchestra.run.smithi027.stderr:Operation aborted. 2024-04-20T10:50:33.987 INFO:teuthology.orchestra.run.smithi027.stdout:Is this ok [y/N]: 2024-04-20T10:50:33.987 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum reinstall -y kernel || true 2024-04-20T10:50:34.502 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:03 ago on Sat 20 Apr 2024 10:50:31 AM UTC. 2024-04-20T10:50:34.622 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-20T10:50:34.623 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-20T10:50:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-04-20T10:50:34.623 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-20T10:50:34.623 INFO:teuthology.orchestra.run.smithi027.stdout:Reinstalling: 2024-04-20T10:50:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-20T10:50:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:50:34.623 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-20T10:50:34.623 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-04-20T10:50:34.623 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:50:34.624 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 255 k 2024-04-20T10:50:34.624 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 0 2024-04-20T10:50:34.624 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-20T10:50:35.078 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-20T10:50:35.078 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:50:35.078 INFO:teuthology.orchestra.run.smithi027.stdout:Total 561 kB/s | 255 kB 00:00 2024-04-20T10:50:35.098 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-20T10:50:35.265 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-20T10:50:35.265 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-20T10:50:35.270 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-20T10:50:35.270 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-20T10:50:35.418 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-20T10:50:35.495 INFO:teuthology.orchestra.run.smithi027.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-20T10:50:35.496 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-20T10:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-20T10:50:36.165 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-20T10:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-20T10:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout:Reinstalled: 2024-04-20T10:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-20T10:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:50:36.858 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-20T10:50:37.018 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-04-20T10:50:37.067 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-20T10:50:37.067 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-20T10:50:37.067 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-20T10:50:37.067 DEBUG:teuthology.parallel:result is None 2024-04-20T10:50:37.068 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T10:50:37.075 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T10:50:37.075 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:50:37.078 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:50:37.096 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T10:50:37.103 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T10:50:37.113 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T10:50:37.113 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:50:37.133 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:50:37.382 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T10:50:37.389 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T10:50:37.389 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:50:37.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:50:37.406 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:50:37.408 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:50:37.435 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:50:37.454 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:50:37.490 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:50:37.505 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:50:37.507 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T10:50:37.514 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T10:50:37.514 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:50:37.516 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:50:37.542 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T10:50:37.550 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T10:50:37.551 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:50:37.581 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:50:37.601 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:50:37.671 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:50:37.731 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:50:37.814 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:50:37.853 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:50:37.853 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:50:37.917 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:50:37.956 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:50:38.023 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:50:38.066 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:50:38.107 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:50:38.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:50:38.171 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-04-20T10:50:38.174 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-04-20T10:50:38.211 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:50:38.247 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:50:38.582 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T10:50:38.591 INFO:teuthology.task.internal:Starting timer... 2024-04-20T10:50:38.591 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T10:50:38.600 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T10:50:38.613 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-20T10:50:38.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-04-20T10:50:38.678 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-04-20T10:50:38.680 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-04-20T10:50:38.742 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-04-20T10:50:38.744 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T10:50:38.744 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-04-20T10:50:38.773 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-04-20T10:50:38.773 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-04-20T10:50:38.791 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-04-20T10:50:38.792 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive'} 2024-04-20T10:50:38.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:50:38.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:50:38.843 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:50:38.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:50:38.870 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T10:50:38.870 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:50:38.907 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:50:38.935 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T10:50:38.948 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T10:50:38.964 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T10:50:38.966 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi137.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T10:52:56.861 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com')] 2024-04-20T10:52:56.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-20T10:52:56.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:52:56.943 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-04-20T10:52:57.022 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-04-20T10:52:57.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-20T10:52:57.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:52:57.100 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-04-20T10:52:57.178 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-20T10:52:57.179 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T10:52:57.190 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T10:52:57.190 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:52:57.191 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:52:57.193 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:52:57.193 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:52:57.220 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:52:57.235 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:52:57.248 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:52:57.259 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-04-20T10:52:57.262 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:52:57.270 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-04-20T10:52:57.284 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:52:57.286 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-04-20T10:52:57.299 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:52:57.299 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-04-20T10:52:57.311 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:52:57.325 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:52:57.359 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:52:57.362 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:52:57.362 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-04-20T10:52:57.362 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:52:57.362 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:52:57.363 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:52:57.363 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:52:57.383 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:52:57.386 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:52:57.386 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-04-20T10:52:57.386 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:52:57.386 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:52:57.386 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:52:57.387 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:52:57.387 INFO:teuthology.run_tasks:Running task install... 2024-04-20T10:52:57.396 DEBUG:teuthology.task.install:project ceph 2024-04-20T10:52:57.396 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T10:52:57.396 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T10:52:57.396 INFO:teuthology.task.install:Using flavor: default 2024-04-20T10:52:57.402 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T10:52:57.402 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T10:52:57.403 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:52:57.403 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:52:57.406 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:52:57.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:52:57.627 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T10:52:57.627 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T10:52:57.641 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T10:52:57.641 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T10:52:57.885 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:52:57.885 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:52:57.885 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:52:57.892 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:52:57.892 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:52:57.892 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:52:57.914 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T10:52:57.914 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:52:57.917 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T10:52:57.917 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:52:57.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:52:57.981 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:52:58.054 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:52:58.056 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:52:58.117 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-04-20T10:52:58.118 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-04-20T10:52:58.121 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-04-20T10:52:58.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-04-20T10:52:58.347 INFO:teuthology.orchestra.run.smithi137.stdout:48 files removed 2024-04-20T10:52:58.358 INFO:teuthology.orchestra.run.smithi027.stdout:48 files removed 2024-04-20T10:52:58.376 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T10:52:58.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T10:52:59.061 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-04-20T10:52:59.111 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 184 kB/s | 84 kB 00:00 2024-04-20T10:52:59.645 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-20T10:52:59.679 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-20T10:53:00.012 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-20T10:53:00.104 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-20T10:53:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.0 MB 00:01 2024-04-20T10:53:03.780 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-20T10:53:08.031 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-04-20T10:53:08.862 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 945 kB/s | 8.0 MB 00:08 2024-04-20T10:53:09.440 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-04-20T10:53:10.794 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-20T10:53:10.805 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-20T10:53:15.808 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-20T10:53:16.722 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-04-20T10:53:17.975 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:53:17.976 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:53:17.987 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T10:53:17.987 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T10:53:17.988 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:53:17.989 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:53:17.991 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:53:17.993 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T10:53:17.994 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:53:18.055 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-04-20T10:53:18.067 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-20T10:53:18.067 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-04-20T10:53:18.067 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-20T10:53:18.067 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T10:53:18.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T10:53:18.069 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T10:53:18.070 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T10:53:18.071 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:53:18.072 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T10:53:18.073 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:53:18.074 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:53:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:53:18.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:53:18.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T10:53:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:53:18.079 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-04-20T10:53:18.079 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-20T10:53:18.079 INFO:teuthology.orchestra.run.smithi137.stdout:Install 148 Packages 2024-04-20T10:53:18.079 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 2 Packages 2024-04-20T10:53:18.079 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:53:18.079 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 218 M 2024-04-20T10:53:18.079 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-04-20T10:53:18.934 INFO:teuthology.orchestra.run.smithi137.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-20T10:53:19.098 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-20T10:53:19.126 INFO:teuthology.orchestra.run.smithi137.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 4.6 MB/s | 893 kB 00:00 2024-04-20T10:53:19.193 INFO:teuthology.orchestra.run.smithi137.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 2.1 MB/s | 145 kB 00:00 2024-04-20T10:53:19.518 INFO:teuthology.orchestra.run.smithi137.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 7.3 MB/s | 2.4 MB 00:00 2024-04-20T10:53:19.818 INFO:teuthology.orchestra.run.smithi137.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 5.0 MB/s | 1.5 MB 00:00 2024-04-20T10:53:20.269 INFO:teuthology.orchestra.run.smithi137.stdout:(6/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 14 MB/s | 21 MB 00:01 2024-04-20T10:53:20.402 INFO:teuthology.orchestra.run.smithi137.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 8.1 MB/s | 4.7 MB 00:00 2024-04-20T10:53:21.116 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-20T10:53:21.319 INFO:teuthology.orchestra.run.smithi137.stdout:(8/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x86 16 MB/s | 17 MB 00:01 2024-04-20T10:53:21.369 INFO:teuthology.orchestra.run.smithi137.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 504 kB/s | 25 kB 00:00 2024-04-20T10:53:21.686 INFO:teuthology.orchestra.run.smithi137.stdout:(10/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 8.3 MB/s | 11 MB 00:01 2024-04-20T10:53:21.745 INFO:teuthology.orchestra.run.smithi137.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 3.8 MB/s | 225 kB 00:00 2024-04-20T10:53:21.812 INFO:teuthology.orchestra.run.smithi137.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 407 kB/s | 27 kB 00:00 2024-04-20T10:53:21.870 INFO:teuthology.orchestra.run.smithi137.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 543 kB/s | 32 kB 00:00 2024-04-20T10:53:21.996 INFO:teuthology.orchestra.run.smithi137.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 5.8 MB/s | 748 kB 00:00 2024-04-20T10:53:22.062 INFO:teuthology.orchestra.run.smithi137.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 2.4 MB/s | 163 kB 00:00 2024-04-20T10:53:22.121 INFO:teuthology.orchestra.run.smithi137.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 2.1 MB/s | 126 kB 00:00 2024-04-20T10:53:22.221 INFO:teuthology.orchestra.run.smithi137.stdout:(17/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x 1.5 MB/s | 5.4 MB 00:03 2024-04-20T10:53:22.247 INFO:teuthology.orchestra.run.smithi137.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 3.9 MB/s | 501 kB 00:00 2024-04-20T10:53:22.305 INFO:teuthology.orchestra.run.smithi137.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 773 kB/s | 45 kB 00:00 2024-04-20T10:53:22.373 INFO:teuthology.orchestra.run.smithi137.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 2.0 MB/s | 135 kB 00:00 2024-04-20T10:53:22.439 INFO:teuthology.orchestra.run.smithi137.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 2.4 MB/s | 162 kB 00:00 2024-04-20T10:53:22.523 INFO:teuthology.orchestra.run.smithi137.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 3.8 MB/s | 321 kB 00:00 2024-04-20T10:53:22.599 INFO:teuthology.orchestra.run.smithi137.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 3.9 MB/s | 299 kB 00:00 2024-04-20T10:53:22.666 INFO:teuthology.orchestra.run.smithi137.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.5 MB/s | 99 kB 00:00 2024-04-20T10:53:22.732 INFO:teuthology.orchestra.run.smithi137.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-20T10:53:23.116 INFO:teuthology.orchestra.run.smithi137.stdout:(26/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 8.1 MB/s | 3.1 MB 00:00 2024-04-20T10:53:23.183 INFO:teuthology.orchestra.run.smithi137.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-20T10:53:23.249 INFO:teuthology.orchestra.run.smithi137.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 514 kB/s | 34 kB 00:00 2024-04-20T10:53:23.325 INFO:teuthology.orchestra.run.smithi137.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.8 MB/s | 142 kB 00:00 2024-04-20T10:53:24.067 INFO:teuthology.orchestra.run.smithi137.stdout:(30/150): ceph-test-19.0.0-3133.ga8c84f59.el9.x 18 MB/s | 49 MB 00:02 2024-04-20T10:53:24.126 INFO:teuthology.orchestra.run.smithi137.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 4.4 MB/s | 3.5 MB 00:00 2024-04-20T10:53:24.201 INFO:teuthology.orchestra.run.smithi137.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 3.3 MB/s | 249 kB 00:00 2024-04-20T10:53:24.259 INFO:teuthology.orchestra.run.smithi137.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 846 kB/s | 49 kB 00:00 2024-04-20T10:53:24.326 INFO:teuthology.orchestra.run.smithi137.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 247 kB/s | 16 kB 00:00 2024-04-20T10:53:24.402 INFO:teuthology.orchestra.run.smithi137.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 3.6 MB/s | 273 kB 00:00 2024-04-20T10:53:24.511 INFO:teuthology.orchestra.run.smithi137.stdout:(36/150): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-04-20T10:53:24.552 INFO:teuthology.orchestra.run.smithi137.stdout:(37/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 5.0 MB/s | 763 kB 00:00 2024-04-20T10:53:24.579 INFO:teuthology.orchestra.run.smithi137.stdout:(38/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 274 kB/s | 18 kB 00:00 2024-04-20T10:53:24.736 INFO:teuthology.orchestra.run.smithi137.stdout:(39/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 2.1 MB/s | 5.4 MB 00:02 2024-04-20T10:53:24.762 INFO:teuthology.orchestra.run.smithi137.stdout:(40/150): fuse-2.9.9-15.el9.x86_64.rpm 385 kB/s | 80 kB 00:00 2024-04-20T10:53:24.828 INFO:teuthology.orchestra.run.smithi137.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 3.1 MB/s | 803 kB 00:00 2024-04-20T10:53:24.854 INFO:teuthology.orchestra.run.smithi137.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-20T10:53:24.904 INFO:teuthology.orchestra.run.smithi137.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-20T10:53:24.929 INFO:teuthology.orchestra.run.smithi137.stdout:(44/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-20T10:53:24.954 INFO:teuthology.orchestra.run.smithi137.stdout:(45/150): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-20T10:53:24.979 INFO:teuthology.orchestra.run.smithi137.stdout:(46/150): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-20T10:53:25.005 INFO:teuthology.orchestra.run.smithi137.stdout:(47/150): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-20T10:53:25.022 INFO:teuthology.orchestra.run.smithi137.stdout:(48/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-20T10:53:25.147 INFO:teuthology.orchestra.run.smithi137.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 855 kB/s | 107 kB 00:00 2024-04-20T10:53:25.198 INFO:teuthology.orchestra.run.smithi137.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-04-20T10:53:25.223 INFO:teuthology.orchestra.run.smithi137.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-04-20T10:53:25.248 INFO:teuthology.orchestra.run.smithi137.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-20T10:53:25.265 INFO:teuthology.orchestra.run.smithi137.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 782 kB/s | 33 kB 00:00 2024-04-20T10:53:25.299 INFO:teuthology.orchestra.run.smithi137.stdout:(54/150): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-04-20T10:53:25.316 INFO:teuthology.orchestra.run.smithi137.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-04-20T10:53:25.349 INFO:teuthology.orchestra.run.smithi137.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-20T10:53:25.408 INFO:teuthology.orchestra.run.smithi137.stdout:(57/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-20T10:53:25.433 INFO:teuthology.orchestra.run.smithi137.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-20T10:53:25.450 INFO:teuthology.orchestra.run.smithi137.stdout:(59/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-20T10:53:25.483 INFO:teuthology.orchestra.run.smithi137.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-20T10:53:25.509 INFO:teuthology.orchestra.run.smithi137.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-04-20T10:53:25.534 INFO:teuthology.orchestra.run.smithi137.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-04-20T10:53:25.559 INFO:teuthology.orchestra.run.smithi137.stdout:(63/150): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-20T10:53:25.585 INFO:teuthology.orchestra.run.smithi137.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-20T10:53:25.610 INFO:teuthology.orchestra.run.smithi137.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-20T10:53:25.644 INFO:teuthology.orchestra.run.smithi137.stdout:(66/150): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-20T10:53:26.027 INFO:teuthology.orchestra.run.smithi137.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-04-20T10:53:26.110 INFO:teuthology.orchestra.run.smithi137.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-20T10:53:26.178 INFO:teuthology.orchestra.run.smithi137.stdout:(69/150): openblas-openmp-0.3.21-2.el9.x86_64.r 8.0 MB/s | 4.7 MB 00:00 2024-04-20T10:53:26.204 INFO:teuthology.orchestra.run.smithi137.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-20T10:53:26.261 INFO:teuthology.orchestra.run.smithi137.stdout:(71/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 6.3 MB/s | 4.1 MB 00:00 2024-04-20T10:53:26.295 INFO:teuthology.orchestra.run.smithi137.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-20T10:53:26.299 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-20T10:53:26.320 INFO:teuthology.orchestra.run.smithi137.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-20T10:53:26.404 INFO:teuthology.orchestra.run.smithi137.stdout:(74/150): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-04-20T10:53:26.429 INFO:teuthology.orchestra.run.smithi137.stdout:(75/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-20T10:53:26.463 INFO:teuthology.orchestra.run.smithi137.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-20T10:53:26.488 INFO:teuthology.orchestra.run.smithi137.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-04-20T10:53:26.546 INFO:teuthology.orchestra.run.smithi137.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-04-20T10:53:26.630 INFO:teuthology.orchestra.run.smithi137.stdout:(79/150): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-20T10:53:26.655 INFO:teuthology.orchestra.run.smithi137.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-04-20T10:53:26.681 INFO:teuthology.orchestra.run.smithi137.stdout:(81/150): python3-mako-1.1.4-6.el9.noarch.rpm 895 kB/s | 172 kB 00:00 2024-04-20T10:53:26.706 INFO:teuthology.orchestra.run.smithi137.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-20T10:53:26.731 INFO:teuthology.orchestra.run.smithi137.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-20T10:53:26.757 INFO:teuthology.orchestra.run.smithi137.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-20T10:53:26.782 INFO:teuthology.orchestra.run.smithi137.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-04-20T10:53:26.807 INFO:teuthology.orchestra.run.smithi137.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-20T10:53:26.833 INFO:teuthology.orchestra.run.smithi137.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-20T10:53:26.858 INFO:teuthology.orchestra.run.smithi137.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-20T10:53:26.883 INFO:teuthology.orchestra.run.smithi137.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-20T10:53:26.950 INFO:teuthology.orchestra.run.smithi137.stdout:(90/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-20T10:53:26.967 INFO:teuthology.orchestra.run.smithi137.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-04-20T10:53:26.992 INFO:teuthology.orchestra.run.smithi137.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-20T10:53:27.026 INFO:teuthology.orchestra.run.smithi137.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-04-20T10:53:27.051 INFO:teuthology.orchestra.run.smithi137.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-20T10:53:27.076 INFO:teuthology.orchestra.run.smithi137.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-04-20T10:53:27.118 INFO:teuthology.orchestra.run.smithi137.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-04-20T10:53:27.170 INFO:teuthology.orchestra.run.smithi137.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-20T10:53:27.185 INFO:teuthology.orchestra.run.smithi137.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 867 kB/s | 58 kB 00:00 2024-04-20T10:53:27.361 INFO:teuthology.orchestra.run.smithi137.stdout:(99/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-20T10:53:27.378 INFO:teuthology.orchestra.run.smithi137.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-04-20T10:53:27.403 INFO:teuthology.orchestra.run.smithi137.stdout:(101/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 400 kB/s | 94 kB 00:00 2024-04-20T10:53:27.420 INFO:teuthology.orchestra.run.smithi137.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-04-20T10:53:27.437 INFO:teuthology.orchestra.run.smithi137.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 378 kB/s | 22 kB 00:00 2024-04-20T10:53:27.579 INFO:teuthology.orchestra.run.smithi137.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-20T10:53:27.821 INFO:teuthology.orchestra.run.smithi137.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2024-04-20T10:53:27.871 INFO:teuthology.orchestra.run.smithi137.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 712 kB/s | 309 kB 00:00 2024-04-20T10:53:27.921 INFO:teuthology.orchestra.run.smithi137.stdout:(107/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-20T10:53:27.946 INFO:teuthology.orchestra.run.smithi137.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-04-20T10:53:27.997 INFO:teuthology.orchestra.run.smithi137.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-04-20T10:53:28.013 INFO:teuthology.orchestra.run.smithi137.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-20T10:53:28.080 INFO:teuthology.orchestra.run.smithi137.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-20T10:53:28.198 INFO:teuthology.orchestra.run.smithi137.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-04-20T10:53:28.223 INFO:teuthology.orchestra.run.smithi137.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-04-20T10:53:28.273 INFO:teuthology.orchestra.run.smithi137.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 578 kB/s | 43 kB 00:00 2024-04-20T10:53:28.298 INFO:teuthology.orchestra.run.smithi137.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-20T10:53:28.365 INFO:teuthology.orchestra.run.smithi137.stdout:(116/150): libarrow-9.0.0-10.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-20T10:53:28.382 INFO:teuthology.orchestra.run.smithi137.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-20T10:53:28.408 INFO:teuthology.orchestra.run.smithi137.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-20T10:53:28.450 INFO:teuthology.orchestra.run.smithi137.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-20T10:53:28.475 INFO:teuthology.orchestra.run.smithi137.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-20T10:53:28.492 INFO:teuthology.orchestra.run.smithi137.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-04-20T10:53:28.525 INFO:teuthology.orchestra.run.smithi137.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-20T10:53:28.584 INFO:teuthology.orchestra.run.smithi137.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-20T10:53:28.585 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:53:28.586 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:53:28.598 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T10:53:28.598 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T10:53:28.599 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:53:28.600 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:53:28.602 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:53:28.606 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T10:53:28.606 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:53:28.609 INFO:teuthology.orchestra.run.smithi137.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-20T10:53:28.634 INFO:teuthology.orchestra.run.smithi137.stdout:(125/150): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-20T10:53:28.660 INFO:teuthology.orchestra.run.smithi137.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-20T10:53:28.672 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-04-20T10:53:28.681 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-04-20T10:53:28.681 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-04-20T10:53:28.681 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T10:53:28.682 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T10:53:28.683 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T10:53:28.684 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T10:53:28.685 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:53:28.686 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:53:28.686 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:53:28.686 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:53:28.686 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:53:28.686 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T10:53:28.686 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:53:28.687 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T10:53:28.688 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:53:28.689 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:53:28.690 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:53:28.691 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:53:28.692 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-04-20T10:53:28.693 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-04-20T10:53:28.694 INFO:teuthology.orchestra.run.smithi027.stdout:Install 148 Packages 2024-04-20T10:53:28.694 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 2 Packages 2024-04-20T10:53:28.694 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:53:28.694 INFO:teuthology.orchestra.run.smithi137.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-20T10:53:28.695 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 218 M 2024-04-20T10:53:28.695 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-04-20T10:53:28.710 INFO:teuthology.orchestra.run.smithi137.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-20T10:53:28.735 INFO:teuthology.orchestra.run.smithi137.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-04-20T10:53:28.769 INFO:teuthology.orchestra.run.smithi137.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-20T10:53:28.794 INFO:teuthology.orchestra.run.smithi137.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2024-04-20T10:53:28.820 INFO:teuthology.orchestra.run.smithi137.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 939 kB/s | 79 kB 00:00 2024-04-20T10:53:28.845 INFO:teuthology.orchestra.run.smithi137.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-04-20T10:53:28.887 INFO:teuthology.orchestra.run.smithi137.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-20T10:53:28.912 INFO:teuthology.orchestra.run.smithi137.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-20T10:53:28.937 INFO:teuthology.orchestra.run.smithi137.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-04-20T10:53:28.963 INFO:teuthology.orchestra.run.smithi137.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-20T10:53:28.988 INFO:teuthology.orchestra.run.smithi137.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-20T10:53:29.013 INFO:teuthology.orchestra.run.smithi137.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-20T10:53:29.038 INFO:teuthology.orchestra.run.smithi137.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-04-20T10:53:29.064 INFO:teuthology.orchestra.run.smithi137.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-20T10:53:29.089 INFO:teuthology.orchestra.run.smithi137.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-20T10:53:29.123 INFO:teuthology.orchestra.run.smithi137.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-20T10:53:29.148 INFO:teuthology.orchestra.run.smithi137.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-20T10:53:29.173 INFO:teuthology.orchestra.run.smithi137.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-20T10:53:29.198 INFO:teuthology.orchestra.run.smithi137.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-20T10:53:29.224 INFO:teuthology.orchestra.run.smithi137.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-20T10:53:29.266 INFO:teuthology.orchestra.run.smithi137.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-20T10:53:29.326 INFO:teuthology.orchestra.run.smithi027.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-04-20T10:53:29.524 INFO:teuthology.orchestra.run.smithi137.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 10 MB/s | 3.4 MB 00:00 2024-04-20T10:53:29.643 INFO:teuthology.orchestra.run.smithi027.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 2.8 MB/s | 893 kB 00:00 2024-04-20T10:53:29.710 INFO:teuthology.orchestra.run.smithi027.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 2.1 MB/s | 145 kB 00:00 2024-04-20T10:53:30.085 INFO:teuthology.orchestra.run.smithi027.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 5.7 MB/s | 5.4 MB 00:00 2024-04-20T10:53:30.241 INFO:teuthology.orchestra.run.smithi137.stdout:(150/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-04-20T10:53:30.248 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:30.248 INFO:teuthology.orchestra.run.smithi137.stdout:Total 18 MB/s | 218 MB 00:12 2024-04-20T10:53:30.248 INFO:teuthology.orchestra.run.smithi027.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 4.4 MB/s | 2.4 MB 00:00 2024-04-20T10:53:30.303 INFO:teuthology.orchestra.run.smithi027.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-04-20T10:53:31.028 INFO:teuthology.orchestra.run.smithi027.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 6.0 MB/s | 4.7 MB 00:00 2024-04-20T10:53:31.303 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-04-20T10:53:31.430 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-04-20T10:53:31.430 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-04-20T10:53:31.561 INFO:teuthology.orchestra.run.smithi027.stdout:(8/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 8.7 MB/s | 21 MB 00:02 2024-04-20T10:53:31.628 INFO:teuthology.orchestra.run.smithi027.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 379 kB/s | 25 kB 00:00 2024-04-20T10:53:31.970 INFO:teuthology.orchestra.run.smithi027.stdout:(10/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 10 MB/s | 17 MB 00:01 2024-04-20T10:53:32.029 INFO:teuthology.orchestra.run.smithi027.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 3.8 MB/s | 225 kB 00:00 2024-04-20T10:53:32.096 INFO:teuthology.orchestra.run.smithi027.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 405 kB/s | 27 kB 00:00 2024-04-20T10:53:32.162 INFO:teuthology.orchestra.run.smithi027.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 477 kB/s | 32 kB 00:00 2024-04-20T10:53:32.288 INFO:teuthology.orchestra.run.smithi027.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 5.8 MB/s | 748 kB 00:00 2024-04-20T10:53:32.405 INFO:teuthology.orchestra.run.smithi027.stdout:(15/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 7.7 MB/s | 11 MB 00:01 2024-04-20T10:53:32.430 INFO:teuthology.orchestra.run.smithi027.stdout:(16/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.1 MB/s | 163 kB 00:00 2024-04-20T10:53:32.464 INFO:teuthology.orchestra.run.smithi027.stdout:(17/150): librados-devel-19.0.0-3133.ga8c84f59. 2.1 MB/s | 126 kB 00:00 2024-04-20T10:53:32.505 INFO:teuthology.orchestra.run.smithi027.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 6.5 MB/s | 501 kB 00:00 2024-04-20T10:53:32.572 INFO:teuthology.orchestra.run.smithi027.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 678 kB/s | 45 kB 00:00 2024-04-20T10:53:32.639 INFO:teuthology.orchestra.run.smithi027.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 2.0 MB/s | 135 kB 00:00 2024-04-20T10:53:32.706 INFO:teuthology.orchestra.run.smithi027.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 2.4 MB/s | 162 kB 00:00 2024-04-20T10:53:32.781 INFO:teuthology.orchestra.run.smithi027.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 4.2 MB/s | 321 kB 00:00 2024-04-20T10:53:32.865 INFO:teuthology.orchestra.run.smithi027.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 3.5 MB/s | 299 kB 00:00 2024-04-20T10:53:32.932 INFO:teuthology.orchestra.run.smithi027.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.5 MB/s | 99 kB 00:00 2024-04-20T10:53:32.991 INFO:teuthology.orchestra.run.smithi027.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-20T10:53:33.066 INFO:teuthology.orchestra.run.smithi027.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 8.9 MB/s | 5.4 MB 00:00 2024-04-20T10:53:33.142 INFO:teuthology.orchestra.run.smithi027.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 2.2 MB/s | 170 kB 00:00 2024-04-20T10:53:33.209 INFO:teuthology.orchestra.run.smithi027.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 514 kB/s | 34 kB 00:00 2024-04-20T10:53:33.249 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-04-20T10:53:33.252 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-04-20T10:53:33.275 INFO:teuthology.orchestra.run.smithi027.stdout:(29/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 11 MB/s | 3.1 MB 00:00 2024-04-20T10:53:33.293 INFO:teuthology.orchestra.run.smithi027.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.7 MB/s | 142 kB 00:00 2024-04-20T10:53:33.651 INFO:teuthology.orchestra.run.smithi027.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 9.4 MB/s | 3.5 MB 00:00 2024-04-20T10:53:33.718 INFO:teuthology.orchestra.run.smithi027.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 3.7 MB/s | 249 kB 00:00 2024-04-20T10:53:33.787 INFO:teuthology.orchestra.run.smithi027.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 717 kB/s | 49 kB 00:00 2024-04-20T10:53:33.843 INFO:teuthology.orchestra.run.smithi027.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 293 kB/s | 16 kB 00:00 2024-04-20T10:53:33.919 INFO:teuthology.orchestra.run.smithi027.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 3.6 MB/s | 273 kB 00:00 2024-04-20T10:53:34.061 INFO:teuthology.orchestra.run.smithi027.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 5.3 MB/s | 763 kB 00:00 2024-04-20T10:53:34.122 INFO:teuthology.orchestra.run.smithi027.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 313 kB/s | 18 kB 00:00 2024-04-20T10:53:34.212 INFO:teuthology.orchestra.run.smithi027.stdout:(38/150): ceph-mgr-diskprediction-local-19.0.0- 8.1 MB/s | 7.4 MB 00:00 2024-04-20T10:53:34.312 INFO:teuthology.orchestra.run.smithi027.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 418 kB/s | 80 kB 00:00 2024-04-20T10:53:34.387 INFO:teuthology.orchestra.run.smithi027.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-20T10:53:34.412 INFO:teuthology.orchestra.run.smithi027.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-20T10:53:34.437 INFO:teuthology.orchestra.run.smithi027.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-20T10:53:34.488 INFO:teuthology.orchestra.run.smithi027.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-20T10:53:34.521 INFO:teuthology.orchestra.run.smithi027.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-20T10:53:34.547 INFO:teuthology.orchestra.run.smithi027.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-20T10:53:34.589 INFO:teuthology.orchestra.run.smithi027.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-20T10:53:34.614 INFO:teuthology.orchestra.run.smithi027.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-20T10:53:34.864 INFO:teuthology.orchestra.run.smithi027.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-20T10:53:34.898 INFO:teuthology.orchestra.run.smithi027.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-04-20T10:53:35.373 INFO:teuthology.orchestra.run.smithi027.stdout:(50/150): ceph-test-19.0.0-3133.ga8c84f59.el9.x 13 MB/s | 49 MB 00:03 2024-04-20T10:53:35.399 INFO:teuthology.orchestra.run.smithi027.stdout:(51/150): boost-program-options-1.75.0-8.el9.x8 200 kB/s | 107 kB 00:00 2024-04-20T10:53:35.424 INFO:teuthology.orchestra.run.smithi027.stdout:(52/150): elfutils-libelf-devel-0.190-2.el9.x86 41 kB/s | 21 kB 00:00 2024-04-20T10:53:35.466 INFO:teuthology.orchestra.run.smithi027.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-04-20T10:53:35.491 INFO:teuthology.orchestra.run.smithi027.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-20T10:53:35.533 INFO:teuthology.orchestra.run.smithi027.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-04-20T10:53:35.625 INFO:teuthology.orchestra.run.smithi027.stdout:(56/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-20T10:53:35.650 INFO:teuthology.orchestra.run.smithi027.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.8 MB/s | 160 kB 00:00 2024-04-20T10:53:35.684 INFO:teuthology.orchestra.run.smithi027.stdout:(58/150): libaio-devel-0.3.111-13.el9.x86_64.rp 58 kB/s | 11 kB 00:00 2024-04-20T10:53:35.709 INFO:teuthology.orchestra.run.smithi027.stdout:(59/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 548 kB/s | 45 kB 00:00 2024-04-20T10:53:35.751 INFO:teuthology.orchestra.run.smithi027.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-04-20T10:53:35.793 INFO:teuthology.orchestra.run.smithi027.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-04-20T10:53:35.818 INFO:teuthology.orchestra.run.smithi027.stdout:(62/150): libtool-2.4.6-45.el9.x86_64.rpm 4.2 MB/s | 582 kB 00:00 2024-04-20T10:53:35.843 INFO:teuthology.orchestra.run.smithi027.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-04-20T10:53:35.869 INFO:teuthology.orchestra.run.smithi027.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-20T10:53:35.894 INFO:teuthology.orchestra.run.smithi027.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-20T10:53:35.919 INFO:teuthology.orchestra.run.smithi027.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-04-20T10:53:36.086 INFO:teuthology.orchestra.run.smithi027.stdout:(67/150): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-04-20T10:53:36.170 INFO:teuthology.orchestra.run.smithi027.stdout:(68/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 15 MB/s | 4.1 MB 00:00 2024-04-20T10:53:36.220 INFO:teuthology.orchestra.run.smithi027.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-20T10:53:36.262 INFO:teuthology.orchestra.run.smithi027.stdout:(70/150): postgresql-13.14-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-20T10:53:36.287 INFO:teuthology.orchestra.run.smithi027.stdout:(71/150): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-20T10:53:36.363 INFO:teuthology.orchestra.run.smithi027.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-20T10:53:36.388 INFO:teuthology.orchestra.run.smithi027.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-04-20T10:53:36.471 INFO:teuthology.orchestra.run.smithi027.stdout:(74/150): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-20T10:53:36.530 INFO:teuthology.orchestra.run.smithi027.stdout:(75/150): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-04-20T10:53:36.556 INFO:teuthology.orchestra.run.smithi027.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-20T10:53:36.589 INFO:teuthology.orchestra.run.smithi027.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-04-20T10:53:36.664 INFO:teuthology.orchestra.run.smithi027.stdout:(78/150): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-04-20T10:53:36.681 INFO:teuthology.orchestra.run.smithi027.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-04-20T10:53:36.715 INFO:teuthology.orchestra.run.smithi027.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-20T10:53:36.732 INFO:teuthology.orchestra.run.smithi027.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-04-20T10:53:36.782 INFO:teuthology.orchestra.run.smithi027.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-20T10:53:36.808 INFO:teuthology.orchestra.run.smithi027.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-20T10:53:36.849 INFO:teuthology.orchestra.run.smithi027.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-20T10:53:36.867 INFO:teuthology.orchestra.run.smithi027.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 716 kB/s | 42 kB 00:00 2024-04-20T10:53:36.933 INFO:teuthology.orchestra.run.smithi027.stdout:(86/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-20T10:53:36.958 INFO:teuthology.orchestra.run.smithi027.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-04-20T10:53:36.986 INFO:teuthology.orchestra.run.smithi027.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-20T10:53:37.009 INFO:teuthology.orchestra.run.smithi027.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-20T10:53:37.034 INFO:teuthology.orchestra.run.smithi027.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-04-20T10:53:37.059 INFO:teuthology.orchestra.run.smithi027.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-04-20T10:53:37.102 INFO:teuthology.orchestra.run.smithi027.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-04-20T10:53:37.152 INFO:teuthology.orchestra.run.smithi027.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 480 kB/s | 44 kB 00:00 2024-04-20T10:53:37.211 INFO:teuthology.orchestra.run.smithi027.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-20T10:53:37.244 INFO:teuthology.orchestra.run.smithi027.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 391 kB/s | 36 kB 00:00 2024-04-20T10:53:37.286 INFO:teuthology.orchestra.run.smithi027.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-04-20T10:53:37.336 INFO:teuthology.orchestra.run.smithi027.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-20T10:53:37.361 INFO:teuthology.orchestra.run.smithi027.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 769 kB/s | 58 kB 00:00 2024-04-20T10:53:37.399 INFO:teuthology.orchestra.run.smithi027.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-20T10:53:37.421 INFO:teuthology.orchestra.run.smithi027.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-20T10:53:37.588 INFO:teuthology.orchestra.run.smithi027.stdout:(101/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-20T10:53:37.613 INFO:teuthology.orchestra.run.smithi027.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 206 kB/s | 45 kB 00:00 2024-04-20T10:53:37.638 INFO:teuthology.orchestra.run.smithi027.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-04-20T10:53:37.880 INFO:teuthology.orchestra.run.smithi027.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-04-20T10:53:38.122 INFO:teuthology.orchestra.run.smithi027.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-04-20T10:53:38.147 INFO:teuthology.orchestra.run.smithi027.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 607 kB/s | 309 kB 00:00 2024-04-20T10:53:38.198 INFO:teuthology.orchestra.run.smithi027.stdout:(107/150): abseil-cpp-20211102.0-3.el9.x86_64.r 938 kB/s | 548 kB 00:00 2024-04-20T10:53:38.239 INFO:teuthology.orchestra.run.smithi027.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-20T10:53:38.298 INFO:teuthology.orchestra.run.smithi027.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-04-20T10:53:38.406 INFO:teuthology.orchestra.run.smithi027.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-20T10:53:38.423 INFO:teuthology.orchestra.run.smithi027.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 366 kB/s | 67 kB 00:00 2024-04-20T10:53:38.582 INFO:teuthology.orchestra.run.smithi027.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-04-20T10:53:38.607 INFO:teuthology.orchestra.run.smithi027.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-04-20T10:53:38.674 INFO:teuthology.orchestra.run.smithi027.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 471 kB/s | 43 kB 00:00 2024-04-20T10:53:38.700 INFO:teuthology.orchestra.run.smithi027.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 349 kB/s | 32 kB 00:00 2024-04-20T10:53:38.766 INFO:teuthology.orchestra.run.smithi027.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-20T10:53:38.800 INFO:teuthology.orchestra.run.smithi027.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-20T10:53:38.876 INFO:teuthology.orchestra.run.smithi027.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-20T10:53:38.942 INFO:teuthology.orchestra.run.smithi027.stdout:(119/150): libarrow-9.0.0-10.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2024-04-20T10:53:38.968 INFO:teuthology.orchestra.run.smithi027.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-04-20T10:53:38.985 INFO:teuthology.orchestra.run.smithi027.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-04-20T10:53:39.035 INFO:teuthology.orchestra.run.smithi027.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 2.1 MB/s | 203 kB 00:00 2024-04-20T10:53:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-04-20T10:53:39.085 INFO:teuthology.orchestra.run.smithi027.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-20T10:53:39.119 INFO:teuthology.orchestra.run.smithi027.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-20T10:53:39.151 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T10:53:39.169 INFO:teuthology.orchestra.run.smithi027.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-04-20T10:53:39.220 INFO:teuthology.orchestra.run.smithi027.stdout:(126/150): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-04-20T10:53:39.236 INFO:teuthology.orchestra.run.smithi027.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-04-20T10:53:39.252 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T10:53:39.253 INFO:teuthology.orchestra.run.smithi027.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-20T10:53:39.303 INFO:teuthology.orchestra.run.smithi027.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-20T10:53:39.354 INFO:teuthology.orchestra.run.smithi027.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-20T10:53:39.370 INFO:teuthology.orchestra.run.smithi027.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 395 kB/s | 46 kB 00:00 2024-04-20T10:53:39.396 INFO:teuthology.orchestra.run.smithi027.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 854 kB/s | 79 kB 00:00 2024-04-20T10:53:39.430 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T10:53:39.446 INFO:teuthology.orchestra.run.smithi027.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-04-20T10:53:39.472 INFO:teuthology.orchestra.run.smithi027.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-20T10:53:39.497 INFO:teuthology.orchestra.run.smithi027.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-20T10:53:39.539 INFO:teuthology.orchestra.run.smithi027.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 975 kB/s | 90 kB 00:00 2024-04-20T10:53:39.564 INFO:teuthology.orchestra.run.smithi027.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-20T10:53:39.589 INFO:teuthology.orchestra.run.smithi027.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-04-20T10:53:39.631 INFO:teuthology.orchestra.run.smithi027.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-20T10:53:39.658 INFO:teuthology.orchestra.run.smithi027.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-04-20T10:53:39.682 INFO:teuthology.orchestra.run.smithi027.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-04-20T10:53:39.724 INFO:teuthology.orchestra.run.smithi027.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-04-20T10:53:39.757 INFO:teuthology.orchestra.run.smithi027.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-20T10:53:39.770 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T10:53:39.782 INFO:teuthology.orchestra.run.smithi027.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-04-20T10:53:39.833 INFO:teuthology.orchestra.run.smithi027.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-20T10:53:39.858 INFO:teuthology.orchestra.run.smithi027.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 198 kB/s | 20 kB 00:00 2024-04-20T10:53:39.884 INFO:teuthology.orchestra.run.smithi027.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-20T10:53:39.886 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:53:39.992 INFO:teuthology.orchestra.run.smithi027.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-20T10:53:40.007 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:53:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:53:40.130 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:53:40.167 INFO:teuthology.orchestra.run.smithi027.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 11 MB/s | 3.4 MB 00:00 2024-04-20T10:53:40.171 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:53:40.200 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T10:53:40.225 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T10:53:40.389 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T10:53:40.439 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T10:53:40.467 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:53:40.518 INFO:teuthology.orchestra.run.smithi027.stdout:(150/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 5.0 MB/s | 3.1 MB 00:00 2024-04-20T10:53:40.523 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:53:40.523 INFO:teuthology.orchestra.run.smithi027.stdout:Total 18 MB/s | 218 MB 00:11 2024-04-20T10:53:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:53:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:53:40.639 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:53:40.661 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:53:40.679 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:53:40.987 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:53:41.019 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:53:41.129 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T10:53:41.226 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T10:53:41.269 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T10:53:41.305 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T10:53:41.379 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T10:53:41.420 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T10:53:41.464 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T10:53:41.496 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T10:53:41.534 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T10:53:41.592 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-04-20T10:53:41.612 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T10:53:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T10:53:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T10:53:41.731 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-04-20T10:53:41.732 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-04-20T10:53:41.746 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T10:53:41.824 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T10:53:41.856 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T10:53:41.902 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T10:53:41.928 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T10:53:42.001 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T10:53:42.043 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T10:53:42.119 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T10:53:42.257 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T10:53:42.313 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T10:53:42.853 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T10:53:42.941 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T10:53:42.970 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T10:53:43.012 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T10:53:43.059 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T10:53:43.092 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T10:53:43.147 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T10:53:43.220 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T10:53:43.251 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T10:53:43.288 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T10:53:43.335 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T10:53:43.370 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T10:53:43.415 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T10:53:43.458 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T10:53:43.566 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-04-20T10:53:43.567 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-04-20T10:53:43.596 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T10:53:43.646 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T10:53:43.677 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T10:53:43.718 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T10:53:43.813 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T10:53:43.843 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T10:53:43.883 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T10:53:43.909 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T10:53:43.933 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T10:53:44.008 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T10:53:44.137 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T10:53:44.174 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T10:53:44.202 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T10:53:44.236 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T10:53:44.280 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T10:53:44.318 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T10:53:44.350 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T10:53:44.379 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T10:53:44.406 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T10:53:44.514 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T10:53:44.631 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T10:53:44.666 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T10:53:44.753 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T10:53:44.804 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T10:53:44.835 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T10:53:44.882 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T10:53:44.982 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T10:53:45.036 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T10:53:45.781 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T10:53:45.825 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T10:53:45.859 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T10:53:45.958 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T10:53:46.017 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T10:53:46.104 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T10:53:46.211 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T10:53:46.387 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T10:53:46.417 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T10:53:46.494 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T10:53:47.147 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T10:53:47.265 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T10:53:48.543 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T10:53:48.629 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T10:53:48.657 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T10:53:48.690 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T10:53:49.001 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T10:53:49.045 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:53:49.135 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:53:49.166 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T10:53:49.202 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T10:53:49.442 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-04-20T10:53:49.630 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T10:53:49.681 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T10:53:49.705 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:53:49.777 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:53:49.788 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T10:53:49.807 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T10:53:49.833 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T10:53:50.181 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T10:53:50.288 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:53:50.396 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:53:50.430 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:53:50.523 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:53:50.566 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:53:50.593 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T10:53:50.619 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T10:53:50.784 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T10:53:50.834 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T10:53:50.861 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:53:50.967 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:53:50.996 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:53:51.059 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:53:51.080 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:53:51.104 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:53:51.408 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:53:51.439 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:53:51.552 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T10:53:51.649 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T10:53:51.687 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T10:53:51.723 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T10:53:51.801 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T10:53:51.839 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T10:53:51.882 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T10:53:51.914 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T10:53:51.942 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T10:53:52.016 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T10:53:52.060 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:53:52.075 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T10:53:52.095 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:53:52.113 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T10:53:52.149 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T10:53:52.229 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T10:53:52.260 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T10:53:52.304 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T10:53:52.332 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T10:53:52.409 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T10:53:52.455 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T10:53:52.534 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T10:53:52.644 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:53:52.687 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T10:53:52.732 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:53:52.744 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T10:53:52.823 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:53:52.823 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:53:52.823 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:53:52.823 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:53:52.850 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:53:53.335 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T10:53:53.433 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T10:53:53.466 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T10:53:53.509 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T10:53:53.547 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T10:53:53.580 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T10:53:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T10:53:53.713 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T10:53:53.756 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T10:53:53.792 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T10:53:53.850 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T10:53:53.883 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T10:53:53.928 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T10:53:53.978 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T10:53:54.126 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T10:53:54.166 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T10:53:54.196 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T10:53:54.238 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T10:53:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T10:53:54.378 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T10:53:54.419 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T10:53:54.444 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T10:53:54.469 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T10:53:54.549 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T10:53:54.688 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T10:53:54.726 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T10:53:54.754 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T10:53:54.889 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T10:53:54.985 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T10:53:55.030 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T10:53:55.070 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T10:53:55.099 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T10:53:55.134 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T10:53:55.167 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T10:53:55.209 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T10:53:55.251 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T10:53:55.344 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T10:53:55.391 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T10:53:55.429 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T10:53:55.478 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T10:53:55.591 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T10:53:55.649 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T10:53:56.454 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T10:53:56.545 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T10:53:56.578 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T10:53:56.686 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T10:53:56.747 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T10:53:56.847 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T10:53:56.880 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T10:53:57.065 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T10:53:57.102 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T10:53:57.183 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T10:53:57.907 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T10:53:58.032 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T10:53:59.390 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T10:53:59.482 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T10:53:59.511 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T10:53:59.543 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T10:53:59.873 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T10:54:00.007 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:54:00.115 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:54:00.144 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T10:54:00.181 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T10:54:00.688 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T10:54:00.717 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:54:00.789 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:54:00.810 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T10:54:03.240 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:54:03.273 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:54:03.849 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:54:03.885 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:54:03.969 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:54:03.970 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:54:03.970 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:54:03.970 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:04.003 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:54:04.269 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:54:04.269 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-04-20T10:54:04.269 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-04-20T10:54:04.269 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-04-20T10:54:04.269 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-04-20T10:54:04.269 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-04-20T10:54:04.269 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-04-20T10:54:04.270 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-04-20T10:54:04.270 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:04.453 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:54:04.468 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:54:04.468 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:04.468 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T10:54:04.468 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:05.947 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:54:05.973 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:54:05.973 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:05.974 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:54:05.974 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:54:05.974 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:54:05.974 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:06.198 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:54:06.220 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:54:06.220 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:06.221 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:54:06.221 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:54:06.221 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:54:06.221 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:06.666 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:54:06.687 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:54:06.688 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:06.688 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:54:06.688 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:54:06.688 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:54:06.688 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:06.813 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T10:54:06.869 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T10:54:06.924 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T10:54:07.110 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T10:54:07.157 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T10:54:07.235 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T10:54:07.324 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T10:54:07.463 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T10:54:07.499 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T10:54:07.540 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T10:54:07.567 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T10:54:08.044 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:54:08.066 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:54:08.090 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T10:54:08.151 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T10:54:08.181 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:54:09.222 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:54:09.258 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:54:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:54:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:54:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:54:10.512 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T10:54:10.535 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:54:10.558 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:54:10.558 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:10.558 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:54:10.558 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:54:10.558 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:54:10.558 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:10.602 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:54:10.613 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:54:10.703 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T10:54:10.751 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T10:54:11.675 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T10:54:11.705 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:54:11.727 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:54:11.727 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:11.727 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:54:11.727 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:54:11.727 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:54:11.727 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:11.772 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:54:11.793 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:54:11.793 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:11.794 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:54:11.794 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:11.839 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:54:11.861 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:54:11.861 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:11.861 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T10:54:11.862 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:54:11.862 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:54:11.862 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:12.173 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:54:12.196 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:54:12.196 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:12.196 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:54:12.196 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:54:12.196 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:54:12.196 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:16.183 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:54:16.183 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-04-20T10:54:16.183 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-04-20T10:54:16.183 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-04-20T10:54:16.183 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-04-20T10:54:16.184 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-04-20T10:54:16.184 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-04-20T10:54:16.184 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-04-20T10:54:16.184 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:54:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:54:16.393 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:16.393 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T10:54:16.393 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:16.802 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T10:54:16.843 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T10:54:16.883 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T10:54:16.916 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T10:54:16.943 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T10:54:17.122 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T10:54:17.164 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T10:54:17.213 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T10:54:17.816 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:54:17.859 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:54:17.878 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:54:17.927 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:54:17.954 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:54:17.954 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:17.954 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:54:17.954 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:54:17.954 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:54:17.954 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:18.194 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:54:18.218 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:54:18.218 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:18.218 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:54:18.218 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:54:18.218 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:54:18.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:18.655 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T10:54:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:54:18.710 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:54:18.710 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:18.710 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:54:18.710 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:54:18.710 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:54:18.710 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:18.738 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T10:54:18.778 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T10:54:18.818 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T10:54:18.859 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T10:54:18.876 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T10:54:18.892 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T10:54:18.892 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:54:18.948 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T10:54:18.976 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:54:18.976 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:54:19.142 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T10:54:19.188 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T10:54:19.269 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T10:54:19.358 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T10:54:19.509 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T10:54:19.555 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T10:54:19.587 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T10:54:19.614 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T10:54:20.034 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:54:20.063 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:54:20.095 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T10:54:20.159 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T10:54:20.185 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:54:21.288 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:54:21.320 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:54:22.339 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:54:22.363 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:54:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:54:22.651 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T10:54:22.681 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:54:22.706 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:54:22.709 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:22.709 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:54:22.709 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:54:22.710 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:54:22.710 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:22.766 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:54:22.777 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:54:22.879 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T10:54:22.932 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T10:54:23.909 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T10:54:23.935 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:54:23.958 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:54:23.958 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:23.959 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:54:23.959 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:54:23.959 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:54:23.959 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:24.002 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:54:24.024 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:54:24.025 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:24.025 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:54:24.025 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:24.069 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:54:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:54:24.093 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:24.093 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T10:54:24.093 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:54:24.093 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:54:24.093 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:24.420 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:54:24.443 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:54:24.444 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:54:24.444 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:54:24.444 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:54:24.444 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:54:24.444 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:24.843 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:54:24.843 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T10:54:24.843 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T10:54:24.843 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T10:54:24.843 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T10:54:24.843 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T10:54:24.843 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:54:24.843 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T10:54:24.844 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T10:54:24.845 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T10:54:24.846 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T10:54:24.859 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T10:54:24.859 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T10:54:24.860 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T10:54:24.861 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T10:54:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T10:54:24.863 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T10:54:24.864 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T10:54:24.865 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T10:54:24.866 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T10:54:24.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T10:54:24.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T10:54:24.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T10:54:24.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T10:54:24.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T10:54:24.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T10:54:24.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T10:54:24.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T10:54:24.868 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T10:54:24.869 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T10:54:24.870 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T10:54:24.870 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T10:54:24.870 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T10:54:24.870 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T10:54:24.870 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T10:54:24.870 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T10:54:24.870 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T10:54:24.870 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T10:54:24.870 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T10:54:26.916 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.917 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T10:54:26.918 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:54:26.919 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T10:54:26.920 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:54:26.921 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:54:26.922 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:54:26.923 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:54:26.924 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:54:26.925 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T10:54:26.926 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T10:54:26.927 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:54:26.928 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T10:54:26.928 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:26.928 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-04-20T10:54:27.256 DEBUG:teuthology.parallel:result is None 2024-04-20T10:54:29.308 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T10:54:29.349 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T10:54:29.391 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T10:54:29.422 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T10:54:29.449 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T10:54:29.652 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T10:54:29.703 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T10:54:29.733 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T10:54:30.340 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:54:30.372 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:54:30.393 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:54:31.250 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T10:54:31.344 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T10:54:31.386 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T10:54:31.472 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T10:54:31.506 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T10:54:31.509 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:54:31.590 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:54:31.591 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:54:37.361 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:54:37.361 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T10:54:37.361 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T10:54:37.361 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T10:54:37.361 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T10:54:37.361 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T10:54:37.361 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:54:37.361 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:54:37.361 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T10:54:37.362 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T10:54:37.363 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T10:54:37.364 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T10:54:37.365 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T10:54:37.366 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T10:54:37.367 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T10:54:37.368 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T10:54:37.369 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T10:54:37.370 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T10:54:37.371 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T10:54:37.372 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T10:54:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:54:39.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:39.477 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.478 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.479 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.480 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:54:39.481 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T10:54:39.482 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:54:39.483 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:54:39.484 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:54:39.485 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:54:39.486 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:54:39.487 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T10:54:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T10:54:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:54:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T10:54:39.488 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:39.488 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-04-20T10:54:39.846 DEBUG:teuthology.parallel:result is None 2024-04-20T10:54:39.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:54:39.989 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:54:40.011 INFO:teuthology.orchestra.run.smithi027.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T10:54:40.012 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T10:54:40.012 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T10:54:40.014 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:54:40.234 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:54:40.260 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T10:54:40.260 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T10:54:40.260 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T10:54:40.262 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T10:54:40.262 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:40.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:54:40.289 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:54:40.289 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:54:40.330 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T10:54:40.331 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:40.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:54:40.355 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:54:40.421 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:54:40.421 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:54:40.448 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:54:40.515 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T10:54:40.515 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:40.515 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:54:40.539 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:54:40.606 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:54:40.606 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:54:40.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:54:40.696 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T10:54:40.696 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:40.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:54:40.720 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:54:40.785 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:54:40.785 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:54:40.809 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:54:40.874 INFO:teuthology.run_tasks:Running task ceph... 2024-04-20T10:54:40.969 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-20T10:54:40.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /var/log/ceph 2024-04-20T10:54:40.972 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /var/log/ceph 2024-04-20T10:54:40.997 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-20T10:54:40.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T10:54:41.038 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T10:54:41.062 INFO:tasks.ceph:Creating extra log directories... 2024-04-20T10:54:41.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T10:54:41.104 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T10:54:41.135 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-20T10:54:41.135 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-20T10:54:41.135 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-20T10:54:41.135 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:41.135 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T10:54:41.183 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T10:54:41.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T10:54:41.313 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:54:41.314 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T10:54:41.328 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T10:54:41.436 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T10:54:41.481 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-20T10:54:41.482 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-20T10:54:41.482 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497', 'branch': 'wip-rishabh-testing-20240416.193735', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}}', 'email': None, 'first_in_suite': False, 'job_id': '7664497', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 7954, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi027.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH6OQS2TZtB4oGflvkl/p/zQruUyfhs+3zLMeL+zbjuRQSdVNAPdlMUtZ77HbtVvr3qonxSOCRcvDwShsLOc6mQ=', 'smithi137.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKRllXXUQ26mcfrH6E+Cd9sxM9XhjEJMmvuw/QCC0oGUM1jPNGGqrhbC9SoT/2O/4ygDa9Ejot2zF49ezENpeVM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'Replacing daemon mds'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['suites/fsstress.sh']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-20_09:50:15', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-20T10:54:41.483 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T10:54:41.485 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T10:54:41.530 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T10:54:41.542 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T10:54:41.594 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:41.595 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:54:41.619 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:54:41.619 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-04-20T10:54:41.675 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:54:41.675 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:54:41.675 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-20T10:54:41.675 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:54:41.675 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:54:41.675 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-20 10:54:32.108702586 +0000 2024-04-20T10:54:41.675 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-20 10:52:03.609015210 +0000 2024-04-20T10:54:41.675 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-20 10:52:03.609015210 +0000 2024-04-20T10:54:41.675 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-20 10:52:03.609015210 +0000 2024-04-20T10:54:41.675 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:54:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-20T10:54:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-20T10:54:41.738 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000221398 s, 2.3 MB/s 2024-04-20T10:54:41.739 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:54:41.795 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-04-20T10:54:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:54:41.850 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:54:41.851 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-20T10:54:41.851 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:54:41.851 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:54:41.851 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-20 10:54:32.108702586 +0000 2024-04-20T10:54:41.851 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-20 10:52:03.960000892 +0000 2024-04-20T10:54:41.851 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-20 10:52:03.960000892 +0000 2024-04-20T10:54:41.851 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-20 10:52:03.960000892 +0000 2024-04-20T10:54:41.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:54:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-20T10:54:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-20T10:54:41.913 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000206375 s, 2.5 MB/s 2024-04-20T10:54:41.914 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:54:41.970 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-04-20T10:54:42.025 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:54:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:54:42.026 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-20T10:54:42.026 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:54:42.026 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:54:42.026 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-20 10:54:32.109702562 +0000 2024-04-20T10:54:42.026 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-20 10:52:04.298987064 +0000 2024-04-20T10:54:42.026 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-20 10:52:04.298987064 +0000 2024-04-20T10:54:42.026 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-20 10:52:04.298987064 +0000 2024-04-20T10:54:42.026 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:54:42.089 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-20T10:54:42.089 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-20T10:54:42.089 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000230645 s, 2.2 MB/s 2024-04-20T10:54:42.090 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:54:42.146 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-04-20T10:54:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:54:42.202 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:54:42.202 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-20T10:54:42.203 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:54:42.203 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:54:42.203 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-04-20 10:54:32.109702562 +0000 2024-04-20T10:54:42.203 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-20 10:52:04.641973073 +0000 2024-04-20T10:54:42.203 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-20 10:52:04.641973073 +0000 2024-04-20T10:54:42.203 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-20 10:52:04.641973073 +0000 2024-04-20T10:54:42.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:54:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-04-20T10:54:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-04-20T10:54:42.265 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000254336 s, 2.0 MB/s 2024-04-20T10:54:42.266 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:54:42.322 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:54:42.322 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:54:42.322 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:54:42.338 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:54:42.338 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-04-20T10:54:42.392 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:54:42.393 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:54:42.393 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-20T10:54:42.393 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:54:42.393 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:54:42.393 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-20 10:54:19.457930894 +0000 2024-04-20T10:54:42.393 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-20 10:52:38.574826898 +0000 2024-04-20T10:54:42.393 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-20 10:52:38.574826898 +0000 2024-04-20T10:54:42.393 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-20 10:52:38.574826898 +0000 2024-04-20T10:54:42.393 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:54:42.454 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-20T10:54:42.454 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-20T10:54:42.454 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000259357 s, 2.0 MB/s 2024-04-20T10:54:42.455 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:54:42.510 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-04-20T10:54:42.568 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:54:42.569 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:54:42.569 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-20T10:54:42.569 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:54:42.569 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:54:42.569 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-20 10:54:19.458930876 +0000 2024-04-20T10:54:42.569 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-20 10:52:38.925820591 +0000 2024-04-20T10:54:42.569 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-20 10:52:38.925820591 +0000 2024-04-20T10:54:42.569 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-20 10:52:38.925820591 +0000 2024-04-20T10:54:42.569 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:54:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-20T10:54:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-20T10:54:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000216208 s, 2.4 MB/s 2024-04-20T10:54:42.633 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:54:42.689 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-04-20T10:54:42.744 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:54:42.744 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:54:42.744 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-20T10:54:42.744 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:54:42.745 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:54:42.745 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-20 10:54:19.458930876 +0000 2024-04-20T10:54:42.745 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-20 10:52:39.266814464 +0000 2024-04-20T10:54:42.745 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-20 10:52:39.266814464 +0000 2024-04-20T10:54:42.745 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-20 10:52:39.266814464 +0000 2024-04-20T10:54:42.745 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:54:42.806 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-20T10:54:42.807 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-20T10:54:42.807 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000192891 s, 2.7 MB/s 2024-04-20T10:54:42.808 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:54:42.863 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-04-20T10:54:42.918 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:54:42.918 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:54:42.918 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-20T10:54:42.918 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:54:42.918 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:54:42.918 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-20 10:54:19.459930858 +0000 2024-04-20T10:54:42.918 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-20 10:52:39.613808229 +0000 2024-04-20T10:54:42.919 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-20 10:52:39.613808229 +0000 2024-04-20T10:54:42.919 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-20 10:52:39.613808229 +0000 2024-04-20T10:54:42.919 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:54:42.980 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-20T10:54:42.980 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-20T10:54:42.981 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000170508 s, 3.0 MB/s 2024-04-20T10:54:42.982 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:54:43.037 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:54:43.037 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T10:54:43.037 INFO:tasks.ceph:Generating config... 2024-04-20T10:54:43.038 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-20T10:54:43.038 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-20T10:54:43.039 INFO:tasks.ceph:[client] debug client = 20 2024-04-20T10:54:43.039 INFO:tasks.ceph:[client] debug ms = 1 2024-04-20T10:54:43.039 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-20T10:54:43.039 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-20T10:54:43.039 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-20T10:54:43.039 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-20T10:54:43.040 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-20T10:54:43.041 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-20T10:54:43.041 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-20T10:54:43.041 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-20T10:54:43.041 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-20T10:54:43.041 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-20T10:54:43.041 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-20T10:54:43.042 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-20T10:54:43.043 INFO:tasks.ceph:Setting up mon.a... 2024-04-20T10:54:43.043 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-20T10:54:43.102 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.keyring 2024-04-20T10:54:43.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-20T10:54:43.148 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T10:54:43.214 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.27'), ('mon.b', '172.21.15.137'), ('mon.c', '[v2:172.21.15.137:3301,v1:172.21.15.137:6790]')] 2024-04-20T10:54:43.215 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.27,172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-20T10:54:43.216 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:43.216 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T10:54:43.270 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.27 --add b 172.21.15.137 --addv c '[v2:172.21.15.137:3301,v1:172.21.15.137:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:54:43.358 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:54:43.358 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: generated fsid 056b9857-269e-4bc1-b62f-8882e32cd3e3 2024-04-20T10:54:43.358 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = quincy 2024-04-20T10:54:43.358 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 0 2024-04-20T10:54:43.358 INFO:teuthology.orchestra.run.smithi027.stdout:fsid 056b9857-269e-4bc1-b62f-8882e32cd3e3 2024-04-20T10:54:43.359 INFO:teuthology.orchestra.run.smithi027.stdout:last_changed 2024-04-20T10:54:43.359395+0000 2024-04-20T10:54:43.359 INFO:teuthology.orchestra.run.smithi027.stdout:created 2024-04-20T10:54:43.359395+0000 2024-04-20T10:54:43.359 INFO:teuthology.orchestra.run.smithi027.stdout:min_mon_release 17 (quincy) 2024-04-20T10:54:43.359 INFO:teuthology.orchestra.run.smithi027.stdout:election_strategy: 1 2024-04-20T10:54:43.359 INFO:teuthology.orchestra.run.smithi027.stdout:0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2024-04-20T10:54:43.359 INFO:teuthology.orchestra.run.smithi027.stdout:1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-20T10:54:43.359 INFO:teuthology.orchestra.run.smithi027.stdout:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-04-20T10:54:43.359 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-20T10:54:43.360 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T10:54:43.378 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 056b9857-269e-4bc1-b62f-8882e32cd3e3... 2024-04-20T10:54:43.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T10:54:43.421 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T10:54:43.460 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2024-04-20T10:54:43.460 INFO:teuthology.orchestra.run.smithi027.stdout: chdir = "" 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: auth supported = cephx 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: filestore xattr use omap = true 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: mon clock drift allowed = 1.000 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: osd crush chooseleaf type = 0 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: auth debug = true 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on old message = true 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on bug = true 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.461 INFO:teuthology.orchestra.run.smithi027.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn max object skew = 0 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default size = 2 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow primary affinity = true 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow pg remap = true 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on legacy crush tunables = false 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on crush straw calc version zero = false 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on no sortbitwise = false 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = false 2024-04-20T10:54:43.462 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on too few osds = false 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: mon_allow_pool_size_one = true 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: osd default data pool replay window = 5 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: mon allow pool delete = true 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: mon cluster log file level = debug 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: debug asserts on shutdown = true 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: mon health detail to clog = false 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: mon host = "172.21.15.27,172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790]" 2024-04-20T10:54:43.463 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on skipped message = False 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 056b9857-269e-4bc1-b62f-8882e32cd3e3 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout:[osd] 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: osd journal size = 100 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub load threshold = 5.0 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub max interval = 600 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock profile = high_recovery_ops 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: osd recover clone overlap = true 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: osd recovery max chunk = 1048576 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug shutdown = true 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug op order = true 2024-04-20T10:54:43.464 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify stray on activate = true 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug trim objects = true 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: osd open classes on start = true 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug pg log writeout = true 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: osd deep scrub update digest min age = 30 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: osd map max advance = 10 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: journal zero on create = true 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: filestore ondisk finisher threads = 3 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: filestore apply finisher threads = 3 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.465 INFO:teuthology.orchestra.run.smithi027.stdout: bdev debug aio = true 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug misdirected ops = true 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: bdev async discard = True 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: bdev enable discard = True 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore allocator = bitmap 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore block size = 96636764160 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore fsck on mount = True 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluefs = 1/20 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluestore = 1/20 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: debug osd = 20 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: debug rocksdb = 4/10 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: mon client directed command retry = 5 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.9 2024-04-20T10:54:43.466 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.8 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject delay max = 1 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject delay probability = 0.005 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject delay type = client mds 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject socket failures = 2500 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: osd blocked scrub grace period = 3600 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: osd failsafe full ratio = 0.95 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: osd max markdown count = 1000 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: osd objectstore = bluestore 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr] 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: debug mgr = 20 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-04-20T10:54:43.467 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: mgr/telemetry/nag = false 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout:[mon] 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: debug paxos = 20 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: mon data avail warn = 5 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: mon mgr mkfs grace = 240 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:54:43.468 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd reporter subtree level = osd 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd prime pg temp = true 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: # 1m isn't quite enough 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: mon_down_mkfs_grace = 2m 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.469 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_filestore_osds = false 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: mon down mkfs grace = 300 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: mon op complaint time = 120 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout:[client] 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: rgw cache enabled = true 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable ops log = true 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable usage log = true 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: client mount timeout = 600 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: client_shutdown_timeout = 120 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-04-20T10:54:43.470 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.a] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.b] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.c] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout:[mds] 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds = 20 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds balancer = 20 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2024-04-20T10:54:43.471 INFO:teuthology.orchestra.run.smithi027.stdout: mds bal fragment size max = 10000 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds bal merge size = 5 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds bal split bits = 3 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds bal split size = 100 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug frag = True 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug scatterstat = True 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds oft prefetch dirfrags = True 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds op complaint time = 180 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds verify scatter = True 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds_dir_prefetch = True 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mds_sleep_rank_change = 5000000.0 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: mon client directed command retry = 5 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject delay max = 1 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject delay probability = 0.005 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject delay type = client mds osd 2024-04-20T10:54:43.472 INFO:teuthology.orchestra.run.smithi027.stdout: ms inject socket failures = 2500 2024-04-20T10:54:43.473 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2024-04-20T10:54:43.473 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2024-04-20T10:54:43.473 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2024-04-20T10:54:43.474 INFO:teuthology.orchestra.run.smithi137.stdout:[global] 2024-04-20T10:54:43.474 INFO:teuthology.orchestra.run.smithi137.stdout: chdir = "" 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: auth supported = cephx 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: filestore xattr use omap = true 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: mon clock drift allowed = 1.000 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: osd crush chooseleaf type = 0 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: auth debug = true 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on old message = true 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on bug = true 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T10:54:43.475 INFO:teuthology.orchestra.run.smithi137.stdout: mon pg warn max object skew = 0 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default size = 2 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow primary affinity = true 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow pg remap = true 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on legacy crush tunables = false 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on crush straw calc version zero = false 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on no sortbitwise = false 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on osd down out interval zero = false 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on too few osds = false 2024-04-20T10:54:43.476 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: mon_allow_pool_size_one = true 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: osd default data pool replay window = 5 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: mon allow pool delete = true 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: mon cluster log file level = debug 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: debug asserts on shutdown = true 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: mon health detail to clog = false 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: mon host = "172.21.15.27,172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790]" 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on skipped message = False 2024-04-20T10:54:43.477 INFO:teuthology.orchestra.run.smithi137.stdout: fsid = 056b9857-269e-4bc1-b62f-8882e32cd3e3 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout:[osd] 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: osd journal size = 100 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub load threshold = 5.0 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub max interval = 600 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: osd mclock profile = high_recovery_ops 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: osd recover clone overlap = true 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: osd recovery max chunk = 1048576 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug shutdown = true 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug op order = true 2024-04-20T10:54:43.478 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug verify stray on activate = true 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug trim objects = true 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: osd open classes on start = true 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug pg log writeout = true 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: osd deep scrub update digest min age = 30 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: osd map max advance = 10 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: journal zero on create = true 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: filestore ondisk finisher threads = 3 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: filestore apply finisher threads = 3 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.479 INFO:teuthology.orchestra.run.smithi137.stdout: bdev debug aio = true 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug misdirected ops = true 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: bdev async discard = True 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: bdev enable discard = True 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore allocator = bitmap 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore block size = 96636764160 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore fsck on mount = True 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluefs = 1/20 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluestore = 1/20 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: debug osd = 20 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: debug rocksdb = 4/10 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: mon client directed command retry = 5 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T10:54:43.480 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd full ratio = 0.9 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd nearfull ratio = 0.8 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: ms inject delay max = 1 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: ms inject delay probability = 0.005 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: ms inject delay type = client mds 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: ms inject socket failures = 2500 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: osd blocked scrub grace period = 3600 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: osd failsafe full ratio = 0.95 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: osd max markdown count = 1000 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: osd objectstore = bluestore 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: osd op complaint time = 180 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr] 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: debug mgr = 20 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-04-20T10:54:43.481 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: mgr/telemetry/nag = false 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: debug client = 20 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout:[mon] 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: debug paxos = 20 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: mon data avail warn = 5 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: mon mgr mkfs grace = 240 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd reporter subtree level = osd 2024-04-20T10:54:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd prime pg temp = true 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: # 1m isn't quite enough 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: mon_down_mkfs_grace = 2m 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_filestore_osds = false 2024-04-20T10:54:43.483 INFO:teuthology.orchestra.run.smithi137.stdout: mon down mkfs grace = 300 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: mon op complaint time = 120 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout:[client] 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: rgw cache enabled = true 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable ops log = true 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable usage log = true 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: client mount timeout = 600 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: client_shutdown_timeout = 120 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: debug client = 20 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: rados mon op timeout = 900 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout: rados osd op timeout = 900 2024-04-20T10:54:43.484 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.a] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.b] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.c] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.a] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.c] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.e] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.b] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.d] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.f] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout:[mds] 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout: debug mds = 20 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout: debug mds balancer = 20 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout: mds bal fragment size max = 10000 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout: mds bal merge size = 5 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout: mds bal split bits = 3 2024-04-20T10:54:43.485 INFO:teuthology.orchestra.run.smithi137.stdout: mds bal split size = 100 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: mds debug frag = True 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: mds debug scatterstat = True 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: mds oft prefetch dirfrags = True 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: mds op complaint time = 180 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: mds verify scatter = True 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: mds_dir_prefetch = True 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: mds_sleep_rank_change = 5000000.0 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: mon client directed command retry = 5 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: ms inject delay max = 1 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: ms inject delay probability = 0.005 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: ms inject delay type = client mds osd 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: ms inject socket failures = 2500 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: osd op complaint time = 180 2024-04-20T10:54:43.486 INFO:teuthology.orchestra.run.smithi137.stdout: rados mon op timeout = 900 2024-04-20T10:54:43.487 INFO:teuthology.orchestra.run.smithi137.stdout: rados osd op timeout = 900 2024-04-20T10:54:43.487 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-20T10:54:43.487 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-20T10:54:43.569 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-20T10:54:43.569 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:43.569 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-20T10:54:43.587 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:43.587 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-20T10:54:43.642 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T10:54:43.642 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:43.642 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T10:54:43.642 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T10:54:43.715 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:54:43.715 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:54:43.771 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T10:54:43.771 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:54:43.771 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T10:54:43.771 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T10:54:43.806 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:54:43.806 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:54:43.862 INFO:tasks.ceph:Setting up mon nodes... 2024-04-20T10:54:43.862 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-20T10:54:43.862 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T10:54:43.912 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T10:54:43.915 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T10:54:43.967 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T10:54:43.969 INFO:tasks.ceph:Setting up mds nodes... 2024-04-20T10:54:43.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T10:54:44.020 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T10:54:44.023 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-20T10:54:44.046 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T10:54:44.137 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T10:54:44.139 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-20T10:54:44.163 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-20T10:54:44.252 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-20T10:54:44.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-20T10:54:44.279 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T10:54:44.327 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T10:54:44.330 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-20T10:54:44.352 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T10:54:44.440 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T10:54:44.442 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-20T10:54:44.465 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-20T10:54:44.553 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-20T10:54:44.556 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-20T10:54:44.578 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-20T10:54:44.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T10:54:44.620 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-20T10:54:44.632 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-20T10:54:44.632 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T10:54:44.632 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-20T10:54:44.697 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:54:44.697 INFO:tasks.ceph:role: osd.0 2024-04-20T10:54:44.697 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T10:54:44.697 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T10:54:44.768 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:54:44.769 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:54:44.769 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:54:44.769 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:54:44.769 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:54:44.769 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:54:44.769 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:54:44.769 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:54:44.769 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:54:44.769 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:54:45.274 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-04-20T10:54:45.276 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-04-20T10:54:45.276 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-20T10:54:45.383 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-20T10:54:45.414 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-20T10:54:45.479 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:54:45.479 INFO:tasks.ceph:role: osd.1 2024-04-20T10:54:45.479 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T10:54:45.479 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T10:54:45.551 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:54:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:54:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:54:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:54:45.552 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:54:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:54:45.552 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:54:45.552 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:54:45.552 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:54:45.552 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:54:46.014 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-04-20T10:54:46.016 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-04-20T10:54:46.016 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-20T10:54:46.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-20T10:54:46.137 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-20T10:54:46.202 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:54:46.202 INFO:tasks.ceph:role: osd.2 2024-04-20T10:54:46.202 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T10:54:46.202 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T10:54:46.272 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:54:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:54:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:54:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:54:46.273 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:54:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:54:46.273 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:54:46.273 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:54:46.273 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:54:46.273 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:54:46.756 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-04-20T10:54:46.758 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-04-20T10:54:46.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-20T10:54:46.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-20T10:54:46.938 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-20T10:54:47.002 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:54:47.003 INFO:tasks.ceph:role: osd.3 2024-04-20T10:54:47.003 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T10:54:47.003 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T10:54:47.073 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:54:47.074 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:54:47.074 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:54:47.074 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:54:47.074 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:54:47.074 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:54:47.074 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:54:47.074 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:54:47.074 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:54:47.074 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:54:47.565 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2024-04-20T10:54:47.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2024-04-20T10:54:47.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-20T10:54:47.704 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-20T10:54:47.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:54:47.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:47.832+0000 7f717b387640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-20T10:54:47.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:47.832+0000 7f717b387640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-20T10:54:47.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:47.832+0000 7f717b387640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T10:54:47.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:47.832+0000 7f717b387640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T10:54:47.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:47.832+0000 7f717b387640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T10:54:47.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:47.833+0000 7f717b387640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-20T10:54:50.692 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T10:54:50.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:54:50.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:50.811+0000 7fa59e80e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-20T10:54:50.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:50.811+0000 7fa59e80e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-20T10:54:50.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:50.811+0000 7fa59e80e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T10:54:50.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:50.811+0000 7fa59e80e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T10:54:50.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:50.811+0000 7fa59e80e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T10:54:50.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:50.812+0000 7fa59e80e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-20T10:54:53.667 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T10:54:53.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:54:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:53.791+0000 7f1fa0e25640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-20T10:54:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:53.791+0000 7f1fa0e25640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-20T10:54:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:53.791+0000 7f1fa0e25640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T10:54:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:53.791+0000 7f1fa0e25640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T10:54:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:53.791+0000 7f1fa0e25640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T10:54:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:53.792+0000 7f1fa0e25640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-20T10:54:56.646 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T10:54:56.675 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:54:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:56.765+0000 7f7ff701f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-20T10:54:56.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:56.766+0000 7f7ff701f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-20T10:54:56.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:56.766+0000 7f7ff701f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T10:54:56.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:56.766+0000 7f7ff701f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T10:54:56.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:56.766+0000 7f7ff701f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T10:54:56.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:54:56.766+0000 7f7ff701f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-20T10:54:59.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T10:54:59.655 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-20T10:54:59.684 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:54:59.684 INFO:tasks.ceph:role: osd.4 2024-04-20T10:54:59.684 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T10:54:59.684 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T10:54:59.760 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:54:59.760 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:54:59.760 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:54:59.760 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:54:59.760 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:54:59.761 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:54:59.761 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:54:59.761 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:54:59.761 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:54:59.761 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:55:00.276 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-04-20T10:55:00.278 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-04-20T10:55:00.279 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-20T10:55:00.420 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-20T10:55:00.454 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-20T10:55:00.519 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:55:00.519 INFO:tasks.ceph:role: osd.5 2024-04-20T10:55:00.519 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T10:55:00.520 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T10:55:00.591 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:55:00.591 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:55:00.591 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:55:00.591 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:55:00.591 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:55:00.591 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:55:00.591 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:55:00.591 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:55:00.591 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:55:00.592 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:55:01.039 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-04-20T10:55:01.042 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-04-20T10:55:01.042 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-20T10:55:01.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-20T10:55:01.165 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-20T10:55:01.231 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:55:01.231 INFO:tasks.ceph:role: osd.6 2024-04-20T10:55:01.231 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T10:55:01.231 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T10:55:01.303 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:55:01.303 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:55:01.304 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:55:01.304 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:55:01.304 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:55:01.304 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:55:01.304 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:55:01.304 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:55:01.304 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:55:01.304 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:55:01.785 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-04-20T10:55:01.787 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-04-20T10:55:01.787 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-20T10:55:01.928 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-20T10:55:01.958 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-20T10:55:02.024 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:55:02.024 INFO:tasks.ceph:role: osd.7 2024-04-20T10:55:02.024 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T10:55:02.024 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:55:02.097 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:55:02.612 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-04-20T10:55:02.615 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-04-20T10:55:02.615 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-20T10:55:02.763 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-20T10:55:02.797 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:55:02.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:02.895+0000 7f8c36815640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-20T10:55:02.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:02.896+0000 7f8c36815640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-20T10:55:02.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:02.896+0000 7f8c36815640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T10:55:02.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:02.896+0000 7f8c36815640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T10:55:02.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:02.896+0000 7f8c36815640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T10:55:02.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:02.896+0000 7f8c36815640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-20T10:55:05.746 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T10:55:05.774 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:55:05.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:05.864+0000 7fd080b1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-20T10:55:05.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:05.864+0000 7fd080b1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-20T10:55:05.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:05.864+0000 7fd080b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T10:55:05.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:05.864+0000 7fd080b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T10:55:05.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:05.864+0000 7fd080b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T10:55:05.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:05.865+0000 7fd080b1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-20T10:55:08.725 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T10:55:08.752 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:55:08.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:08.841+0000 7f5cba696640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-20T10:55:08.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:08.841+0000 7f5cba696640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-20T10:55:08.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:08.841+0000 7f5cba696640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T10:55:08.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:08.841+0000 7f5cba696640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T10:55:08.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:08.841+0000 7f5cba696640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T10:55:08.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:08.842+0000 7f5cba696640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-20T10:55:11.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:55:11.486 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:55:11.701 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-20T10:55:11.728 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:55:11.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:11.815+0000 7f48bac0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-20T10:55:11.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:11.815+0000 7f48bac0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-20T10:55:11.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:11.815+0000 7f48bac0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T10:55:11.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:11.815+0000 7f48bac0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T10:55:11.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:11.815+0000 7f48bac0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T10:55:11.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:11.816+0000 7f48bac0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-20T10:55:14.669 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-20T10:55:14.697 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-20T10:55:14.697 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:14.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-20T10:55:14.726 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:14.727 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-20T10:55:14.794 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:14.795 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-20T10:55:14.861 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:14.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-20T10:55:14.928 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:14.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-20T10:55:14.993 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:14.994 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-20T10:55:15.059 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:15.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-20T10:55:15.125 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:15.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-20T10:55:15.190 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:15.190 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-20T10:55:15.219 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:15.219 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-20T10:55:15.286 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:15.286 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-20T10:55:15.352 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:15.352 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-20T10:55:15.418 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:15.418 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-20T10:55:15.482 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:15.482 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-20T10:55:15.547 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:15.547 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-20T10:55:15.612 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:15.612 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-20T10:55:15.676 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:15.676 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-20T10:55:15.694 INFO:tasks.ceph:Adding keys to all mons... 2024-04-20T10:55:15.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T10:55:15.737 DEBUG:teuthology.orchestra.run.smithi137:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T10:55:15.759 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.y] 2024-04-20T10:55:15.759 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQDzniNmjtp3NhAA4ygam3g1GQfzJUXhNrYScQ== 2024-04-20T10:55:15.759 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.a] 2024-04-20T10:55:15.759 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD0niNm4a9JARAAdPu2kBv5dS4wX75ytIA12w== 2024-04-20T10:55:15.759 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.c] 2024-04-20T10:55:15.759 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD0niNmR3c6CBAARsoQm4KlCiseab+UsFIXkQ== 2024-04-20T10:55:15.759 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.e] 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD0niNmWYEkDxAAHEKId/zTFiIP0N5rLwNGHQ== 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.0] 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD3niNmyQapMRAAYUaEehGAhag4njq6DwtDSw== 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.1] 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD6niNmtWBoMBAAqURuixQLdNZjVr1b6VpoKQ== 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.2] 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD9niNm/Ac0LxAAE3I59Kjs7BT/PJEjUdhsWg== 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.3] 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAAnyNmDVGwLRAA5IzdBYU3u3uxsSFG5Vvu/Q== 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.x] 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQDzniNmOui4ORAANjmr1wTrXrTVhGZxdI07oA== 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.b] 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD0niNmFeWVExAA4sP6oXNx5SRiI6g953Ms4w== 2024-04-20T10:55:15.760 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.d] 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD0niNme09LGhAAiB0Y9xz6f0/NNRFSDtNpiw== 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.f] 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD0niNml6kIIRAAYQW5ZoHlaf1PkPI3cjv06w== 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.4] 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAGnyNmHc1tNRAAEvrPgumqykWtvPRToA7QPw== 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.5] 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAJnyNmhouQMxAAHL8HieB2cjp4EG3WT3djdg== 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.6] 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAMnyNmE2ctMhAAiH1yI5hriQX5+dfU2Gyyyg== 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.7] 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAPnyNmNyqtMBAAN+xUzpPdofgTZTaDczaG8Q== 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout:[client.0] 2024-04-20T10:55:15.761 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD0niNm9DsGJRAAn6U/Wv9+9ErdJWAtW2OW0g== 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.y] 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDzniNmjtp3NhAA4ygam3g1GQfzJUXhNrYScQ== 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD0niNm4a9JARAAdPu2kBv5dS4wX75ytIA12w== 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD0niNmR3c6CBAARsoQm4KlCiseab+UsFIXkQ== 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.e] 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD0niNmWYEkDxAAHEKId/zTFiIP0N5rLwNGHQ== 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.0] 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD3niNmyQapMRAAYUaEehGAhag4njq6DwtDSw== 2024-04-20T10:55:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.1] 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD6niNmtWBoMBAAqURuixQLdNZjVr1b6VpoKQ== 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.2] 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD9niNm/Ac0LxAAE3I59Kjs7BT/PJEjUdhsWg== 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.3] 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAAnyNmDVGwLRAA5IzdBYU3u3uxsSFG5Vvu/Q== 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.x] 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQDzniNmOui4ORAANjmr1wTrXrTVhGZxdI07oA== 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD0niNmFeWVExAA4sP6oXNx5SRiI6g953Ms4w== 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD0niNme09LGhAAiB0Y9xz6f0/NNRFSDtNpiw== 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.f] 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD0niNml6kIIRAAYQW5ZoHlaf1PkPI3cjv06w== 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.4] 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAGnyNmHc1tNRAAEvrPgumqykWtvPRToA7QPw== 2024-04-20T10:55:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.5] 2024-04-20T10:55:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAJnyNmhouQMxAAHL8HieB2cjp4EG3WT3djdg== 2024-04-20T10:55:15.764 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.6] 2024-04-20T10:55:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAMnyNmE2ctMhAAiH1yI5hriQX5+dfU2Gyyyg== 2024-04-20T10:55:15.764 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.7] 2024-04-20T10:55:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQAPnyNmNyqtMBAAN+xUzpPdofgTZTaDczaG8Q== 2024-04-20T10:55:15.764 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2024-04-20T10:55:15.764 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQD0niNm9DsGJRAAn6U/Wv9+9ErdJWAtW2OW0g== 2024-04-20T10:55:15.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:55:15.808 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:55:15.860 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:15.862 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:15.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:15.917 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:15.969 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:15.972 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:16.023 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.025 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.075 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.126 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.174 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.177 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.225 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:55:16.227 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:55:16.275 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:16.277 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:16.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:16.331 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:16.382 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:16.384 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:55:16.436 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.438 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.486 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.539 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.588 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:55:16.638 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:55:16.641 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:55:16.694 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-20T10:55:16.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-20T10:55:16.718 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T10:55:16.813 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-20T10:55:16.837 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-20T10:55:16.861 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T10:55:16.959 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-20T10:55:16.982 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-20T10:55:17.045 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T10:55:17.139 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-20T10:55:17.162 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:55:17.165 DEBUG:teuthology.orchestra.run.smithi137:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:55:17.216 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-20T10:55:17.217 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T10:55:17.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T10:55:17.220 INFO:tasks.ceph.mon.a:Started 2024-04-20T10:55:17.220 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T10:55:17.221 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T10:55:17.259 INFO:tasks.ceph.mon.b:Started 2024-04-20T10:55:17.259 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T10:55:17.259 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T10:55:17.261 INFO:tasks.ceph.mon.c:Started 2024-04-20T10:55:17.261 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-20T10:55:17.262 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-20T10:55:17.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-20T10:55:17.265 INFO:tasks.ceph.mgr.y:Started 2024-04-20T10:55:17.265 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-20T10:55:17.265 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-20T10:55:17.267 INFO:tasks.ceph.mgr.x:Started 2024-04-20T10:55:17.267 INFO:tasks.ceph:Setting crush tunables to default 2024-04-20T10:55:17.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd crush tunables default 2024-04-20T10:55:17.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.368+0000 7fd03552e640 1 Processor -- start 2024-04-20T10:55:17.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.368+0000 7fd03552e640 1 -- start start 2024-04-20T10:55:17.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.368+0000 7fd03552e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 0x7fd030153280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:17.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.368+0000 7fd03552e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd030159ea0 0x7fd0301537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:17.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.368+0000 7fd03552e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0300c96e0 con 0x7fd0300cb8a0 2024-04-20T10:55:17.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.368+0000 7fd03552e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd0300c99c0 con 0x7fd030159ea0 2024-04-20T10:55:17.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.368+0000 7fd03552e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd0300c9ca0 con 0x7fd030073860 2024-04-20T10:55:17.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd02f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd030159ea0 0x7fd0301537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:17.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd02effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd0300cb8a0 0x7fd0300cbc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40916/0 (socket says 172.21.15.27:40916) 2024-04-20T10:55:17.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd02effd640 1 -- 172.21.15.27:0/3605521099 learned_addr learned my addr 172.21.15.27:0/3605521099 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:17.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd02f7fe640 1 -- 172.21.15.27:0/3605521099 >> v2:172.21.15.27:3300/0 conn(0x7fd030159ea0 msgr2=0x7fd0301537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T10:55:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd02f7fe640 1 -- 172.21.15.27:0/3605521099 >> v2:172.21.15.27:3300/0 conn(0x7fd030159ea0 msgr2=0x7fd0301537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T10:55:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd02f7fe640 1 --2- 172.21.15.27:0/3605521099 >> v2:172.21.15.27:3300/0 conn(0x7fd030159ea0 0x7fd0301537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:55:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd02f7fe640 1 --2- 172.21.15.27:0/3605521099 >> v2:172.21.15.27:3300/0 conn(0x7fd030159ea0 0x7fd0301537c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:55:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd027fff640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 0x7fd030153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd027fff640 1 -- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 msgr2=0x7fd030153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T10:55:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd027fff640 1 -- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 msgr2=0x7fd030153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T10:55:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd027fff640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 0x7fd030153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:55:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.369+0000 7fd027fff640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 0x7fd030153280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:55:17.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.530+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 913194911 0 0) 0x7fd0300c96e0 con 0x7fd0300cb8a0 2024-04-20T10:55:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.530+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd01c003520 con 0x7fd0300cb8a0 2024-04-20T10:55:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.530+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1974599606 0 0) 0x7fd01c003520 con 0x7fd0300cb8a0 2024-04-20T10:55:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.530+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 msgr2=0x7fd030153280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:55:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.530+0000 7fd02e7fc640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 0x7fd030153280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.530+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 >> v2:172.21.15.27:3300/0 conn(0x7fd030159ea0 msgr2=0x7fd0301537c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:55:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.530+0000 7fd02e7fc640 1 --2- 172.21.15.27:0/3605521099 >> v2:172.21.15.27:3300/0 conn(0x7fd030159ea0 0x7fd0301537c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0300c9f80 con 0x7fd0300cb8a0 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7fd020001670 con 0x7fd0300cb8a0 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 >> v1:172.21.15.27:6789/0 conn(0x7fd0300cb8a0 legacy=0x7fd0300cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02e7fc640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c0039b0 0x7fd01c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02e7fc640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd01c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02e7fc640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 0x7fd01c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd0300c99c0 con 0x7fd01c004800 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0300c9ca0 con 0x7fd01c0039b0 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd01c00bb50 con 0x7fd030073860 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd027fff640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd01c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02f7fe640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 0x7fd01c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd02effd640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c0039b0 0x7fd01c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd027fff640 1 -- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 msgr2=0x7fd01c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd027fff640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd030073860 0x7fd01c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd027fff640 1 -- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c0039b0 msgr2=0x7fd01c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd027fff640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd01c0039b0 0x7fd01c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.531+0000 7fd027fff640 1 -- 172.21.15.27:0/3605521099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c003520 con 0x7fd01c004800 2024-04-20T10:55:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.532+0000 7fd027fff640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd01c004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd018006fd0 tx=0x7fd01c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3517a9ca74bcbcb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.532+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01800b5e0 con 0x7fd01c004800 2024-04-20T10:55:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.532+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd018038070 con 0x7fd01c004800 2024-04-20T10:55:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.532+0000 7fd02e7fc640 1 -- 172.21.15.27:0/3605521099 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01802d040 con 0x7fd01c004800 2024-04-20T10:55:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.533+0000 7fd03552e640 1 -- 172.21.15.27:0/3605521099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 msgr2=0x7fd01c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.533+0000 7fd03552e640 1 --2- 172.21.15.27:0/3605521099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd01c004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd018006fd0 tx=0x7fd01c005830 comp rx=0 tx=0).stop 2024-04-20T10:55:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.533+0000 7fd02f7fe640 1 -- 172.21.15.27:0/3605521099 reap_dead start 2024-04-20T10:55:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.533+0000 7fd03552e640 1 -- 172.21.15.27:0/3605521099 shutdown_connections 2024-04-20T10:55:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.533+0000 7fd03552e640 1 -- 172.21.15.27:0/3605521099 >> 172.21.15.27:0/3605521099 conn(0x7fd03006f760 msgr2=0x7fd030070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.533+0000 7fd03552e640 1 -- 172.21.15.27:0/3605521099 shutdown_connections 2024-04-20T10:55:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.533+0000 7fd03552e640 1 -- 172.21.15.27:0/3605521099 wait complete. 2024-04-20T10:55:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.534+0000 7fd03552e640 1 Processor -- start 2024-04-20T10:55:17.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.534+0000 7fd03552e640 1 -- start start 2024-04-20T10:55:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd03552e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd030156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd03552e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0301e3a00 0x7fd0301eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd03552e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0300c4930 0x7fd030151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd03552e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd030153400 con 0x7fd01c004800 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd03552e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd030153280 con 0x7fd0300c4930 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd03552e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd030153580 con 0x7fd0301e3a00 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd02effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd030156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd02effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd030156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58610/0 (socket says 172.21.15.27:58610) 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd02effd640 1 -- 172.21.15.27:0/3486700476 learned_addr learned my addr 172.21.15.27:0/3486700476 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.535+0000 7fd027fff640 1 --2- 172.21.15.27:0/3486700476 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0301e3a00 0x7fd0301eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.536+0000 7fd02f7fe640 1 --2- 172.21.15.27:0/3486700476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0300c4930 0x7fd030151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.536+0000 7fd02effd640 1 -- 172.21.15.27:0/3486700476 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0301e3a00 msgr2=0x7fd0301eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.536+0000 7fd02effd640 1 --2- 172.21.15.27:0/3486700476 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0301e3a00 0x7fd0301eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.536+0000 7fd02effd640 1 -- 172.21.15.27:0/3486700476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0300c4930 msgr2=0x7fd030151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.536+0000 7fd02effd640 1 --2- 172.21.15.27:0/3486700476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0300c4930 0x7fd030151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:17.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.536+0000 7fd02effd640 1 -- 172.21.15.27:0/3486700476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd030156690 con 0x7fd01c004800 2024-04-20T10:55:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.536+0000 7fd02effd640 1 --2- 172.21.15.27:0/3486700476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd030156150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd020007370 tx=0x7fd020007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.537+0000 7fd0277fe640 1 -- 172.21.15.27:0/3486700476 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02001a070 con 0x7fd01c004800 2024-04-20T10:55:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.537+0000 7fd0277fe640 1 -- 172.21.15.27:0/3486700476 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02000f940 con 0x7fd01c004800 2024-04-20T10:55:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.537+0000 7fd0277fe640 1 -- 172.21.15.27:0/3486700476 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02000fd10 con 0x7fd01c004800 2024-04-20T10:55:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.537+0000 7fd03552e640 1 -- 172.21.15.27:0/3486700476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0301e6210 con 0x7fd01c004800 2024-04-20T10:55:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.537+0000 7fd03552e640 1 -- 172.21.15.27:0/3486700476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0301e3200 con 0x7fd01c004800 2024-04-20T10:55:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.537+0000 7fd0277fe640 1 -- 172.21.15.27:0/3486700476 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd02001d070 con 0x7fd01c004800 2024-04-20T10:55:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.537+0000 7fd0277fe640 1 -- 172.21.15.27:0/3486700476 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd02001f780 con 0x7fd01c004800 2024-04-20T10:55:17.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.538+0000 7fd03552e640 1 -- 172.21.15.27:0/3486700476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03004fcf0 con 0x7fd01c004800 2024-04-20T10:55:17.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.541+0000 7fd0277fe640 1 -- 172.21.15.27:0/3486700476 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fd020014450 con 0x7fd01c004800 2024-04-20T10:55:17.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.608+0000 7fd03552e640 1 -- 172.21.15.27:0/3486700476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd030073e40 con 0x7fd01c004800 2024-04-20T10:55:17.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.609+0000 7fd0277fe640 1 -- 172.21.15.27:0/3486700476 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fd02001f9d0 con 0x7fd01c004800 2024-04-20T10:55:17.610 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2024-04-20T10:55:17.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.610+0000 7fd03552e640 1 -- 172.21.15.27:0/3486700476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 msgr2=0x7fd030156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:17.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.610+0000 7fd03552e640 1 --2- 172.21.15.27:0/3486700476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd030156150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd020007370 tx=0x7fd020007830 comp rx=0 tx=0).stop 2024-04-20T10:55:17.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.610+0000 7fd03552e640 1 -- 172.21.15.27:0/3486700476 shutdown_connections 2024-04-20T10:55:17.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.610+0000 7fd03552e640 1 --2- 172.21.15.27:0/3486700476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0300c4930 0x7fd030151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:17.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.610+0000 7fd03552e640 1 --2- 172.21.15.27:0/3486700476 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0301e3a00 0x7fd0301eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:17.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.610+0000 7fd03552e640 1 --2- 172.21.15.27:0/3486700476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd01c004800 0x7fd030156150 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:17.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.610+0000 7fd03552e640 1 -- 172.21.15.27:0/3486700476 >> 172.21.15.27:0/3486700476 conn(0x7fd03006f760 msgr2=0x7fd0300c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:17.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.610+0000 7fd03552e640 1 -- 172.21.15.27:0/3486700476 shutdown_connections 2024-04-20T10:55:17.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:17.610+0000 7fd03552e640 1 -- 172.21.15.27:0/3486700476 wait complete. 2024-04-20T10:55:17.621 INFO:tasks.ceph:check_enable_crimson: False 2024-04-20T10:55:17.622 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-20T10:55:17.622 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:17.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-20T10:55:17.636 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:17.636+0000 7f5b56e3f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:55:17.644 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:17.644+0000 7f242960d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:55:17.646 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:17.646 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-20T10:55:17.711 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:17.711 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-20T10:55:17.774 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:55:17.774 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-20T10:55:17.776 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:17.776+0000 7f5b56e3f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:55:17.804 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:17.803+0000 7f242960d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:55:17.827 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:17.827 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-20T10:55:17.830 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:17.829+0000 7f5b56e3f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:55:17.851 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:17.851 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-20T10:55:17.861 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:17.860+0000 7f242960d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:55:17.914 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:17.914 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-20T10:55:17.978 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-20T10:55:17.979 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-20T10:55:18.042 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 4a4dfaa3-f7b0-4a8e-bed9-018df61c508b 0 2024-04-20T10:55:18.088 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:18.088+0000 7f5b56e3f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:55:18.135 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:18.135+0000 7f242960d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:55:18.137 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:18.136+0000 7f5b56e3f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:55:18.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.179+0000 7f498a832640 1 Processor -- start 2024-04-20T10:55:18.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.179+0000 7f498a832640 1 -- start start 2024-04-20T10:55:18.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f498a832640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f498415eca0 0x7f49840c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f498a832640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415f080 0x7f498415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f498a832640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f49841551d0 0x7f4984155660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f498a832640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f49840c9ea0 con 0x7f498415eca0 2024-04-20T10:55:18.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f498a832640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f4984155ba0 con 0x7f49841551d0 2024-04-20T10:55:18.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f498a832640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f498415f9e0 con 0x7f498415f080 2024-04-20T10:55:18.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f49837fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415f080 0x7f498415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f4988dcf640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f49841551d0 0x7f4984155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f49837fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415f080 0x7f498415f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49298/0 (socket says 172.21.15.137:49298) 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f4988dcf640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f49841551d0 0x7f4984155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41982/0 (socket says 172.21.15.137:41982) 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f49837fe640 1 -- 172.21.15.137:0/3041699894 learned_addr learned my addr 172.21.15.137:0/3041699894 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f4983fff640 1 --2- 172.21.15.137:0/3041699894 >> v2:172.21.15.27:3300/0 conn(0x7f498415eca0 0x7f49840c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f4988dcf640 1 -- 172.21.15.137:0/3041699894 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415f080 msgr2=0x7f498415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f4988dcf640 1 --2- 172.21.15.137:0/3041699894 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415f080 0x7f498415f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f4988dcf640 1 -- 172.21.15.137:0/3041699894 >> v2:172.21.15.27:3300/0 conn(0x7f498415eca0 msgr2=0x7f49840c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f4988dcf640 1 --2- 172.21.15.137:0/3041699894 >> v2:172.21.15.27:3300/0 conn(0x7f498415eca0 0x7f49840c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f4988dcf640 1 -- 172.21.15.137:0/3041699894 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49840c2f40 con 0x7f49841551d0 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.180+0000 7f49837fe640 1 --2- 172.21.15.137:0/3041699894 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415f080 0x7f498415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.181+0000 7f4988dcf640 1 --2- 172.21.15.137:0/3041699894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 0x7f4984155660 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f498415c5d0 tx=0x7f497c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a4f012dc888f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.181+0000 7f4982ffd640 1 -- 172.21.15.137:0/3041699894 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c039070 con 0x7f49841551d0 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.181+0000 7f4982ffd640 1 -- 172.21.15.137:0/3041699894 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c02cc60 con 0x7f49841551d0 2024-04-20T10:55:18.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.181+0000 7f4982ffd640 1 -- 172.21.15.137:0/3041699894 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c034660 con 0x7f49841551d0 2024-04-20T10:55:18.184 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:18.183+0000 7f242960d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:55:18.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.182+0000 7f498a832640 1 -- 172.21.15.137:0/3041699894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 msgr2=0x7f4984155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.182+0000 7f498a832640 1 --2- 172.21.15.137:0/3041699894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 0x7f4984155660 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f498415c5d0 tx=0x7f497c02beb0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.182+0000 7f498a832640 1 -- 172.21.15.137:0/3041699894 shutdown_connections 2024-04-20T10:55:18.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.182+0000 7f498a832640 1 --2- 172.21.15.137:0/3041699894 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415f080 0x7f498415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.182+0000 7f498a832640 1 --2- 172.21.15.137:0/3041699894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 0x7f4984155660 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.182+0000 7f498a832640 1 --2- 172.21.15.137:0/3041699894 >> v2:172.21.15.27:3300/0 conn(0x7f498415eca0 0x7f49840c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.182+0000 7f498a832640 1 -- 172.21.15.137:0/3041699894 >> 172.21.15.137:0/3041699894 conn(0x7f498406fc00 msgr2=0x7f498406ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.182+0000 7f498a832640 1 -- 172.21.15.137:0/3041699894 shutdown_connections 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.182+0000 7f498a832640 1 -- 172.21.15.137:0/3041699894 wait complete. 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.183+0000 7f498a832640 1 Processor -- start 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.183+0000 7f498a832640 1 -- start start 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f498a832640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 0x7f498414ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f498a832640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415eca0 0x7f49841f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f498a832640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498415f080 0x7f49841580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f498a832640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f49840c93b0 con 0x7f498415f080 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f498a832640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49840c90d0 con 0x7f49841551d0 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f498a832640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f49840c4920 con 0x7f498415eca0 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4983fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 0x7f498414ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f49837fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415eca0 0x7f49841f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4983fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 0x7f498414ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41994/0 (socket says 172.21.15.137:41994) 2024-04-20T10:55:18.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f49837fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415eca0 0x7f49841f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49310/0 (socket says 172.21.15.137:49310) 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4983fff640 1 -- 172.21.15.137:0/771385898 learned_addr learned my addr 172.21.15.137:0/771385898 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4988dcf640 1 --2- 172.21.15.137:0/771385898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498415f080 0x7f49841580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4983fff640 1 -- 172.21.15.137:0/771385898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415eca0 msgr2=0x7f49841f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4983fff640 1 --2- 172.21.15.137:0/771385898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415eca0 0x7f49841f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4983fff640 1 -- 172.21.15.137:0/771385898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498415f080 msgr2=0x7f49841580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4983fff640 1 --2- 172.21.15.137:0/771385898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498415f080 0x7f49841580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4983fff640 1 -- 172.21.15.137:0/771385898 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49840ccc70 con 0x7f49841551d0 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.184+0000 7f4983fff640 1 --2- 172.21.15.137:0/771385898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 0x7f498414ccd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f497800e8f0 tx=0x7f497800edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.185+0000 7f4980ff9640 1 -- 172.21.15.137:0/771385898 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4978017070 con 0x7f49841551d0 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.185+0000 7f4980ff9640 1 -- 172.21.15.137:0/771385898 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497800c940 con 0x7f49841551d0 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.185+0000 7f498a832640 1 -- 172.21.15.137:0/771385898 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49841f43c0 con 0x7f49841551d0 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.185+0000 7f4980ff9640 1 -- 172.21.15.137:0/771385898 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497800cd10 con 0x7f49841551d0 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.185+0000 7f498a832640 1 -- 172.21.15.137:0/771385898 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49840cdc50 con 0x7f49841551d0 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.185+0000 7f4980ff9640 1 -- 172.21.15.137:0/771385898 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f497801a070 con 0x7f49841551d0 2024-04-20T10:55:18.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.185+0000 7f4980ff9640 1 -- 172.21.15.137:0/771385898 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f4978013470 con 0x7f49841551d0 2024-04-20T10:55:18.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.186+0000 7f498a832640 1 -- 172.21.15.137:0/771385898 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498404fcf0 con 0x7f49841551d0 2024-04-20T10:55:18.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.188+0000 7f4980ff9640 1 -- 172.21.15.137:0/771385898 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f49840ccc70 con 0x7f49841551d0 2024-04-20T10:55:18.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.259+0000 7f498a832640 1 -- 172.21.15.137:0/771385898 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4a4dfaa3-f7b0-4a8e-bed9-018df61c508b", "id": 0} v 0) -- 0x7f49840ce970 con 0x7f49841551d0 2024-04-20T10:55:18.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.262+0000 7f4980ff9640 1 -- 172.21.15.137:0/771385898 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4a4dfaa3-f7b0-4a8e-bed9-018df61c508b", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f49780136c0 con 0x7f49841551d0 2024-04-20T10:55:18.263 INFO:teuthology.orchestra.run.smithi137.stdout:0 2024-04-20T10:55:18.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.263+0000 7f498a832640 1 -- 172.21.15.137:0/771385898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 msgr2=0x7f498414ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.263+0000 7f498a832640 1 --2- 172.21.15.137:0/771385898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 0x7f498414ccd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f497800e8f0 tx=0x7f497800edb0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.263+0000 7f498a832640 1 -- 172.21.15.137:0/771385898 shutdown_connections 2024-04-20T10:55:18.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.263+0000 7f498a832640 1 --2- 172.21.15.137:0/771385898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498415f080 0x7f49841580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.263+0000 7f498a832640 1 --2- 172.21.15.137:0/771385898 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f498415eca0 0x7f49841f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.263+0000 7f498a832640 1 --2- 172.21.15.137:0/771385898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841551d0 0x7f498414ccd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.263+0000 7f498a832640 1 -- 172.21.15.137:0/771385898 >> 172.21.15.137:0/771385898 conn(0x7f498406fc00 msgr2=0x7f498415df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:18.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.263+0000 7f498a832640 1 -- 172.21.15.137:0/771385898 shutdown_connections 2024-04-20T10:55:18.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.263+0000 7f498a832640 1 -- 172.21.15.137:0/771385898 wait complete. 2024-04-20T10:55:18.274 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new f540862a-9f5e-411a-9f8a-5096226ef3b5 1 2024-04-20T10:55:18.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.410+0000 7fd72246d640 1 Processor -- start 2024-04-20T10:55:18.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.410+0000 7fd72246d640 1 -- start start 2024-04-20T10:55:18.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd72246d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c0c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd72246d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd71c0ced50 0x7fd71c0cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd72246d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd71c0c9e90 con 0x7fd71c15c200 2024-04-20T10:55:18.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd72246d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fd71c06db30 con 0x7fd71c0ced50 2024-04-20T10:55:18.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd72246d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c0716c0 con 0x7fd71c152490 2024-04-20T10:55:18.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd71bfff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c0c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd720a0a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd71c0ced50 0x7fd71c0cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd71bfff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c0c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49326/0 (socket says 172.21.15.137:49326) 2024-04-20T10:55:18.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd71bfff640 1 -- 172.21.15.137:0/2786093555 learned_addr learned my addr 172.21.15.137:0/2786093555 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:18.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd720a0a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd71c0ced50 0x7fd71c0cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42004/0 (socket says 172.21.15.137:42004) 2024-04-20T10:55:18.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd71bfff640 1 -- 172.21.15.137:0/2786093555 >> v2:172.21.15.137:3300/0 conn(0x7fd71c0ced50 msgr2=0x7fd71c0cf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:18.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd71bfff640 1 --2- 172.21.15.137:0/2786093555 >> v2:172.21.15.137:3300/0 conn(0x7fd71c0ced50 0x7fd71c0cf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd71bfff640 1 -- 172.21.15.137:0/2786093555 >> v1:172.21.15.27:6789/0 conn(0x7fd71c15c200 legacy=0x7fd71c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd71bfff640 1 -- 172.21.15.137:0/2786093555 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c071960 con 0x7fd71c152490 2024-04-20T10:55:18.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.411+0000 7fd720a0a640 1 --2- 172.21.15.137:0/2786093555 >> v2:172.21.15.137:3300/0 conn(0x7fd71c0ced50 0x7fd71c0cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:18.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.412+0000 7fd71bfff640 1 --2- 172.21.15.137:0/2786093555 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c0c9800 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009870 tx=0x7fd70c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec4fc11b1ef7b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:18.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.412+0000 7fd71affd640 1 -- 172.21.15.137:0/2786093555 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c034070 con 0x7fd71c152490 2024-04-20T10:55:18.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.412+0000 7fd71affd640 1 -- 172.21.15.137:0/2786093555 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70c030840 con 0x7fd71c152490 2024-04-20T10:55:18.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.412+0000 7fd71affd640 1 -- 172.21.15.137:0/2786093555 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c030c10 con 0x7fd71c152490 2024-04-20T10:55:18.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.412+0000 7fd72246d640 1 -- 172.21.15.137:0/2786093555 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 msgr2=0x7fd71c0c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.412+0000 7fd72246d640 1 --2- 172.21.15.137:0/2786093555 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c0c9800 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009870 tx=0x7fd70c02b980 comp rx=0 tx=0).stop 2024-04-20T10:55:18.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.413+0000 7fd72246d640 1 -- 172.21.15.137:0/2786093555 shutdown_connections 2024-04-20T10:55:18.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.413+0000 7fd72246d640 1 --2- 172.21.15.137:0/2786093555 >> v2:172.21.15.137:3300/0 conn(0x7fd71c0ced50 0x7fd71c0cf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.413+0000 7fd72246d640 1 --2- 172.21.15.137:0/2786093555 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c0c9800 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.413+0000 7fd72246d640 1 -- 172.21.15.137:0/2786093555 >> 172.21.15.137:0/2786093555 conn(0x7fd71c0676c0 msgr2=0x7fd71c067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:18.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.413+0000 7fd72246d640 1 -- 172.21.15.137:0/2786093555 shutdown_connections 2024-04-20T10:55:18.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.413+0000 7fd72246d640 1 -- 172.21.15.137:0/2786093555 wait complete. 2024-04-20T10:55:18.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.414+0000 7fd72246d640 1 Processor -- start 2024-04-20T10:55:18.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.414+0000 7fd72246d640 1 -- start start 2024-04-20T10:55:18.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.414+0000 7fd72246d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd71c0ced50 0x7fd71c150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.414+0000 7fd72246d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c1e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd72246d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd71c15c200 0x7fd71c14a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd72246d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c152870 con 0x7fd71c0ced50 2024-04-20T10:55:18.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd72246d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c06cf00 con 0x7fd71c15c200 2024-04-20T10:55:18.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd72246d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c15c620 con 0x7fd71c152490 2024-04-20T10:55:18.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c1e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd720a0a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd71c15c200 0x7fd71c14a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c1e60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49338/0 (socket says 172.21.15.137:49338) 2024-04-20T10:55:18.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71b7fe640 1 -- 172.21.15.137:0/2451410932 learned_addr learned my addr 172.21.15.137:0/2451410932 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:18.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71bfff640 1 --2- 172.21.15.137:0/2451410932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd71c0ced50 0x7fd71c150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71b7fe640 1 -- 172.21.15.137:0/2451410932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd71c15c200 msgr2=0x7fd71c14a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71b7fe640 1 --2- 172.21.15.137:0/2451410932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd71c15c200 0x7fd71c14a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71b7fe640 1 -- 172.21.15.137:0/2451410932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd71c0ced50 msgr2=0x7fd71c150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71b7fe640 1 --2- 172.21.15.137:0/2451410932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd71c0ced50 0x7fd71c150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71b7fe640 1 -- 172.21.15.137:0/2451410932 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c0cab00 con 0x7fd71c152490 2024-04-20T10:55:18.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd720a0a640 1 --2- 172.21.15.137:0/2451410932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd71c15c200 0x7fd71c14a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:18.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd71b7fe640 1 --2- 172.21.15.137:0/2451410932 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c1e60e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c3c0 tx=0x7fd71000c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:18.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd718ff9640 1 -- 172.21.15.137:0/2451410932 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7100052d0 con 0x7fd71c152490 2024-04-20T10:55:18.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd718ff9640 1 -- 172.21.15.137:0/2451410932 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd710005490 con 0x7fd71c152490 2024-04-20T10:55:18.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd72246d640 1 -- 172.21.15.137:0/2451410932 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71c1e6b20 con 0x7fd71c152490 2024-04-20T10:55:18.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd718ff9640 1 -- 172.21.15.137:0/2451410932 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710007260 con 0x7fd71c152490 2024-04-20T10:55:18.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd72246d640 1 -- 172.21.15.137:0/2451410932 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71c1e70e0 con 0x7fd71c152490 2024-04-20T10:55:18.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd718ff9640 1 -- 172.21.15.137:0/2451410932 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd710007400 con 0x7fd71c152490 2024-04-20T10:55:18.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.415+0000 7fd718ff9640 1 -- 172.21.15.137:0/2451410932 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd710003a90 con 0x7fd71c152490 2024-04-20T10:55:18.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.416+0000 7fd72246d640 1 -- 172.21.15.137:0/2451410932 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71c04fcf0 con 0x7fd71c152490 2024-04-20T10:55:18.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.418+0000 7fd718ff9640 1 -- 172.21.15.137:0/2451410932 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fd7100035f0 con 0x7fd71c152490 2024-04-20T10:55:18.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.489+0000 7fd72246d640 1 -- 172.21.15.137:0/2451410932 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f540862a-9f5e-411a-9f8a-5096226ef3b5", "id": 1} v 0) -- 0x7fd71c1f2c90 con 0x7fd71c152490 2024-04-20T10:55:18.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.492+0000 7fd718ff9640 1 -- 172.21.15.137:0/2451410932 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f540862a-9f5e-411a-9f8a-5096226ef3b5", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd710003ce0 con 0x7fd71c152490 2024-04-20T10:55:18.493 INFO:teuthology.orchestra.run.smithi137.stdout:1 2024-04-20T10:55:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.493+0000 7fd72246d640 1 -- 172.21.15.137:0/2451410932 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 msgr2=0x7fd71c1e60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.493+0000 7fd72246d640 1 --2- 172.21.15.137:0/2451410932 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c1e60e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c3c0 tx=0x7fd71000c7c0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.493+0000 7fd72246d640 1 -- 172.21.15.137:0/2451410932 shutdown_connections 2024-04-20T10:55:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.493+0000 7fd72246d640 1 --2- 172.21.15.137:0/2451410932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd71c15c200 0x7fd71c14a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.493+0000 7fd72246d640 1 --2- 172.21.15.137:0/2451410932 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd71c152490 0x7fd71c1e60e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.493+0000 7fd72246d640 1 --2- 172.21.15.137:0/2451410932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd71c0ced50 0x7fd71c150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.493+0000 7fd72246d640 1 -- 172.21.15.137:0/2451410932 >> 172.21.15.137:0/2451410932 conn(0x7fd71c0676c0 msgr2=0x7fd71c0c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.493+0000 7fd72246d640 1 -- 172.21.15.137:0/2451410932 shutdown_connections 2024-04-20T10:55:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.493+0000 7fd72246d640 1 -- 172.21.15.137:0/2451410932 wait complete. 2024-04-20T10:55:18.504 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 0059bcea-d2c5-47f3-9100-8d862a129dd1 2 2024-04-20T10:55:18.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.601+0000 7f3f9db85640 1 Processor -- start 2024-04-20T10:55:18.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.601+0000 7f3f9db85640 1 -- start start 2024-04-20T10:55:18.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f9db85640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3f980c9490 0x7f3f9806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f9db85640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f98159c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f9db85640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f980c6ca0 con 0x7f3f980677e0 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f9db85640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f3f980c6a70 con 0x7f3f980c9490 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f9db85640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f3f98071910 con 0x7f3f980c9870 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f977fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3f980c9490 0x7f3f9806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f96ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f98159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f96ffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f98159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49346/0 (socket says 172.21.15.137:49346) 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f977fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3f980c9490 0x7f3f9806d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42016/0 (socket says 172.21.15.137:42016) 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.602+0000 7f3f96ffd640 1 -- 172.21.15.137:0/768478182 learned_addr learned my addr 172.21.15.137:0/768478182 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.603+0000 7f3f96ffd640 1 -- 172.21.15.137:0/768478182 >> v2:172.21.15.137:3300/0 conn(0x7f3f980c9490 msgr2=0x7f3f9806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.603+0000 7f3f96ffd640 1 --2- 172.21.15.137:0/768478182 >> v2:172.21.15.137:3300/0 conn(0x7f3f980c9490 0x7f3f9806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.603+0000 7f3f96ffd640 1 -- 172.21.15.137:0/768478182 >> v1:172.21.15.27:6789/0 conn(0x7f3f980677e0 legacy=0x7f3f981530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.603+0000 7f3f96ffd640 1 -- 172.21.15.137:0/768478182 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f980c6f40 con 0x7f3f980c9870 2024-04-20T10:55:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.603+0000 7f3f977fe640 1 --2- 172.21.15.137:0/768478182 >> v2:172.21.15.137:3300/0 conn(0x7f3f980c9490 0x7f3f9806d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:18.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.603+0000 7f3f96ffd640 1 --2- 172.21.15.137:0/768478182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f98159c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009870 tx=0x7f3f8802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e73bd7597682cbb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:18.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.603+0000 7f3f967fc640 1 -- 172.21.15.137:0/768478182 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88035020 con 0x7f3f980c9870 2024-04-20T10:55:18.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f967fc640 1 -- 172.21.15.137:0/768478182 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f88031640 con 0x7f3f980c9870 2024-04-20T10:55:18.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f967fc640 1 -- 172.21.15.137:0/768478182 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8802d040 con 0x7f3f980c9870 2024-04-20T10:55:18.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f9db85640 1 -- 172.21.15.137:0/768478182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 msgr2=0x7f3f98159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f9db85640 1 --2- 172.21.15.137:0/768478182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f98159c80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009870 tx=0x7f3f8802b8f0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f9db85640 1 -- 172.21.15.137:0/768478182 shutdown_connections 2024-04-20T10:55:18.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f9db85640 1 --2- 172.21.15.137:0/768478182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f98159c80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f9db85640 1 --2- 172.21.15.137:0/768478182 >> v2:172.21.15.137:3300/0 conn(0x7f3f980c9490 0x7f3f9806d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f9db85640 1 -- 172.21.15.137:0/768478182 >> 172.21.15.137:0/768478182 conn(0x7f3f9806b930 msgr2=0x7f3f9806bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:18.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f9db85640 1 -- 172.21.15.137:0/768478182 shutdown_connections 2024-04-20T10:55:18.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.604+0000 7f3f9db85640 1 -- 172.21.15.137:0/768478182 wait complete. 2024-04-20T10:55:18.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.605+0000 7f3f9db85640 1 Processor -- start 2024-04-20T10:55:18.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.605+0000 7f3f9db85640 1 -- start start 2024-04-20T10:55:18.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f9db85640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f980677e0 0x7f3f9814cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f9db85640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f980c9490 0x7f3f981f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f9db85640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f981f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f9db85640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f980c74a0 con 0x7f3f980677e0 2024-04-20T10:55:18.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f9db85640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3f981597f0 con 0x7f3f980c9490 2024-04-20T10:55:18.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f9db85640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f3f98159480 con 0x7f3f980c9870 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f96ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f980c9490 0x7f3f981f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f981f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f97fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f981f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49358/0 (socket says 172.21.15.137:49358) 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f96ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f980c9490 0x7f3f981f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42030/0 (socket says 172.21.15.137:42030) 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f97fff640 1 -- 172.21.15.137:0/1667159105 learned_addr learned my addr 172.21.15.137:0/1667159105 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f977fe640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f980677e0 0x7f3f9814cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f97fff640 1 -- 172.21.15.137:0/1667159105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f980c9490 msgr2=0x7f3f981f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f97fff640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f980c9490 0x7f3f981f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f97fff640 1 -- 172.21.15.137:0/1667159105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f980677e0 msgr2=0x7f3f9814cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f97fff640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f980677e0 0x7f3f9814cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f97fff640 1 -- 172.21.15.137:0/1667159105 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f98149790 con 0x7f3f980c9870 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f96ffd640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f980c9490 0x7f3f981f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.606+0000 7f3f977fe640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f980677e0 0x7f3f9814cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:18.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.607+0000 7f3f97fff640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f981f4bf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000b9c0 tx=0x7f3f9000be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:18.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.607+0000 7f3f7ffff640 1 -- 172.21.15.137:0/1667159105 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90019070 con 0x7f3f980c9870 2024-04-20T10:55:18.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.607+0000 7f3f7ffff640 1 -- 172.21.15.137:0/1667159105 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9000cba0 con 0x7f3f980c9870 2024-04-20T10:55:18.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.607+0000 7f3f9db85640 1 -- 172.21.15.137:0/1667159105 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f98149a70 con 0x7f3f980c9870 2024-04-20T10:55:18.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.607+0000 7f3f7ffff640 1 -- 172.21.15.137:0/1667159105 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90014440 con 0x7f3f980c9870 2024-04-20T10:55:18.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.607+0000 7f3f9db85640 1 -- 172.21.15.137:0/1667159105 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9814a030 con 0x7f3f980c9870 2024-04-20T10:55:18.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.607+0000 7f3f7ffff640 1 -- 172.21.15.137:0/1667159105 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f90003970 con 0x7f3f980c9870 2024-04-20T10:55:18.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.607+0000 7f3f7ffff640 1 -- 172.21.15.137:0/1667159105 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3f90007790 con 0x7f3f980c9870 2024-04-20T10:55:18.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.608+0000 7f3f9db85640 1 -- 172.21.15.137:0/1667159105 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9804fcf0 con 0x7f3f980c9870 2024-04-20T10:55:18.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.610+0000 7f3f7ffff640 1 -- 172.21.15.137:0/1667159105 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f3f90007370 con 0x7f3f980c9870 2024-04-20T10:55:18.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.682+0000 7f3f9db85640 1 -- 172.21.15.137:0/1667159105 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0059bcea-d2c5-47f3-9100-8d862a129dd1", "id": 2} v 0) -- 0x7f3f9814e730 con 0x7f3f980c9870 2024-04-20T10:55:18.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.687+0000 7f3f7ffff640 1 -- 172.21.15.137:0/1667159105 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0059bcea-d2c5-47f3-9100-8d862a129dd1", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f3f90007a90 con 0x7f3f980c9870 2024-04-20T10:55:18.688 INFO:teuthology.orchestra.run.smithi137.stdout:2 2024-04-20T10:55:18.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.688+0000 7f3f9db85640 1 -- 172.21.15.137:0/1667159105 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 msgr2=0x7f3f981f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.688+0000 7f3f9db85640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f981f4bf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000b9c0 tx=0x7f3f9000be80 comp rx=0 tx=0).stop 2024-04-20T10:55:18.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.688+0000 7f3f9db85640 1 -- 172.21.15.137:0/1667159105 shutdown_connections 2024-04-20T10:55:18.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.688+0000 7f3f9db85640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3f980c9870 0x7f3f981f4bf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.688+0000 7f3f9db85640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f980c9490 0x7f3f981f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.688+0000 7f3f9db85640 1 --2- 172.21.15.137:0/1667159105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f980677e0 0x7f3f9814cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.688+0000 7f3f9db85640 1 -- 172.21.15.137:0/1667159105 >> 172.21.15.137:0/1667159105 conn(0x7f3f9806b930 msgr2=0x7f3f98071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:18.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.688+0000 7f3f9db85640 1 -- 172.21.15.137:0/1667159105 shutdown_connections 2024-04-20T10:55:18.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.688+0000 7f3f9db85640 1 -- 172.21.15.137:0/1667159105 wait complete. 2024-04-20T10:55:18.700 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new d33806ff-f093-4414-8a74-f48775c0e13d 3 2024-04-20T10:55:18.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.796+0000 7f5a71df4640 1 Processor -- start 2024-04-20T10:55:18.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.796+0000 7f5a71df4640 1 -- start start 2024-04-20T10:55:18.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.797+0000 7f5a71df4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a6c072b30 0x7f5a6c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.797+0000 7f5a71df4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6c0738b0 con 0x7f5a6c159de0 2024-04-20T10:55:18.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.797+0000 7f5a71df4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6c073370 con 0x7f5a6c153600 2024-04-20T10:55:18.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.797+0000 7f5a71df4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6c073140 con 0x7f5a6c072b30 2024-04-20T10:55:18.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.797+0000 7f5a6b7fe640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f5a6c153600 0x7f5a6c0c6fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:55548/0 (socket says 172.21.15.137:55548) 2024-04-20T10:55:18.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.797+0000 7f5a6b7fe640 1 -- 172.21.15.137:0/1749667668 learned_addr learned my addr 172.21.15.137:0/1749667668 (peer_addr_for_me v1:172.21.15.137:0/0) 2024-04-20T10:55:18.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.797+0000 7f5a6bfff640 1 --2- 172.21.15.137:0/1749667668 >> v2:172.21.15.27:3300/0 conn(0x7f5a6c072b30 0x7f5a6c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3503515163 0 0) 0x7f5a6c073370 con 0x7f5a6c153600 2024-04-20T10:55:18.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a54003520 con 0x7f5a6c153600 2024-04-20T10:55:18.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 <== mon.1 v1:172.21.15.137:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3148467803 0 0) 0x7f5a54003520 con 0x7f5a6c153600 2024-04-20T10:55:18.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 >> v2:172.21.15.27:3300/0 conn(0x7f5a6c072b30 msgr2=0x7f5a6c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:18.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 --2- 172.21.15.137:0/1749667668 >> v2:172.21.15.27:3300/0 conn(0x7f5a6c072b30 0x7f5a6c06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 >> v1:172.21.15.27:6789/0 conn(0x7f5a6c159de0 legacy=0x7f5a6c072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 --> v1:172.21.15.137:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c073610 con 0x7f5a6c153600 2024-04-20T10:55:18.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 <== mon.1 v1:172.21.15.137:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f5a60001670 con 0x7f5a6c153600 2024-04-20T10:55:18.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6bfff640 1 --2- 172.21.15.137:0/1749667668 >> v2:172.21.15.27:3300/0 conn(0x7f5a6c072b30 0x7f5a6c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:18.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 >> v1:172.21.15.137:6789/0 conn(0x7f5a6c153600 legacy=0x7f5a6c0c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a540039b0 0x7f5a54003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a540051c0 0x7f5a5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5400c040 con 0x7f5a540051c0 2024-04-20T10:55:18.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5400c1f0 con 0x7f5a54004800 2024-04-20T10:55:18.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5400c3f0 con 0x7f5a540039b0 2024-04-20T10:55:18.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6b7fe640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a540039b0 0x7f5a54003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6affd640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6bfff640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a540051c0 0x7f5a5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6affd640 1 -- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a540039b0 msgr2=0x7f5a54003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:18.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6affd640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a540039b0 0x7f5a54003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6affd640 1 -- 172.21.15.137:0/1749667668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a540051c0 msgr2=0x7f5a5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:18.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6affd640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a540051c0 0x7f5a5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6affd640 1 -- 172.21.15.137:0/1749667668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a54003520 con 0x7f5a54004800 2024-04-20T10:55:18.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.798+0000 7f5a6bfff640 1 -- 172.21.15.137:0/1749667668 reap_dead start 2024-04-20T10:55:18.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.799+0000 7f5a6affd640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a54004c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c001750 tx=0x7f5a54005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d8ef5ac330cb686 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:18.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.799+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c001370 con 0x7f5a54004800 2024-04-20T10:55:18.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.799+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c0052d0 con 0x7f5a54004800 2024-04-20T10:55:18.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.799+0000 7f5a6a7fc640 1 -- 172.21.15.137:0/1749667668 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c003830 con 0x7f5a54004800 2024-04-20T10:55:18.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.800+0000 7f5a71df4640 1 -- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 msgr2=0x7f5a54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.800+0000 7f5a71df4640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a54004c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c001750 tx=0x7f5a54005830 comp rx=0 tx=0).stop 2024-04-20T10:55:18.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.800+0000 7f5a71df4640 1 -- 172.21.15.137:0/1749667668 shutdown_connections 2024-04-20T10:55:18.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.800+0000 7f5a71df4640 1 --2- 172.21.15.137:0/1749667668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a54004c80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.800+0000 7f5a71df4640 1 -- 172.21.15.137:0/1749667668 >> 172.21.15.137:0/1749667668 conn(0x7f5a6c0cd9c0 msgr2=0x7f5a6c0cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:18.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.800+0000 7f5a71df4640 1 -- 172.21.15.137:0/1749667668 shutdown_connections 2024-04-20T10:55:18.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.800+0000 7f5a71df4640 1 -- 172.21.15.137:0/1749667668 wait complete. 2024-04-20T10:55:18.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.801+0000 7f5a71df4640 1 Processor -- start 2024-04-20T10:55:18.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.801+0000 7f5a71df4640 1 -- start start 2024-04-20T10:55:18.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a71df4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a6c1f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a71df4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a6c0c6420 0x7f5a6c0c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a71df4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c157900 0x7f5a6c0c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a71df4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1598c0 con 0x7f5a6c157900 2024-04-20T10:55:18.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a71df4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1595e0 con 0x7f5a54004800 2024-04-20T10:55:18.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a71df4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c06d9b0 con 0x7f5a6c0c6420 2024-04-20T10:55:18.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a6c1f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a6c1f1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42046/0 (socket says 172.21.15.137:42046) 2024-04-20T10:55:18.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6b7fe640 1 -- 172.21.15.137:0/1030234873 learned_addr learned my addr 172.21.15.137:0/1030234873 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:18.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6affd640 1 --2- 172.21.15.137:0/1030234873 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a6c0c6420 0x7f5a6c0c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6bfff640 1 --2- 172.21.15.137:0/1030234873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c157900 0x7f5a6c0c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6b7fe640 1 -- 172.21.15.137:0/1030234873 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a6c0c6420 msgr2=0x7f5a6c0c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6b7fe640 1 --2- 172.21.15.137:0/1030234873 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a6c0c6420 0x7f5a6c0c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6b7fe640 1 -- 172.21.15.137:0/1030234873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c157900 msgr2=0x7f5a6c0c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6b7fe640 1 --2- 172.21.15.137:0/1030234873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c157900 0x7f5a6c0c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6b7fe640 1 -- 172.21.15.137:0/1030234873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c0c6a30 con 0x7f5a54004800 2024-04-20T10:55:18.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a6b7fe640 1 --2- 172.21.15.137:0/1030234873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a6c1f1850 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5a60001050 tx=0x7f5a60007720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:18.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.802+0000 7f5a53fff640 1 -- 172.21.15.137:0/1030234873 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000f860 con 0x7f5a54004800 2024-04-20T10:55:18.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.803+0000 7f5a53fff640 1 -- 172.21.15.137:0/1030234873 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a60018070 con 0x7f5a54004800 2024-04-20T10:55:18.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.803+0000 7f5a71df4640 1 -- 172.21.15.137:0/1030234873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c0c6cd0 con 0x7f5a54004800 2024-04-20T10:55:18.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.803+0000 7f5a53fff640 1 -- 172.21.15.137:0/1030234873 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000fc40 con 0x7f5a54004800 2024-04-20T10:55:18.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.803+0000 7f5a71df4640 1 -- 172.21.15.137:0/1030234873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c0c8730 con 0x7f5a54004800 2024-04-20T10:55:18.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.803+0000 7f5a53fff640 1 -- 172.21.15.137:0/1030234873 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5a6001d460 con 0x7f5a54004800 2024-04-20T10:55:18.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.803+0000 7f5a53fff640 1 -- 172.21.15.137:0/1030234873 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f5a60013a10 con 0x7f5a54004800 2024-04-20T10:55:18.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.804+0000 7f5a71df4640 1 -- 172.21.15.137:0/1030234873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6c04fcf0 con 0x7f5a54004800 2024-04-20T10:55:18.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.806+0000 7f5a53fff640 1 -- 172.21.15.137:0/1030234873 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f5a60013cb0 con 0x7f5a54004800 2024-04-20T10:55:18.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.877+0000 7f5a71df4640 1 -- 172.21.15.137:0/1030234873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d33806ff-f093-4414-8a74-f48775c0e13d", "id": 3} v 0) -- 0x7f5a6c072f20 con 0x7f5a54004800 2024-04-20T10:55:18.882 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:18.882+0000 7f5b56e3f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:55:18.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.882+0000 7f5a53fff640 1 -- 172.21.15.137:0/1030234873 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d33806ff-f093-4414-8a74-f48775c0e13d", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f5a60013580 con 0x7f5a54004800 2024-04-20T10:55:18.883 INFO:teuthology.orchestra.run.smithi137.stdout:3 2024-04-20T10:55:18.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.883+0000 7f5a71df4640 1 -- 172.21.15.137:0/1030234873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 msgr2=0x7f5a6c1f1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.883+0000 7f5a71df4640 1 --2- 172.21.15.137:0/1030234873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a6c1f1850 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5a60001050 tx=0x7f5a60007720 comp rx=0 tx=0).stop 2024-04-20T10:55:18.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.883+0000 7f5a71df4640 1 -- 172.21.15.137:0/1030234873 shutdown_connections 2024-04-20T10:55:18.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.883+0000 7f5a71df4640 1 --2- 172.21.15.137:0/1030234873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a6c157900 0x7f5a6c0c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.883+0000 7f5a71df4640 1 --2- 172.21.15.137:0/1030234873 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a6c0c6420 0x7f5a6c0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.883+0000 7f5a71df4640 1 --2- 172.21.15.137:0/1030234873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a54004800 0x7f5a6c1f1850 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.883+0000 7f5a71df4640 1 -- 172.21.15.137:0/1030234873 >> 172.21.15.137:0/1030234873 conn(0x7f5a6c0cd9c0 msgr2=0x7f5a6c152460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.883+0000 7f5a71df4640 1 -- 172.21.15.137:0/1030234873 shutdown_connections 2024-04-20T10:55:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.883+0000 7f5a71df4640 1 -- 172.21.15.137:0/1030234873 wait complete. 2024-04-20T10:55:18.894 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new ace9e9ac-5375-4880-b660-097a18353f38 4 2024-04-20T10:55:18.906 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:18.905+0000 7f242960d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:55:18.975 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:18.975+0000 7f5b56e3f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:55:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.990+0000 7fbe9ca8c640 1 Processor -- start 2024-04-20T10:55:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.990+0000 7fbe9ca8c640 1 -- start start 2024-04-20T10:55:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.991+0000 7fbe9ca8c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe98152280 0x7fbe980c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.991+0000 7fbe9ca8c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fbe980c8ac0 0x7fbe980c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.991+0000 7fbe9ca8c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe98159080 con 0x7fbe980c9d80 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.991+0000 7fbe9ca8c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9815b870 con 0x7fbe980c8ac0 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.991+0000 7fbe9ca8c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbe98158500 con 0x7fbe98152280 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe9659c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe98152280 0x7fbe980c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe96d9d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fbe980c8ac0 0x7fbe980c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe9659c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe98152280 0x7fbe980c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49382/0 (socket says 172.21.15.137:49382) 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe96d9d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fbe980c8ac0 0x7fbe980c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42056/0 (socket says 172.21.15.137:42056) 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe9659c640 1 -- 172.21.15.137:0/3796153065 learned_addr learned my addr 172.21.15.137:0/3796153065 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe9659c640 1 -- 172.21.15.137:0/3796153065 >> v2:172.21.15.137:3300/0 conn(0x7fbe980c8ac0 msgr2=0x7fbe980c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe9659c640 1 --2- 172.21.15.137:0/3796153065 >> v2:172.21.15.137:3300/0 conn(0x7fbe980c8ac0 0x7fbe980c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe9659c640 1 -- 172.21.15.137:0/3796153065 >> v1:172.21.15.27:6789/0 conn(0x7fbe980c9d80 legacy=0x7fbe9815bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe9659c640 1 -- 172.21.15.137:0/3796153065 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe981587e0 con 0x7fbe98152280 2024-04-20T10:55:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.992+0000 7fbe96d9d640 1 --2- 172.21.15.137:0/3796153065 >> v2:172.21.15.137:3300/0 conn(0x7fbe980c8ac0 0x7fbe980c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.993+0000 7fbe9659c640 1 --2- 172.21.15.137:0/3796153065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe98152280 0x7fbe980c9840 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009870 tx=0x7fbe8002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f68ce90f609e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.993+0000 7fbe9559a640 1 -- 172.21.15.137:0/3796153065 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80034070 con 0x7fbe98152280 2024-04-20T10:55:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.993+0000 7fbe9559a640 1 -- 172.21.15.137:0/3796153065 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe80030840 con 0x7fbe98152280 2024-04-20T10:55:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.993+0000 7fbe9559a640 1 -- 172.21.15.137:0/3796153065 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80030c10 con 0x7fbe98152280 2024-04-20T10:55:18.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.993+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3796153065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe98152280 msgr2=0x7fbe980c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:18.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.993+0000 7fbe9ca8c640 1 --2- 172.21.15.137:0/3796153065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe98152280 0x7fbe980c9840 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009870 tx=0x7fbe8002b980 comp rx=0 tx=0).stop 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.994+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3796153065 shutdown_connections 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.994+0000 7fbe9ca8c640 1 --2- 172.21.15.137:0/3796153065 >> v2:172.21.15.137:3300/0 conn(0x7fbe980c8ac0 0x7fbe980c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.994+0000 7fbe9ca8c640 1 --2- 172.21.15.137:0/3796153065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe98152280 0x7fbe980c9840 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.994+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3796153065 >> 172.21.15.137:0/3796153065 conn(0x7fbe980ca610 msgr2=0x7fbe980caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.994+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3796153065 shutdown_connections 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.994+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3796153065 wait complete. 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.995+0000 7fbe9ca8c640 1 Processor -- start 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.995+0000 7fbe9ca8c640 1 -- start start 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9ca8c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe980c8ac0 0x7fbe9815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9ca8c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe980c9d80 0x7fbe9815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9ca8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe98152280 0x7fbe981f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9ca8c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981580c0 con 0x7fbe980c9d80 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9ca8c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98158da0 con 0x7fbe98152280 2024-04-20T10:55:18.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9ca8c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbe98158240 con 0x7fbe980c8ac0 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9659c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe980c8ac0 0x7fbe9815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9659c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe980c8ac0 0x7fbe9815a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49388/0 (socket says 172.21.15.137:49388) 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe96d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe98152280 0x7fbe981f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9659c640 1 -- 172.21.15.137:0/3203510157 learned_addr learned my addr 172.21.15.137:0/3203510157 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe95d9b640 1 --2- 172.21.15.137:0/3203510157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe980c9d80 0x7fbe9815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9659c640 1 -- 172.21.15.137:0/3203510157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe98152280 msgr2=0x7fbe981f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9659c640 1 --2- 172.21.15.137:0/3203510157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe98152280 0x7fbe981f4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9659c640 1 -- 172.21.15.137:0/3203510157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe980c9d80 msgr2=0x7fbe9815acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9659c640 1 --2- 172.21.15.137:0/3203510157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe980c9d80 0x7fbe9815acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9659c640 1 -- 172.21.15.137:0/3203510157 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe98072eb0 con 0x7fbe980c8ac0 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9659c640 1 --2- 172.21.15.137:0/3203510157 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe980c8ac0 0x7fbe9815a0d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe8002b9c0 tx=0x7fbe800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe8effd640 1 -- 172.21.15.137:0/3203510157 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80034040 con 0x7fbe980c8ac0 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe8effd640 1 -- 172.21.15.137:0/3203510157 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe80030640 con 0x7fbe980c8ac0 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3203510157 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe980739d0 con 0x7fbe980c8ac0 2024-04-20T10:55:19.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe8effd640 1 -- 172.21.15.137:0/3203510157 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe800423f0 con 0x7fbe980c8ac0 2024-04-20T10:55:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.996+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3203510157 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe98073410 con 0x7fbe980c8ac0 2024-04-20T10:55:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.997+0000 7fbe8effd640 1 -- 172.21.15.137:0/3203510157 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbe80037070 con 0x7fbe980c8ac0 2024-04-20T10:55:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.997+0000 7fbe8effd640 1 -- 172.21.15.137:0/3203510157 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fbe80042d00 con 0x7fbe980c8ac0 2024-04-20T10:55:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:18.998+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3203510157 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9804fcf0 con 0x7fbe980c8ac0 2024-04-20T10:55:19.001 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:18.997+0000 7f242960d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:55:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.000+0000 7fbe8effd640 1 -- 172.21.15.137:0/3203510157 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fbe98072eb0 con 0x7fbe980c8ac0 2024-04-20T10:55:19.029 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:19.028+0000 7f5b56e3f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:55:19.043 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:19.042+0000 7f242960d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:55:19.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.081+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3203510157 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ace9e9ac-5375-4880-b660-097a18353f38", "id": 4} v 0) -- 0x7fbe980cae30 con 0x7fbe980c8ac0 2024-04-20T10:55:19.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.086+0000 7fbe8effd640 1 -- 172.21.15.137:0/3203510157 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ace9e9ac-5375-4880-b660-097a18353f38", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fbe800307f0 con 0x7fbe980c8ac0 2024-04-20T10:55:19.087 INFO:teuthology.orchestra.run.smithi137.stdout:4 2024-04-20T10:55:19.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.087+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3203510157 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe980c8ac0 msgr2=0x7fbe9815a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.087+0000 7fbe9ca8c640 1 --2- 172.21.15.137:0/3203510157 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe980c8ac0 0x7fbe9815a0d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe8002b9c0 tx=0x7fbe800046c0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.087+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3203510157 shutdown_connections 2024-04-20T10:55:19.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.087+0000 7fbe9ca8c640 1 --2- 172.21.15.137:0/3203510157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe98152280 0x7fbe981f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.087+0000 7fbe9ca8c640 1 --2- 172.21.15.137:0/3203510157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe980c9d80 0x7fbe9815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.087+0000 7fbe9ca8c640 1 --2- 172.21.15.137:0/3203510157 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe980c8ac0 0x7fbe9815a0d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.087+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3203510157 >> 172.21.15.137:0/3203510157 conn(0x7fbe980ca610 msgr2=0x7fbe9814f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:19.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.087+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3203510157 shutdown_connections 2024-04-20T10:55:19.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.087+0000 7fbe9ca8c640 1 -- 172.21.15.137:0/3203510157 wait complete. 2024-04-20T10:55:19.099 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new aad00479-b2ec-4539-8260-774ffc288dc4 5 2024-04-20T10:55:19.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.194+0000 7f0146274640 1 Processor -- start 2024-04-20T10:55:19.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.194+0000 7f0146274640 1 -- start start 2024-04-20T10:55:19.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f0146274640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f01400c2ea0 0x7f014015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f0146274640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01400ca230 0x7f01400ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f0146274640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01400c42b0 con 0x7f0140155790 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f0146274640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f01400c4870 con 0x7f01400c2ea0 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f0146274640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f01400c4590 con 0x7f01400ca230 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f0144811640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01400ca230 0x7f01400ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f013f7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f01400c2ea0 0x7f014015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f013f7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f01400c2ea0 0x7f014015f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42084/0 (socket says 172.21.15.137:42084) 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f0144811640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01400ca230 0x7f01400ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49402/0 (socket says 172.21.15.137:49402) 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f013ffff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f0140155790 0x7f014015beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:55558/0 (socket says 172.21.15.137:55558) 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.195+0000 7f013f7fe640 1 -- 172.21.15.137:0/1028212291 learned_addr learned my addr 172.21.15.137:0/1028212291 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.196+0000 7f013f7fe640 1 -- 172.21.15.137:0/1028212291 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01400ca230 msgr2=0x7f01400ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.196+0000 7f013f7fe640 1 --2- 172.21.15.137:0/1028212291 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01400ca230 0x7f01400ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.196+0000 7f013f7fe640 1 -- 172.21.15.137:0/1028212291 >> v1:172.21.15.137:6789/0 conn(0x7f0140155790 legacy=0x7f014015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.196+0000 7f013f7fe640 1 -- 172.21.15.137:0/1028212291 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014015b6b0 con 0x7f01400c2ea0 2024-04-20T10:55:19.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.196+0000 7f013effd640 1 -- 172.21.15.137:0/1028212291 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 992145108 0 0) 0x7f01400c42b0 con 0x7f0140155790 2024-04-20T10:55:19.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.196+0000 7f0144811640 1 --2- 172.21.15.137:0/1028212291 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01400ca230 0x7f01400ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:19.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.196+0000 7f013f7fe640 1 --2- 172.21.15.137:0/1028212291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 0x7f014015f4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0130009870 tx=0x7f013002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17b1efe3e7315ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.196+0000 7f013effd640 1 -- 172.21.15.137:0/1028212291 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130034070 con 0x7f01400c2ea0 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f013effd640 1 -- 172.21.15.137:0/1028212291 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0130030840 con 0x7f01400c2ea0 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f013effd640 1 -- 172.21.15.137:0/1028212291 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130030c10 con 0x7f01400c2ea0 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f0146274640 1 -- 172.21.15.137:0/1028212291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 msgr2=0x7f014015f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f0146274640 1 --2- 172.21.15.137:0/1028212291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 0x7f014015f4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0130009870 tx=0x7f013002b980 comp rx=0 tx=0).stop 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f0146274640 1 -- 172.21.15.137:0/1028212291 shutdown_connections 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f0146274640 1 --2- 172.21.15.137:0/1028212291 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01400ca230 0x7f01400ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f0146274640 1 --2- 172.21.15.137:0/1028212291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 0x7f014015f4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f0146274640 1 -- 172.21.15.137:0/1028212291 >> 172.21.15.137:0/1028212291 conn(0x7f014006fb50 msgr2=0x7f014006ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f0146274640 1 -- 172.21.15.137:0/1028212291 shutdown_connections 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.197+0000 7f0146274640 1 -- 172.21.15.137:0/1028212291 wait complete. 2024-04-20T10:55:19.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.198+0000 7f0146274640 1 Processor -- start 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f0146274640 1 -- start start 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f0146274640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 0x7f0140150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f0146274640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01400ca230 0x7f0140151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f0146274640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0140155790 0x7f014015dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f0146274640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0140155b70 con 0x7f01400ca230 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f0146274640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01400c4c20 con 0x7f01400c2ea0 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f0146274640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f0140071f60 con 0x7f0140155790 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f013ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 0x7f0140150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f013ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 0x7f0140150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42098/0 (socket says 172.21.15.137:42098) 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f013ffff640 1 -- 172.21.15.137:0/1819936031 learned_addr learned my addr 172.21.15.137:0/1819936031 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f0144811640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0140155790 0x7f014015dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.199+0000 7f013f7fe640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01400ca230 0x7f0140151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f0144811640 1 -- 172.21.15.137:0/1819936031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 msgr2=0x7f0140150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f0144811640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 0x7f0140150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f0144811640 1 -- 172.21.15.137:0/1819936031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01400ca230 msgr2=0x7f0140151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f0144811640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01400ca230 0x7f0140151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f0144811640 1 -- 172.21.15.137:0/1819936031 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01401f1150 con 0x7f0140155790 2024-04-20T10:55:19.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f013f7fe640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01400ca230 0x7f0140151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:19.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f0144811640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0140155790 0x7f014015dcd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0138009e50 tx=0x7f013800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:19.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f013cff9640 1 -- 172.21.15.137:0/1819936031 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138018030 con 0x7f0140155790 2024-04-20T10:55:19.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f013cff9640 1 -- 172.21.15.137:0/1819936031 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f013800cd90 con 0x7f0140155790 2024-04-20T10:55:19.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f0146274640 1 -- 172.21.15.137:0/1819936031 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01401f1410 con 0x7f0140155790 2024-04-20T10:55:19.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f013cff9640 1 -- 172.21.15.137:0/1819936031 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138003bb0 con 0x7f0140155790 2024-04-20T10:55:19.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f0146274640 1 -- 172.21.15.137:0/1819936031 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f01401f19d0 con 0x7f0140155790 2024-04-20T10:55:19.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f013cff9640 1 -- 172.21.15.137:0/1819936031 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0138005d10 con 0x7f0140155790 2024-04-20T10:55:19.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.200+0000 7f013cff9640 1 -- 172.21.15.137:0/1819936031 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f013801da30 con 0x7f0140155790 2024-04-20T10:55:19.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.201+0000 7f0146274640 1 -- 172.21.15.137:0/1819936031 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014004fcf0 con 0x7f0140155790 2024-04-20T10:55:19.206 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:19.202+0000 7f5b56e3f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:55:19.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.203+0000 7f013cff9640 1 -- 172.21.15.137:0/1819936031 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f0138022070 con 0x7f0140155790 2024-04-20T10:55:19.216 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:19.215+0000 7f242960d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:55:19.268 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:19.267+0000 7f5b56e3f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:55:19.273 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:19.272+0000 7f242960d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:55:19.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.275+0000 7f0146274640 1 -- 172.21.15.137:0/1819936031 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "aad00479-b2ec-4539-8260-774ffc288dc4", "id": 5} v 0) -- 0x7f01400cd8b0 con 0x7f0140155790 2024-04-20T10:55:19.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.281+0000 7f013cff9640 1 -- 172.21.15.137:0/1819936031 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aad00479-b2ec-4539-8260-774ffc288dc4", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f0138010040 con 0x7f0140155790 2024-04-20T10:55:19.282 INFO:teuthology.orchestra.run.smithi137.stdout:5 2024-04-20T10:55:19.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.282+0000 7f0146274640 1 -- 172.21.15.137:0/1819936031 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0140155790 msgr2=0x7f014015dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.282+0000 7f0146274640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0140155790 0x7f014015dcd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0138009e50 tx=0x7f013800c490 comp rx=0 tx=0).stop 2024-04-20T10:55:19.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.282+0000 7f0146274640 1 -- 172.21.15.137:0/1819936031 shutdown_connections 2024-04-20T10:55:19.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.282+0000 7f0146274640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0140155790 0x7f014015dcd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.282+0000 7f0146274640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01400ca230 0x7f0140151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.282+0000 7f0146274640 1 --2- 172.21.15.137:0/1819936031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01400c2ea0 0x7f0140150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.282+0000 7f0146274640 1 -- 172.21.15.137:0/1819936031 >> 172.21.15.137:0/1819936031 conn(0x7f014006fb50 msgr2=0x7f01400cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:19.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.282+0000 7f0146274640 1 -- 172.21.15.137:0/1819936031 shutdown_connections 2024-04-20T10:55:19.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.282+0000 7f0146274640 1 -- 172.21.15.137:0/1819936031 wait complete. 2024-04-20T10:55:19.293 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new a220d407-a929-41f9-9f73-4044163f0f68 6 2024-04-20T10:55:19.324 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:19.323+0000 7f5b56e3f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:55:19.328 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:19.328+0000 7f242960d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:55:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.389+0000 7f08cf43f640 1 Processor -- start 2024-04-20T10:55:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.390+0000 7f08cf43f640 1 -- start start 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.390+0000 7f08cf43f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f08c81522f0 0x7f08c80723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.390+0000 7f08cf43f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f08c815c150 0x7f08c8158880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.390+0000 7f08cf43f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c8072a40 con 0x7f08c8158dc0 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.390+0000 7f08cf43f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f08c81583f0 con 0x7f08c81522f0 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.390+0000 7f08cf43f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f08c8072f40 con 0x7f08c815c150 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.391+0000 7f08cc9da640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f08c815c150 0x7f08c8158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.391+0000 7f08cc9da640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f08c815c150 0x7f08c8158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42102/0 (socket says 172.21.15.137:42102) 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.391+0000 7f08cc9da640 1 -- 172.21.15.137:0/3277418885 learned_addr learned my addr 172.21.15.137:0/3277418885 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.391+0000 7f08cd1db640 1 --2- 172.21.15.137:0/3277418885 >> v2:172.21.15.27:3300/0 conn(0x7f08c81522f0 0x7f08c80723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.391+0000 7f08cc9da640 1 -- 172.21.15.137:0/3277418885 >> v2:172.21.15.27:3300/0 conn(0x7f08c81522f0 msgr2=0x7f08c80723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:19.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.391+0000 7f08cc9da640 1 --2- 172.21.15.137:0/3277418885 >> v2:172.21.15.27:3300/0 conn(0x7f08c81522f0 0x7f08c80723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.391+0000 7f08cc9da640 1 -- 172.21.15.137:0/3277418885 >> v1:172.21.15.27:6789/0 conn(0x7f08c8158dc0 legacy=0x7f08c806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.391+0000 7f08cc9da640 1 -- 172.21.15.137:0/3277418885 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c80731a0 con 0x7f08c815c150 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08cc9da640 1 --2- 172.21.15.137:0/3277418885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 0x7f08c8158880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f08c80ca3f0 tx=0x7f08c402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f060fa691c46358 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08bbfff640 1 -- 172.21.15.137:0/3277418885 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4034070 con 0x7f08c815c150 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08bbfff640 1 -- 172.21.15.137:0/3277418885 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c4030860 con 0x7f08c815c150 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08bbfff640 1 -- 172.21.15.137:0/3277418885 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4030c30 con 0x7f08c815c150 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08cf43f640 1 -- 172.21.15.137:0/3277418885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 msgr2=0x7f08c8158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08cf43f640 1 --2- 172.21.15.137:0/3277418885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 0x7f08c8158880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f08c80ca3f0 tx=0x7f08c402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08cf43f640 1 -- 172.21.15.137:0/3277418885 shutdown_connections 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08cf43f640 1 --2- 172.21.15.137:0/3277418885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 0x7f08c8158880 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08cf43f640 1 --2- 172.21.15.137:0/3277418885 >> v2:172.21.15.27:3300/0 conn(0x7f08c81522f0 0x7f08c80723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.392+0000 7f08cf43f640 1 -- 172.21.15.137:0/3277418885 >> 172.21.15.137:0/3277418885 conn(0x7f08c80cd9c0 msgr2=0x7f08c80cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.393+0000 7f08cf43f640 1 -- 172.21.15.137:0/3277418885 shutdown_connections 2024-04-20T10:55:19.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.393+0000 7f08cf43f640 1 -- 172.21.15.137:0/3277418885 wait complete. 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cf43f640 1 Processor -- start 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cf43f640 1 -- start start 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cf43f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08c81522f0 0x7f08c80c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cf43f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c8158dc0 0x7f08c81550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cf43f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 0x7f08c8155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cf43f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f08c815c580 con 0x7f08c8158dc0 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cf43f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08c80c49e0 con 0x7f08c815c150 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cf43f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f08c81526d0 con 0x7f08c81522f0 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cd1db640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08c81522f0 0x7f08c80c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cd9dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 0x7f08c8155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cd1db640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08c81522f0 0x7f08c80c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49418/0 (socket says 172.21.15.137:49418) 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cd1db640 1 -- 172.21.15.137:0/599736998 learned_addr learned my addr 172.21.15.137:0/599736998 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cd9dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 0x7f08c8155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42110/0 (socket says 172.21.15.137:42110) 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.394+0000 7f08cc9da640 1 --2- 172.21.15.137:0/599736998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c8158dc0 0x7f08c81550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08cd9dc640 1 -- 172.21.15.137:0/599736998 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08c81522f0 msgr2=0x7f08c80c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08cd9dc640 1 --2- 172.21.15.137:0/599736998 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08c81522f0 0x7f08c80c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08cd9dc640 1 -- 172.21.15.137:0/599736998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c8158dc0 msgr2=0x7f08c81550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08cd9dc640 1 --2- 172.21.15.137:0/599736998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c8158dc0 0x7f08c81550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08cd9dc640 1 -- 172.21.15.137:0/599736998 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c81f20b0 con 0x7f08c815c150 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08cd1db640 1 --2- 172.21.15.137:0/599736998 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08c81522f0 0x7f08c80c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08cd9dc640 1 --2- 172.21.15.137:0/599736998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 0x7f08c8155630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f08c000c6f0 tx=0x7f08c000cbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08b9ffb640 1 -- 172.21.15.137:0/599736998 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0003970 con 0x7f08c815c150 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08b9ffb640 1 -- 172.21.15.137:0/599736998 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c0003b10 con 0x7f08c815c150 2024-04-20T10:55:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08cf43f640 1 -- 172.21.15.137:0/599736998 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c8154090 con 0x7f08c815c150 2024-04-20T10:55:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08b9ffb640 1 -- 172.21.15.137:0/599736998 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c000a850 con 0x7f08c815c150 2024-04-20T10:55:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08cf43f640 1 -- 172.21.15.137:0/599736998 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c8154610 con 0x7f08c815c150 2024-04-20T10:55:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08b9ffb640 1 -- 172.21.15.137:0/599736998 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f08c000a9f0 con 0x7f08c815c150 2024-04-20T10:55:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.395+0000 7f08b9ffb640 1 -- 172.21.15.137:0/599736998 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f08c0012a60 con 0x7f08c815c150 2024-04-20T10:55:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.397+0000 7f08cf43f640 1 -- 172.21.15.137:0/599736998 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c804fcf0 con 0x7f08c815c150 2024-04-20T10:55:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.398+0000 7f08b9ffb640 1 -- 172.21.15.137:0/599736998 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f08c0012d00 con 0x7f08c815c150 2024-04-20T10:55:19.443 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:19.443+0000 7f5b56e3f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:55:19.447 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:19.447+0000 7f242960d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:55:19.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.471+0000 7f08cf43f640 1 -- 172.21.15.137:0/599736998 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a220d407-a929-41f9-9f73-4044163f0f68", "id": 6} v 0) -- 0x7f08c8155b70 con 0x7f08c815c150 2024-04-20T10:55:19.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.476+0000 7f08b9ffb640 1 -- 172.21.15.137:0/599736998 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a220d407-a929-41f9-9f73-4044163f0f68", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f08c0012470 con 0x7f08c815c150 2024-04-20T10:55:19.477 INFO:teuthology.orchestra.run.smithi137.stdout:6 2024-04-20T10:55:19.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.478+0000 7f08cf43f640 1 -- 172.21.15.137:0/599736998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 msgr2=0x7f08c8155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.478+0000 7f08cf43f640 1 --2- 172.21.15.137:0/599736998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 0x7f08c8155630 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f08c000c6f0 tx=0x7f08c000cbb0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.478+0000 7f08cf43f640 1 -- 172.21.15.137:0/599736998 shutdown_connections 2024-04-20T10:55:19.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.478+0000 7f08cf43f640 1 --2- 172.21.15.137:0/599736998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c815c150 0x7f08c8155630 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.478+0000 7f08cf43f640 1 --2- 172.21.15.137:0/599736998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08c8158dc0 0x7f08c81550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.478+0000 7f08cf43f640 1 --2- 172.21.15.137:0/599736998 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08c81522f0 0x7f08c80c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.478+0000 7f08cf43f640 1 -- 172.21.15.137:0/599736998 >> 172.21.15.137:0/599736998 conn(0x7f08c80cd9c0 msgr2=0x7f08c80cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:19.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.478+0000 7f08cf43f640 1 -- 172.21.15.137:0/599736998 shutdown_connections 2024-04-20T10:55:19.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.478+0000 7f08cf43f640 1 -- 172.21.15.137:0/599736998 wait complete. 2024-04-20T10:55:19.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 14cc12e6-57c0-42bf-8b76-aca9b5edb063 7 2024-04-20T10:55:19.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.626+0000 7f8585ea5640 1 Processor -- start 2024-04-20T10:55:19.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.626+0000 7f8585ea5640 1 -- start start 2024-04-20T10:55:19.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f8585ea5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f85800c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f8585ea5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85801594a0 0x7f85801598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f8585ea5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85800c9c40 con 0x7f858015d150 2024-04-20T10:55:19.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f8585ea5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8580159e30 con 0x7f85801594a0 2024-04-20T10:55:19.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f8585ea5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f85800c9680 con 0x7f8580153530 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f85800c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f85800c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49434/0 (socket says 172.21.15.137:49434) 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f857f7fe640 1 -- 172.21.15.137:0/155220164 learned_addr learned my addr 172.21.15.137:0/155220164 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f857ffff640 1 --2- 172.21.15.137:0/155220164 >> v2:172.21.15.27:3300/0 conn(0x7f85801594a0 0x7f85801598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f857f7fe640 1 -- 172.21.15.137:0/155220164 >> v2:172.21.15.27:3300/0 conn(0x7f85801594a0 msgr2=0x7f85801598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f857f7fe640 1 --2- 172.21.15.137:0/155220164 >> v2:172.21.15.27:3300/0 conn(0x7f85801594a0 0x7f85801598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f857f7fe640 1 -- 172.21.15.137:0/155220164 >> v1:172.21.15.27:6789/0 conn(0x7f858015d150 legacy=0x7f858006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f857f7fe640 1 -- 172.21.15.137:0/155220164 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85800c9960 con 0x7f8580153530 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.627+0000 7f857ffff640 1 --2- 172.21.15.137:0/155220164 >> v2:172.21.15.27:3300/0 conn(0x7f85801594a0 0x7f85801598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.628+0000 7f857f7fe640 1 --2- 172.21.15.137:0/155220164 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f85800c3be0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8574009870 tx=0x7f857402ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc7ce3946496745 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.628+0000 7f857e7fc640 1 -- 172.21.15.137:0/155220164 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857402f840 con 0x7f8580153530 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.628+0000 7f857e7fc640 1 -- 172.21.15.137:0/155220164 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857402f9e0 con 0x7f8580153530 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.628+0000 7f857e7fc640 1 -- 172.21.15.137:0/155220164 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857402fdb0 con 0x7f8580153530 2024-04-20T10:55:19.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.628+0000 7f8585ea5640 1 -- 172.21.15.137:0/155220164 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 msgr2=0x7f85800c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.628+0000 7f8585ea5640 1 --2- 172.21.15.137:0/155220164 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f85800c3be0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8574009870 tx=0x7f857402ba20 comp rx=0 tx=0).stop 2024-04-20T10:55:19.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.629+0000 7f8585ea5640 1 -- 172.21.15.137:0/155220164 shutdown_connections 2024-04-20T10:55:19.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.629+0000 7f8585ea5640 1 --2- 172.21.15.137:0/155220164 >> v2:172.21.15.27:3300/0 conn(0x7f85801594a0 0x7f85801598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.629+0000 7f8585ea5640 1 --2- 172.21.15.137:0/155220164 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f85800c3be0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.629+0000 7f8585ea5640 1 -- 172.21.15.137:0/155220164 >> 172.21.15.137:0/155220164 conn(0x7f85800678a0 msgr2=0x7f8580067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:19.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.629+0000 7f8585ea5640 1 -- 172.21.15.137:0/155220164 shutdown_connections 2024-04-20T10:55:19.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.629+0000 7f8585ea5640 1 -- 172.21.15.137:0/155220164 wait complete. 2024-04-20T10:55:19.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.630+0000 7f8585ea5640 1 Processor -- start 2024-04-20T10:55:19.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.630+0000 7f8585ea5640 1 -- start start 2024-04-20T10:55:19.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.630+0000 7f8585ea5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f8580151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.630+0000 7f8585ea5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85801594a0 0x7f85801f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.630+0000 7f8585ea5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f858015d150 0x7f85801f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f8585ea5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f85800c5df0 con 0x7f85801594a0 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f8585ea5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f85800c5b10 con 0x7f858015d150 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f8585ea5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f858015d570 con 0x7f8580153530 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f8580151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f858015d150 0x7f85801f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f8580151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:49448/0 (socket says 172.21.15.137:49448) 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857f7fe640 1 -- 172.21.15.137:0/590086798 learned_addr learned my addr 172.21.15.137:0/590086798 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857effd640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85801594a0 0x7f85801f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857ffff640 1 -- 172.21.15.137:0/590086798 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 msgr2=0x7f8580151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857ffff640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f8580151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857ffff640 1 -- 172.21.15.137:0/590086798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85801594a0 msgr2=0x7f85801f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857ffff640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85801594a0 0x7f85801f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857ffff640 1 -- 172.21.15.137:0/590086798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85800cd510 con 0x7f858015d150 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857f7fe640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f8580151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857effd640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85801594a0 0x7f85801f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f857ffff640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f858015d150 0x7f85801f2020 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8578007e60 tx=0x7f857800d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f8567fff640 1 -- 172.21.15.137:0/590086798 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857801a070 con 0x7f858015d150 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f8567fff640 1 -- 172.21.15.137:0/590086798 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8578003b90 con 0x7f858015d150 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f8585ea5640 1 -- 172.21.15.137:0/590086798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801f4a20 con 0x7f858015d150 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f8567fff640 1 -- 172.21.15.137:0/590086798 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578004030 con 0x7f858015d150 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.631+0000 7f8585ea5640 1 -- 172.21.15.137:0/590086798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801f4fa0 con 0x7f858015d150 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.632+0000 7f8567fff640 1 -- 172.21.15.137:0/590086798 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f857801d070 con 0x7f858015d150 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.632+0000 7f8567fff640 1 -- 172.21.15.137:0/590086798 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f8578004830 con 0x7f858015d150 2024-04-20T10:55:19.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.632+0000 7f8585ea5640 1 -- 172.21.15.137:0/590086798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8580068c90 con 0x7f858015d150 2024-04-20T10:55:19.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.634+0000 7f8567fff640 1 -- 172.21.15.137:0/590086798 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f8580068c90 con 0x7f858015d150 2024-04-20T10:55:19.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.705+0000 7f8585ea5640 1 -- 172.21.15.137:0/590086798 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "14cc12e6-57c0-42bf-8b76-aca9b5edb063", "id": 7} v 0) -- 0x7f8580067c80 con 0x7f858015d150 2024-04-20T10:55:19.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.711+0000 7f8567fff640 1 -- 172.21.15.137:0/590086798 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "14cc12e6-57c0-42bf-8b76-aca9b5edb063", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f8578024020 con 0x7f858015d150 2024-04-20T10:55:19.712 INFO:teuthology.orchestra.run.smithi137.stdout:7 2024-04-20T10:55:19.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.712+0000 7f8585ea5640 1 -- 172.21.15.137:0/590086798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f858015d150 msgr2=0x7f85801f2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.712+0000 7f8585ea5640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f858015d150 0x7f85801f2020 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8578007e60 tx=0x7f857800d4a0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.712+0000 7f8585ea5640 1 -- 172.21.15.137:0/590086798 shutdown_connections 2024-04-20T10:55:19.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.712+0000 7f8585ea5640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f858015d150 0x7f85801f2020 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.712+0000 7f8585ea5640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85801594a0 0x7f85801f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.712+0000 7f8585ea5640 1 --2- 172.21.15.137:0/590086798 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8580153530 0x7f8580151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.712+0000 7f8585ea5640 1 -- 172.21.15.137:0/590086798 >> 172.21.15.137:0/590086798 conn(0x7f85800678a0 msgr2=0x7f85801510f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:19.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.712+0000 7f8585ea5640 1 -- 172.21.15.137:0/590086798 shutdown_connections 2024-04-20T10:55:19.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-20T10:55:19.712+0000 7f8585ea5640 1 -- 172.21.15.137:0/590086798 wait complete. 2024-04-20T10:55:19.723 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-20T10:55:19.723 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-20T10:55:19.725 INFO:tasks.ceph.osd.0:Started 2024-04-20T10:55:19.725 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-20T10:55:19.726 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-20T10:55:19.728 INFO:tasks.ceph.osd.1:Started 2024-04-20T10:55:19.728 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-20T10:55:19.728 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-20T10:55:19.730 INFO:tasks.ceph.osd.2:Started 2024-04-20T10:55:19.730 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-20T10:55:19.730 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-20T10:55:19.732 INFO:tasks.ceph.osd.3:Started 2024-04-20T10:55:19.732 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-20T10:55:19.733 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-20T10:55:19.735 INFO:tasks.ceph.osd.4:Started 2024-04-20T10:55:19.735 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-20T10:55:19.735 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-20T10:55:19.737 INFO:tasks.ceph.osd.5:Started 2024-04-20T10:55:19.737 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-20T10:55:19.737 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-20T10:55:19.739 INFO:tasks.ceph.osd.6:Started 2024-04-20T10:55:19.739 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-20T10:55:19.740 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-20T10:55:19.742 INFO:tasks.ceph.osd.7:Started 2024-04-20T10:55:19.742 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.866+0000 7f436b6c7640 1 Processor -- start 2024-04-20T10:55:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.866+0000 7f436b6c7640 1 -- start start 2024-04-20T10:55:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f436b6c7640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4364157fd0 0x7f436415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f436b6c7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436415c0e0 con 0x7f4364152400 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f436b6c7640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4364071950 con 0x7f436415bb00 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f436b6c7640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f4364071c30 con 0x7f4364157fd0 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f4369463640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4364152400 0x7f43640c7ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40958/0 (socket says 172.21.15.27:40958) 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f4369463640 1 -- 172.21.15.27:0/4250096801 learned_addr learned my addr 172.21.15.27:0/4250096801 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3288614560 0 0) 0x7f436415c0e0 con 0x7f4364152400 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4350003520 con 0x7f4364152400 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f4368c62640 1 --2- 172.21.15.27:0/4250096801 >> v2:172.21.15.137:3300/0 conn(0x7f4364157fd0 0x7f436415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4136342951 0 0) 0x7f4350003520 con 0x7f4364152400 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 >> v2:172.21.15.137:3300/0 conn(0x7f4364157fd0 msgr2=0x7f436415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f435bfff640 1 --2- 172.21.15.27:0/4250096801 >> v2:172.21.15.137:3300/0 conn(0x7f4364157fd0 0x7f436415b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 >> v1:172.21.15.137:6789/0 conn(0x7f436415bb00 legacy=0x7f436406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4364071f10 con 0x7f4364152400 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f4354001670 con 0x7f4364152400 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.867+0000 7f4368c62640 1 --2- 172.21.15.27:0/4250096801 >> v2:172.21.15.137:3300/0 conn(0x7f4364157fd0 0x7f436415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:19.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 >> v1:172.21.15.27:6789/0 conn(0x7f4364152400 legacy=0x7f43640c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f435bfff640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43500039b0 0x7f4350003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f435bfff640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4350004800 0x7f4350004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f435bfff640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43500051c0 0x7f435000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4364071c30 con 0x7f43500051c0 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f435000c110 con 0x7f4350004800 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f435000c310 con 0x7f43500039b0 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f4368c62640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4350004800 0x7f4350004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f4369c64640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43500051c0 0x7f435000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f4369463640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43500039b0 0x7f4350003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f4369c64640 1 -- 172.21.15.27:0/4250096801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43500039b0 msgr2=0x7f4350003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f4369c64640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43500039b0 0x7f4350003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f4369c64640 1 -- 172.21.15.27:0/4250096801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4350004800 msgr2=0x7f4350004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f4369c64640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4350004800 0x7f4350004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f4369c64640 1 -- 172.21.15.27:0/4250096801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4350003520 con 0x7f43500051c0 2024-04-20T10:55:19.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.868+0000 7f4369463640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43500039b0 0x7f4350003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.869+0000 7f4369c64640 1 -- 172.21.15.27:0/4250096801 reap_dead start 2024-04-20T10:55:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.869+0000 7f4369c64640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43500051c0 0x7f435000b9c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f435c001e50 tx=0x7f4350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edeb8d8e4f8db0b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354007d10 con 0x7f43500051c0 2024-04-20T10:55:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435c003b90 con 0x7f43500051c0 2024-04-20T10:55:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f435bfff640 1 -- 172.21.15.27:0/4250096801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c001a70 con 0x7f43500051c0 2024-04-20T10:55:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f436b6c7640 1 -- 172.21.15.27:0/4250096801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43500051c0 msgr2=0x7f435000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f436b6c7640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43500051c0 0x7f435000b9c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f435c001e50 tx=0x7f4350005830 comp rx=0 tx=0).stop 2024-04-20T10:55:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f436b6c7640 1 -- 172.21.15.27:0/4250096801 shutdown_connections 2024-04-20T10:55:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f436b6c7640 1 --2- 172.21.15.27:0/4250096801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43500051c0 0x7f435000b9c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f436b6c7640 1 -- 172.21.15.27:0/4250096801 >> 172.21.15.27:0/4250096801 conn(0x7f436406d570 msgr2=0x7f436406d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f436b6c7640 1 -- 172.21.15.27:0/4250096801 shutdown_connections 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.870+0000 7f436b6c7640 1 -- 172.21.15.27:0/4250096801 wait complete. 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.872+0000 7f436b6c7640 1 Processor -- start 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.872+0000 7f436b6c7640 1 -- start start 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.872+0000 7f436b6c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43500051c0 0x7f4364135f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.872+0000 7f436b6c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43641ef5c0 0x7f43640cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.872+0000 7f436b6c7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43640ccb90 0x7f436415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f436b6c7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4364072710 con 0x7f43641ef5c0 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f436b6c7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f436415c3c0 con 0x7f43500051c0 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f436b6c7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f4364072890 con 0x7f43640ccb90 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4368c62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43641ef5c0 0x7f43640cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4368c62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43641ef5c0 0x7f43640cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58642/0 (socket says 172.21.15.27:58642) 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4369463640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43500051c0 0x7f4364135f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4368c62640 1 -- 172.21.15.27:0/3558946758 learned_addr learned my addr 172.21.15.27:0/3558946758 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4369c64640 1 --2- 172.21.15.27:0/3558946758 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43640ccb90 0x7f436415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4368c62640 1 -- 172.21.15.27:0/3558946758 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43640ccb90 msgr2=0x7f436415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4368c62640 1 --2- 172.21.15.27:0/3558946758 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43640ccb90 0x7f436415ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4368c62640 1 -- 172.21.15.27:0/3558946758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43500051c0 msgr2=0x7f4364135f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4368c62640 1 --2- 172.21.15.27:0/3558946758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43500051c0 0x7f4364135f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4368c62640 1 -- 172.21.15.27:0/3558946758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43640cb1a0 con 0x7f43641ef5c0 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4368c62640 1 --2- 172.21.15.27:0/3558946758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43641ef5c0 0x7f43640cc650 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4360011630 tx=0x7f4360011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4359ffb640 1 -- 172.21.15.27:0/3558946758 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360019070 con 0x7f43641ef5c0 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4359ffb640 1 -- 172.21.15.27:0/3558946758 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4360004220 con 0x7f43641ef5c0 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f436b6c7640 1 -- 172.21.15.27:0/3558946758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43640cb480 con 0x7f43641ef5c0 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f4359ffb640 1 -- 172.21.15.27:0/3558946758 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436000f040 con 0x7f43641ef5c0 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.873+0000 7f436b6c7640 1 -- 172.21.15.27:0/3558946758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43640cba40 con 0x7f43641ef5c0 2024-04-20T10:55:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.874+0000 7f4359ffb640 1 -- 172.21.15.27:0/3558946758 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4360015670 con 0x7f43641ef5c0 2024-04-20T10:55:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.874+0000 7f4359ffb640 1 -- 172.21.15.27:0/3558946758 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4360003c00 con 0x7f43641ef5c0 2024-04-20T10:55:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.874+0000 7f436b6c7640 1 -- 172.21.15.27:0/3558946758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436404fcf0 con 0x7f43641ef5c0 2024-04-20T10:55:19.877 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:19.873+0000 7f5b56e3f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:55:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.877+0000 7f4359ffb640 1 -- 172.21.15.27:0/3558946758 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f4360003610 con 0x7f43641ef5c0 2024-04-20T10:55:19.887 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:19.887+0000 7f242960d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:55:19.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.939+0000 7f436b6c7640 1 -- 172.21.15.27:0/3558946758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4364165760 con 0x7f43641ef5c0 2024-04-20T10:55:19.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.939+0000 7f4359ffb640 1 -- 172.21.15.27:0/3558946758 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f43600037f0 con 0x7f43641ef5c0 2024-04-20T10:55:19.940 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:19.940 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:19.707863+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:19.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.940+0000 7f436b6c7640 1 -- 172.21.15.27:0/3558946758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43641ef5c0 msgr2=0x7f43640cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:19.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.940+0000 7f436b6c7640 1 --2- 172.21.15.27:0/3558946758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43641ef5c0 0x7f43640cc650 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4360011630 tx=0x7f4360011a30 comp rx=0 tx=0).stop 2024-04-20T10:55:19.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.940+0000 7f436b6c7640 1 -- 172.21.15.27:0/3558946758 shutdown_connections 2024-04-20T10:55:19.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.940+0000 7f436b6c7640 1 --2- 172.21.15.27:0/3558946758 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43640ccb90 0x7f436415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.940+0000 7f436b6c7640 1 --2- 172.21.15.27:0/3558946758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43641ef5c0 0x7f43640cc650 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.940+0000 7f436b6c7640 1 --2- 172.21.15.27:0/3558946758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43500051c0 0x7f4364135f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:19.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.940+0000 7f436b6c7640 1 -- 172.21.15.27:0/3558946758 >> 172.21.15.27:0/3558946758 conn(0x7f436406d570 msgr2=0x7f436414fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:19.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.940+0000 7f436b6c7640 1 -- 172.21.15.27:0/3558946758 shutdown_connections 2024-04-20T10:55:19.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:19.940+0000 7f436b6c7640 1 -- 172.21.15.27:0/3558946758 wait complete. 2024-04-20T10:55:19.951 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-20T10:55:19.952 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-20T10:55:20.134 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:20.134+0000 7f5b56e3f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:55:20.161 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:20.161+0000 7f242960d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:55:20.186 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:20.186+0000 7f5b56e3f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:55:20.215 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:20.216+0000 7f242960d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:55:20.301 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:20.301+0000 7f5b56e3f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:55:20.334 INFO:tasks.ceph.osd.4.smithi137.stderr:2024-04-20T10:55:20.334+0000 7f8aa6613640 -1 Falling back to public interface 2024-04-20T10:55:20.336 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-20T10:55:20.335+0000 7fcda461a640 -1 Falling back to public interface 2024-04-20T10:55:20.338 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:20.338+0000 7f242960d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:55:20.338 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-04-20T10:55:20.338+0000 7fc742e18640 -1 Falling back to public interface 2024-04-20T10:55:20.340 INFO:tasks.ceph.osd.7.smithi137.stderr:2024-04-20T10:55:20.339+0000 7f745d401640 -1 Falling back to public interface 2024-04-20T10:55:20.341 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-04-20T10:55:20.341+0000 7f3460e11640 -1 Falling back to public interface 2024-04-20T10:55:20.348 INFO:tasks.ceph.osd.5.smithi137.stderr:2024-04-20T10:55:20.347+0000 7fb30fe22640 -1 Falling back to public interface 2024-04-20T10:55:20.354 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-04-20T10:55:20.354+0000 7f58dc2a4640 -1 Falling back to public interface 2024-04-20T10:55:20.355 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-04-20T10:55:20.355+0000 7f3196744640 -1 Falling back to public interface 2024-04-20T10:55:20.585 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:20.584+0000 7f5b56e3f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:55:20.634 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:20.633+0000 7f5b56e3f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:55:20.635 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:20.636+0000 7f242960d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:55:20.685 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:20.686+0000 7f242960d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:55:20.756 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:20.790 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:20.789+0000 7f5b56e3f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:55:20.850 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.850+0000 7fe531dc7640 1 Processor -- start 2024-04-20T10:55:20.851 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.850+0000 7fe531dc7640 1 -- start start 2024-04-20T10:55:20.851 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.851+0000 7fe531dc7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe52c0c3ff0 0x7fe52c15c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.851+0000 7fe531dc7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe52c06fcf0 con 0x7fe52c1533c0 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.851+0000 7fe531dc7640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe52c0c4630 con 0x7fe52c0cee90 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.851+0000 7fe531dc7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe52c0c4bf0 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.851+0000 7fe52affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe52c0c3ff0 0x7fe52c15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.851+0000 7fe52b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe52c1533c0 0x7fe52c0b49b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40964/0 (socket says 172.21.15.27:40964) 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.851+0000 7fe52b7fe640 1 -- 172.21.15.27:0/3724311047 learned_addr learned my addr 172.21.15.27:0/3724311047 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.852+0000 7fe52affd640 1 -- 172.21.15.27:0/3724311047 >> v1:172.21.15.137:6789/0 conn(0x7fe52c0cee90 legacy=0x7fe52c163ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.852+0000 7fe52affd640 1 -- 172.21.15.27:0/3724311047 >> v1:172.21.15.27:6789/0 conn(0x7fe52c1533c0 legacy=0x7fe52c0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.852+0000 7fe52a7fc640 1 -- 172.21.15.27:0/3724311047 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 455608507 0 0) 0x7fe52c06fcf0 con 0x7fe52c1533c0 2024-04-20T10:55:20.852 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.852+0000 7fe52affd640 1 -- 172.21.15.27:0/3724311047 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c0c4910 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.853 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.853+0000 7fe52affd640 1 --2- 172.21.15.27:0/3724311047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 0x7fe52c15c720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0097a0 tx=0x7fe51c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e5af5bdd24307d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:20.853 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.853+0000 7fe52a7fc640 1 -- 172.21.15.27:0/3724311047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c034070 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.853 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.853+0000 7fe52a7fc640 1 -- 172.21.15.27:0/3724311047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51c030610 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.853 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.853+0000 7fe52a7fc640 1 -- 172.21.15.27:0/3724311047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0309e0 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.854 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.854+0000 7fe531dc7640 1 -- 172.21.15.27:0/3724311047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 msgr2=0x7fe52c15c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:20.854 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.854+0000 7fe531dc7640 1 --2- 172.21.15.27:0/3724311047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 0x7fe52c15c720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0097a0 tx=0x7fe51c02b960 comp rx=0 tx=0).stop 2024-04-20T10:55:20.854 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.854+0000 7fe531dc7640 1 -- 172.21.15.27:0/3724311047 shutdown_connections 2024-04-20T10:55:20.854 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.854+0000 7fe531dc7640 1 --2- 172.21.15.27:0/3724311047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 0x7fe52c15c720 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:20.854 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.854+0000 7fe531dc7640 1 -- 172.21.15.27:0/3724311047 >> 172.21.15.27:0/3724311047 conn(0x7fe52c067810 msgr2=0x7fe52c067bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:20.854 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.854+0000 7fe531dc7640 1 -- 172.21.15.27:0/3724311047 shutdown_connections 2024-04-20T10:55:20.854 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.854+0000 7fe531dc7640 1 -- 172.21.15.27:0/3724311047 wait complete. 2024-04-20T10:55:20.855 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:20.854+0000 7f242960d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:55:20.857 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.855+0000 7fe531dc7640 1 Processor -- start 2024-04-20T10:55:20.857 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.855+0000 7fe531dc7640 1 -- start start 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe531dc7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 0x7fe52c0c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe531dc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe52c0cee90 0x7fe52c15ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe531dc7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe52c1533c0 0x7fe52c14df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe531dc7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c0707b0 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe531dc7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c070290 con 0x7fe52c0cee90 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe531dc7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe52c1537a0 con 0x7fe52c1533c0 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 0x7fe52c0c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 0x7fe52c0c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58654/0 (socket says 172.21.15.27:58654) 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52b7fe640 1 -- 172.21.15.27:0/620358007 learned_addr learned my addr 172.21.15.27:0/620358007 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52affd640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe52c0cee90 0x7fe52c15ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52bfff640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe52c1533c0 0x7fe52c14df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52b7fe640 1 -- 172.21.15.27:0/620358007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe52c1533c0 msgr2=0x7fe52c14df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:20.858 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52b7fe640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe52c1533c0 0x7fe52c14df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:20.859 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52b7fe640 1 -- 172.21.15.27:0/620358007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe52c0cee90 msgr2=0x7fe52c15ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:20.859 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52b7fe640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe52c0cee90 0x7fe52c15ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:20.859 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52b7fe640 1 -- 172.21.15.27:0/620358007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c0c8e20 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.859 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52affd640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe52c0cee90 0x7fe52c15ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:20.859 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe52b7fe640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 0x7fe52c0c65b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe52000c3f0 tx=0x7fe52000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:20.859 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.856+0000 7fe513fff640 1 -- 172.21.15.27:0/620358007 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5200074e0 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.859 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.857+0000 7fe513fff640 1 -- 172.21.15.27:0/620358007 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5200047e0 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.859 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.857+0000 7fe513fff640 1 -- 172.21.15.27:0/620358007 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520004c10 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.860 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.857+0000 7fe531dc7640 1 -- 172.21.15.27:0/620358007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c152bb0 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.860 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.857+0000 7fe531dc7640 1 -- 172.21.15.27:0/620358007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c0c8580 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.860 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.857+0000 7fe513fff640 1 -- 172.21.15.27:0/620358007 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe520011070 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.860 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.857+0000 7fe513fff640 1 -- 172.21.15.27:0/620358007 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fe520004db0 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.860 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.858+0000 7fe531dc7640 1 -- 172.21.15.27:0/620358007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52c0695c0 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.861 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.861+0000 7fe513fff640 1 -- 172.21.15.27:0/620358007 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fe52001d4a0 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.923 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.924+0000 7fe531dc7640 1 -- 172.21.15.27:0/620358007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe52c0c8860 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.924 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.924+0000 7fe513fff640 1 -- 172.21.15.27:0/620358007 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fe52001d730 con 0x7fe52c0c3ff0 2024-04-20T10:55:20.924 INFO:teuthology.misc.health.smithi027.stdout: 2024-04-20T10:55:20.924 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":10,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:19.707863+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:20.925 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.925+0000 7fe531dc7640 1 -- 172.21.15.27:0/620358007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 msgr2=0x7fe52c0c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:20.925 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.925+0000 7fe531dc7640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 0x7fe52c0c65b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe52000c3f0 tx=0x7fe52000c8b0 comp rx=0 tx=0).stop 2024-04-20T10:55:20.925 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.925+0000 7fe531dc7640 1 -- 172.21.15.27:0/620358007 shutdown_connections 2024-04-20T10:55:20.925 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.925+0000 7fe531dc7640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe52c1533c0 0x7fe52c14df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:20.925 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.925+0000 7fe531dc7640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe52c0cee90 0x7fe52c15ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:20.926 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.925+0000 7fe531dc7640 1 --2- 172.21.15.27:0/620358007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe52c0c3ff0 0x7fe52c0c65b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:20.926 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.925+0000 7fe531dc7640 1 -- 172.21.15.27:0/620358007 >> 172.21.15.27:0/620358007 conn(0x7fe52c067810 msgr2=0x7fe52c0caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:20.926 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.926+0000 7fe531dc7640 1 -- 172.21.15.27:0/620358007 shutdown_connections 2024-04-20T10:55:20.926 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:20.926+0000 7fe531dc7640 1 -- 172.21.15.27:0/620358007 wait complete. 2024-04-20T10:55:20.929 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:20.928+0000 7f5b56e3f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:55:20.934 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T10:55:20.979 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:20.978+0000 7f5b56e3f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:55:21.002 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:21.002+0000 7f242960d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:55:21.048 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:21.047+0000 7f5b56e3f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:55:21.054 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:21.054+0000 7f242960d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:55:21.093 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:21.092+0000 7f5b56e3f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:55:21.127 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:21.128+0000 7f242960d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:55:21.175 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:21.176+0000 7f242960d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:55:21.572 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:21.571+0000 7f5b56e3f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:55:21.677 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:21.678+0000 7f242960d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:55:21.685 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:21.685+0000 7f5b56e3f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:55:21.802 INFO:tasks.ceph.mgr.y.smithi027.stderr:2024-04-20T10:55:21.803+0000 7f242960d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:55:22.703 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:22.702+0000 7f5af7a54640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-20T10:55:22.703 INFO:tasks.ceph.mgr.x.smithi137.stderr:2024-04-20T10:55:22.703+0000 7f5af7a54640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-20T10:55:23.838 INFO:tasks.ceph.osd.4.smithi137.stderr:2024-04-20T10:55:23.838+0000 7f8aa6613640 -1 osd.4 0 log_to_monitors true 2024-04-20T10:55:23.853 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-20T10:55:23.852+0000 7fcda461a640 -1 osd.6 0 log_to_monitors true 2024-04-20T10:55:23.855 INFO:tasks.ceph.osd.7.smithi137.stderr:2024-04-20T10:55:23.855+0000 7f745d401640 -1 osd.7 0 log_to_monitors true 2024-04-20T10:55:23.857 INFO:tasks.ceph.osd.5.smithi137.stderr:2024-04-20T10:55:23.856+0000 7fb30fe22640 -1 osd.5 0 log_to_monitors true 2024-04-20T10:55:23.873 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-04-20T10:55:23.873+0000 7f3460e11640 -1 osd.0 0 log_to_monitors true 2024-04-20T10:55:23.873 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-04-20T10:55:23.873+0000 7f3196744640 -1 osd.3 0 log_to_monitors true 2024-04-20T10:55:23.878 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-04-20T10:55:23.878+0000 7f58dc2a4640 -1 osd.2 0 log_to_monitors true 2024-04-20T10:55:23.878 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-04-20T10:55:23.878+0000 7fc742e18640 -1 osd.1 0 log_to_monitors true 2024-04-20T10:55:27.742 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:27.766 INFO:tasks.ceph.osd.4.smithi137.stderr:2024-04-20T10:55:27.765+0000 7f8aa1faa640 -1 osd.4 0 waiting for initial osdmap 2024-04-20T10:55:27.781 INFO:tasks.ceph.osd.4.smithi137.stderr:2024-04-20T10:55:27.780+0000 7f8a99dca640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:55:27.816 INFO:tasks.ceph.osd.7.smithi137.stderr:2024-04-20T10:55:27.815+0000 7f7459397640 -1 osd.7 0 waiting for initial osdmap 2024-04-20T10:55:27.819 INFO:tasks.ceph.osd.5.smithi137.stderr:2024-04-20T10:55:27.818+0000 7fb30b6f6640 -1 osd.5 0 waiting for initial osdmap 2024-04-20T10:55:27.820 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-20T10:55:27.819+0000 7fcda007d640 -1 osd.6 0 waiting for initial osdmap 2024-04-20T10:55:27.822 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-04-20T10:55:27.823+0000 7f345c7ff640 -1 osd.0 0 waiting for initial osdmap 2024-04-20T10:55:27.827 INFO:tasks.ceph.osd.7.smithi137.stderr:2024-04-20T10:55:27.827+0000 7f74511b7640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:55:27.828 INFO:tasks.ceph.osd.5.smithi137.stderr:2024-04-20T10:55:27.828+0000 7fb303516640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:55:27.835 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-20T10:55:27.835+0000 7fcd97e9d640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:55:27.837 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-04-20T10:55:27.838+0000 7f31926da640 -1 osd.3 0 waiting for initial osdmap 2024-04-20T10:55:27.838 INFO:tasks.ceph.osd.0.smithi027.stderr:2024-04-20T10:55:27.838+0000 7f345461f640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:55:27.849 INFO:tasks.ceph.osd.3.smithi027.stderr:2024-04-20T10:55:27.850+0000 7f318a4fa640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:55:27.861 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-04-20T10:55:27.861+0000 7fc73e87d640 -1 osd.1 0 waiting for initial osdmap 2024-04-20T10:55:27.875 INFO:tasks.ceph.osd.1.smithi027.stderr:2024-04-20T10:55:27.876+0000 7fc73669d640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:55:27.890 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.891+0000 7f9571df4640 1 Processor -- start 2024-04-20T10:55:27.891 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.891+0000 7f9571df4640 1 -- start start 2024-04-20T10:55:27.892 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f9571df4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c155850 0x7f956c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:27.892 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f9571df4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f956c0cb280 0x7f956c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:27.892 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f9571df4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956c070340 con 0x7f956c06f5b0 2024-04-20T10:55:27.892 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f9571df4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f956c06ce00 con 0x7f956c0cb280 2024-04-20T10:55:27.892 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f9571df4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f956c071d40 con 0x7f956c155850 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c155850 0x7f956c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956bfff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f956c0cb280 0x7f956c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956affd640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f956c06f5b0 0x7f956c0cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:32822/0 (socket says 172.21.15.27:32822) 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c155850 0x7f956c0cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:50050/0 (socket says 172.21.15.27:50050) 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956affd640 1 -- 172.21.15.27:0/3093267114 learned_addr learned my addr 172.21.15.27:0/3093267114 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956bfff640 1 --2- 172.21.15.27:0/3093267114 >> v2:172.21.15.137:3300/0 conn(0x7f956c0cb280 0x7f956c0cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:48510/0 (socket says 172.21.15.27:48510) 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956bfff640 1 -- 172.21.15.27:0/3093267114 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c155850 msgr2=0x7f956c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956bfff640 1 --2- 172.21.15.27:0/3093267114 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c155850 0x7f956c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956bfff640 1 -- 172.21.15.27:0/3093267114 >> v1:172.21.15.137:6789/0 conn(0x7f956c06f5b0 legacy=0x7f956c0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956bfff640 1 -- 172.21.15.27:0/3093267114 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c0ca7b0 con 0x7f956c0cb280 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956a7fc640 1 -- 172.21.15.27:0/3093267114 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826209846 0 0) 0x7f956c070340 con 0x7f956c06f5b0 2024-04-20T10:55:27.893 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.892+0000 7f956b7fe640 1 --2- 172.21.15.27:0/3093267114 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c155850 0x7f956c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:27.895 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f956bfff640 1 --2- 172.21.15.27:0/3093267114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c0cb280 0x7f956c0cf800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9564009d90 tx=0x7f956402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ee98c97728992e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:27.895 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f956a7fc640 1 -- 172.21.15.27:0/3093267114 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564039070 con 0x7f956c0cb280 2024-04-20T10:55:27.895 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f956a7fc640 1 -- 172.21.15.27:0/3093267114 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956402cce0 con 0x7f956c0cb280 2024-04-20T10:55:27.895 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f956a7fc640 1 -- 172.21.15.27:0/3093267114 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564034660 con 0x7f956c0cb280 2024-04-20T10:55:27.895 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f9571df4640 1 -- 172.21.15.27:0/3093267114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c0cb280 msgr2=0x7f956c0cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:27.895 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f9571df4640 1 --2- 172.21.15.27:0/3093267114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c0cb280 0x7f956c0cf800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9564009d90 tx=0x7f956402bf30 comp rx=0 tx=0).stop 2024-04-20T10:55:27.896 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f9571df4640 1 -- 172.21.15.27:0/3093267114 shutdown_connections 2024-04-20T10:55:27.896 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f9571df4640 1 --2- 172.21.15.27:0/3093267114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c0cb280 0x7f956c0cf800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:27.896 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f9571df4640 1 --2- 172.21.15.27:0/3093267114 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c155850 0x7f956c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:27.896 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f9571df4640 1 -- 172.21.15.27:0/3093267114 >> 172.21.15.27:0/3093267114 conn(0x7f956c073420 msgr2=0x7f956c073800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:27.896 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f9571df4640 1 -- 172.21.15.27:0/3093267114 shutdown_connections 2024-04-20T10:55:27.896 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.894+0000 7f9571df4640 1 -- 172.21.15.27:0/3093267114 wait complete. 2024-04-20T10:55:27.897 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.895+0000 7f9571df4640 1 Processor -- start 2024-04-20T10:55:27.897 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.896+0000 7f9571df4640 1 -- start start 2024-04-20T10:55:27.897 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.896+0000 7f9571df4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c06f5b0 0x7f956c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:27.897 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.896+0000 7f9571df4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f956c155850 0x7f956c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:27.898 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.896+0000 7f9571df4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c149020 0x7f956c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:27.898 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.896+0000 7f9571df4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f956c0b43f0 con 0x7f956c155850 2024-04-20T10:55:27.898 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.896+0000 7f9571df4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f956c06e3b0 con 0x7f956c149020 2024-04-20T10:55:27.898 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.896+0000 7f9571df4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f956c06ca50 con 0x7f956c06f5b0 2024-04-20T10:55:27.898 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f956c155850 0x7f956c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:27.898 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c149020 0x7f956c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:27.898 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f956c155850 0x7f956c1471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43910/0 (socket says 172.21.15.27:43910) 2024-04-20T10:55:27.898 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956b7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c06f5b0 0x7f956c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:27.898 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956affd640 1 -- 172.21.15.27:0/3765005600 learned_addr learned my addr 172.21.15.27:0/3765005600 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:27.898 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f58d823a640 -1 osd.2 0 waiting for initial osdmap 2024-04-20T10:55:27.900 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956bfff640 1 -- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c06f5b0 msgr2=0x7f956c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:27.900 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956bfff640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c06f5b0 0x7f956c156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:27.900 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956bfff640 1 -- 172.21.15.27:0/3765005600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f956c155850 msgr2=0x7f956c1471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956bfff640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f956c155850 0x7f956c1471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956bfff640 1 -- 172.21.15.27:0/3765005600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c146720 con 0x7f956c149020 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f956bfff640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c149020 0x7f956c0ce020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f956c070ec0 tx=0x7f9564004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.897+0000 7f9553fff640 1 -- 172.21.15.27:0/3765005600 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564039050 con 0x7f956c149020 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.898+0000 7f9553fff640 1 -- 172.21.15.27:0/3765005600 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9564045070 con 0x7f956c149020 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.898+0000 7f9553fff640 1 -- 172.21.15.27:0/3765005600 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564009450 con 0x7f956c149020 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.898+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956c1f1c10 con 0x7f956c149020 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.898+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956c146cc0 con 0x7f956c149020 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.899+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f956c069160 con 0x7f956c149020 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.899+0000 7f9553fff640 1 -- 172.21.15.27:0/3765005600 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f956403e850 con 0x7f956c149020 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.899+0000 7f9553fff640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f955806c860 0x7f955806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.899+0000 7f9553fff640 1 -- 172.21.15.27:0/3765005600 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f95640ad8e0 con 0x7f956c149020 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.900+0000 7f956b7fe640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f955806c860 0x7f955806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:27.901 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.900+0000 7f956b7fe640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f955806c860 0x7f955806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f955c009560 tx=0x7f955c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:27.907 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:27.907+0000 7f9553fff640 1 -- 172.21.15.27:0/3765005600 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f956407c640 con 0x7f956c149020 2024-04-20T10:55:27.909 INFO:tasks.ceph.osd.2.smithi027.stderr:2024-04-20T10:55:27.910+0000 7f58d005a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:55:28.094 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.094+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f956c076410 con 0x7f956c149020 2024-04-20T10:55:28.095 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.095+0000 7f9553fff640 1 -- 172.21.15.27:0/3765005600 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f956c076410 con 0x7f956c149020 2024-04-20T10:55:28.095 INFO:teuthology.misc.health.smithi027.stdout: 2024-04-20T10:55:28.095 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":12,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:25.710453+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:28.097 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f955806c860 msgr2=0x7f955806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:28.097 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f955806c860 0x7f955806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f955c009560 tx=0x7f955c009190 comp rx=0 tx=0).stop 2024-04-20T10:55:28.097 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c149020 msgr2=0x7f956c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:28.097 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c149020 0x7f956c0ce020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f956c070ec0 tx=0x7f9564004a40 comp rx=0 tx=0).stop 2024-04-20T10:55:28.097 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 shutdown_connections 2024-04-20T10:55:28.098 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f955806c860 0x7f955806ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:28.098 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f956c149020 0x7f956c0ce020 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:28.098 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f956c155850 0x7f956c1471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:28.098 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 --2- 172.21.15.27:0/3765005600 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f956c06f5b0 0x7f956c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:28.098 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 >> 172.21.15.27:0/3765005600 conn(0x7f956c073420 msgr2=0x7f956c15aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:28.098 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 shutdown_connections 2024-04-20T10:55:28.098 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:28.097+0000 7f9571df4640 1 -- 172.21.15.27:0/3765005600 wait complete. 2024-04-20T10:55:28.106 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T10:55:34.913 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:35.017 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.017+0000 7f04bbe43640 1 Processor -- start 2024-04-20T10:55:35.017 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.017+0000 7f04bbe43640 1 -- start start 2024-04-20T10:55:35.018 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04bbe43640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f04b4073100 0x7f04b4071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.018 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04bbe43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4073c40 0x7f04b40c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.018 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04bbe43640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04b40c3580 con 0x7f04b4072d20 2024-04-20T10:55:35.018 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04bbe43640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f04b40c3820 con 0x7f04b4073100 2024-04-20T10:55:35.018 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04bbe43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f04b40c3b00 con 0x7f04b4073c40 2024-04-20T10:55:35.018 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04b9bdf640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f04b4072d20 0x7f04b4152180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33450/0 (socket says 172.21.15.27:33450) 2024-04-20T10:55:35.018 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04b9bdf640 1 -- 172.21.15.27:0/32559527 learned_addr learned my addr 172.21.15.27:0/32559527 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:35.018 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04b93de640 1 --2- 172.21.15.27:0/32559527 >> v2:172.21.15.27:3300/0 conn(0x7f04b4073100 0x7f04b4071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.019 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04ba3e0640 1 --2- 172.21.15.27:0/32559527 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4073c40 0x7f04b40c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.019 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.018+0000 7f04b8bdd640 1 -- 172.21.15.27:0/32559527 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2712130113 0 0) 0x7f04b40c3580 con 0x7f04b4072d20 2024-04-20T10:55:35.020 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.019+0000 7f04b8bdd640 1 -- 172.21.15.27:0/32559527 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f049c005730 con 0x7f04b4072d20 2024-04-20T10:55:35.020 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.019+0000 7f04b93de640 1 -- 172.21.15.27:0/32559527 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4073c40 msgr2=0x7f04b40c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:35.020 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.019+0000 7f04b93de640 1 --2- 172.21.15.27:0/32559527 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4073c40 0x7f04b40c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.020 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.019+0000 7f04b93de640 1 -- 172.21.15.27:0/32559527 >> v1:172.21.15.27:6789/0 conn(0x7f04b4072d20 legacy=0x7f04b4152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.020 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.019+0000 7f04b93de640 1 -- 172.21.15.27:0/32559527 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b40c3de0 con 0x7f04b4073100 2024-04-20T10:55:35.020 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.019+0000 7f04ba3e0640 1 --2- 172.21.15.27:0/32559527 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4073c40 0x7f04b40c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:35.020 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.019+0000 7f04b93de640 1 --2- 172.21.15.27:0/32559527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073100 0x7f04b4071130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04a80097a0 tx=0x7f04a802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef79f1a61feb93c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.020 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04b8bdd640 1 -- 172.21.15.27:0/32559527 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8035020 con 0x7f04b4073100 2024-04-20T10:55:35.020 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04b8bdd640 1 -- 172.21.15.27:0/32559527 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a8031810 con 0x7f04b4073100 2024-04-20T10:55:35.021 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04b8bdd640 1 -- 172.21.15.27:0/32559527 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8031bc0 con 0x7f04b4073100 2024-04-20T10:55:35.021 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04bbe43640 1 -- 172.21.15.27:0/32559527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073100 msgr2=0x7f04b4071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.021 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04bbe43640 1 --2- 172.21.15.27:0/32559527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073100 0x7f04b4071130 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04a80097a0 tx=0x7f04a802b8b0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.021 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04bbe43640 1 -- 172.21.15.27:0/32559527 shutdown_connections 2024-04-20T10:55:35.021 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04bbe43640 1 --2- 172.21.15.27:0/32559527 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4073c40 0x7f04b40c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.021 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04bbe43640 1 --2- 172.21.15.27:0/32559527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073100 0x7f04b4071130 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.021 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04bbe43640 1 -- 172.21.15.27:0/32559527 >> 172.21.15.27:0/32559527 conn(0x7f04b406eab0 msgr2=0x7f04b406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:35.021 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04bbe43640 1 -- 172.21.15.27:0/32559527 shutdown_connections 2024-04-20T10:55:35.021 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.020+0000 7f04bbe43640 1 -- 172.21.15.27:0/32559527 wait complete. 2024-04-20T10:55:35.023 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.021+0000 7f04bbe43640 1 Processor -- start 2024-04-20T10:55:35.023 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04bbe43640 1 -- start start 2024-04-20T10:55:35.023 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04bbe43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4072d20 0x7f04b40c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04bbe43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4073100 0x7f04b4149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04bbe43640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073c40 0x7f04b41ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04bbe43640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f04b406f6c0 con 0x7f04b4073c40 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04bbe43640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04b40c1ef0 con 0x7f04b4073100 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04bbe43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f04b406f840 con 0x7f04b4072d20 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04ba3e0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073c40 0x7f04b41ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04ba3e0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073c40 0x7f04b41ecc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40378/0 (socket says 172.21.15.27:40378) 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.022+0000 7f04ba3e0640 1 -- 172.21.15.27:0/2338405656 learned_addr learned my addr 172.21.15.27:0/2338405656 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04b9bdf640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4072d20 0x7f04b40c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04b93de640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4073100 0x7f04b4149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04ba3e0640 1 -- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4072d20 msgr2=0x7f04b40c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04ba3e0640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4072d20 0x7f04b40c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04ba3e0640 1 -- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4073100 msgr2=0x7f04b4149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04ba3e0640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4073100 0x7f04b4149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.024 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04ba3e0640 1 -- 172.21.15.27:0/2338405656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b40c5d30 con 0x7f04b4073c40 2024-04-20T10:55:35.025 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04b9bdf640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4072d20 0x7f04b40c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:35.025 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04b93de640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4073100 0x7f04b4149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:35.025 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04ba3e0640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073c40 0x7f04b41ecc00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007e60 tx=0x7f04a400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.025 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04b27fc640 1 -- 172.21.15.27:0/2338405656 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a401a070 con 0x7f04b4073c40 2024-04-20T10:55:35.025 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04b27fc640 1 -- 172.21.15.27:0/2338405656 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a4015600 con 0x7f04b4073c40 2024-04-20T10:55:35.025 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b40c5ff0 con 0x7f04b4073c40 2024-04-20T10:55:35.025 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04b27fc640 1 -- 172.21.15.27:0/2338405656 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a40159f0 con 0x7f04b4073c40 2024-04-20T10:55:35.025 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.023+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b41f0430 con 0x7f04b4073c40 2024-04-20T10:55:35.026 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.024+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b406e3b0 con 0x7f04b4073c40 2024-04-20T10:55:35.026 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.025+0000 7f04b27fc640 1 -- 172.21.15.27:0/2338405656 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04a401d070 con 0x7f04b4073c40 2024-04-20T10:55:35.026 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.025+0000 7f04b27fc640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f04a006c8f0 0x7f04a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.026 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.025+0000 7f04b27fc640 1 -- 172.21.15.27:0/2338405656 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f04a408ba10 con 0x7f04b4073c40 2024-04-20T10:55:35.026 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.025+0000 7f04b9bdf640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f04a006c8f0 0x7f04a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.026 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.026+0000 7f04b9bdf640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f04a006c8f0 0x7f04a006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f04ac007fd0 tx=0x7f04ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.030 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.030+0000 7f04b27fc640 1 -- 172.21.15.27:0/2338405656 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f04b406e3b0 con 0x7f04b4073c40 2024-04-20T10:55:35.214 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.215+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04b40695c0 con 0x7f04b4073c40 2024-04-20T10:55:35.215 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.216+0000 7f04b27fc640 1 -- 172.21.15.27:0/2338405656 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12848 (secure 0 0 0) 0x7f04a4010070 con 0x7f04b4073c40 2024-04-20T10:55:35.215 INFO:teuthology.misc.health.smithi027.stdout: 2024-04-20T10:55:35.215 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":16,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:31.776479+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:35.217 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.217+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f04a006c8f0 msgr2=0x7f04a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.217+0000 7f04bbe43640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f04a006c8f0 0x7f04a006edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f04ac007fd0 tx=0x7f04ac009040 comp rx=0 tx=0).stop 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.217+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073c40 msgr2=0x7f04b41ecc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.217+0000 7f04bbe43640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073c40 0x7f04b41ecc00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007e60 tx=0x7f04a400d4a0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.217+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 shutdown_connections 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.217+0000 7f04bbe43640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f04a006c8f0 0x7f04a006edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.217+0000 7f04bbe43640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04b4073c40 0x7f04b41ecc00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.217+0000 7f04bbe43640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04b4073100 0x7f04b4149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.218+0000 7f04bbe43640 1 --2- 172.21.15.27:0/2338405656 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f04b4072d20 0x7f04b40c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.218+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 >> 172.21.15.27:0/2338405656 conn(0x7f04b406eab0 msgr2=0x7f04b414fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.218+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 shutdown_connections 2024-04-20T10:55:35.218 INFO:teuthology.misc.health.smithi027.stderr:2024-04-20T10:55:35.218+0000 7f04bbe43640 1 -- 172.21.15.27:0/2338405656 wait complete. 2024-04-20T10:55:35.227 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-20T10:55:35.227 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-20T10:55:35.227 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T10:55:35.228 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T10:55:35.230 INFO:tasks.ceph.mds.a:Started 2024-04-20T10:55:35.230 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T10:55:35.230 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T10:55:35.232 INFO:tasks.ceph.mds.c:Started 2024-04-20T10:55:35.232 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-20T10:55:35.232 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-20T10:55:35.234 INFO:tasks.ceph.mds.e:Started 2024-04-20T10:55:35.234 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T10:55:35.235 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T10:55:35.237 INFO:tasks.ceph.mds.b:Started 2024-04-20T10:55:35.237 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T10:55:35.237 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T10:55:35.239 INFO:tasks.ceph.mds.d:Started 2024-04-20T10:55:35.240 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-20T10:55:35.240 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-20T10:55:35.242 INFO:tasks.ceph.mds.f:Started 2024-04-20T10:55:35.242 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-20T10:55:35.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:35.308 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-04-20T10:55:35.308 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-04-20T10:55:35.308 INFO:tasks.ceph.mds.e.smithi027.stdout:starting mds.e at 2024-04-20T10:55:35.323 INFO:tasks.ceph.mds.f.smithi137.stdout:starting mds.f at 2024-04-20T10:55:35.325 INFO:tasks.ceph.mds.b.smithi137.stdout:starting mds.b at 2024-04-20T10:55:35.326 INFO:tasks.ceph.mds.d.smithi137.stdout:starting mds.d at 2024-04-20T10:55:35.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.350+0000 7fca9e196640 1 Processor -- start 2024-04-20T10:55:35.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.350+0000 7fca9e196640 1 -- start start 2024-04-20T10:55:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.351+0000 7fca9e196640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fca98153420 0x7fca98159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.351+0000 7fca9e196640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca980caea0 con 0x7fca9815c8f0 2024-04-20T10:55:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.351+0000 7fca9e196640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca980ca8e0 con 0x7fca980c2ea0 2024-04-20T10:55:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.351+0000 7fca9e196640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fca9815cd60 con 0x7fca98153420 2024-04-20T10:55:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.351+0000 7fca97fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fca9815c8f0 0x7fca980c4110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33466/0 (socket says 172.21.15.27:33466) 2024-04-20T10:55:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.351+0000 7fca97fff640 1 -- 172.21.15.27:0/3928870918 learned_addr learned my addr 172.21.15.27:0/3928870918 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.351+0000 7fca977fe640 1 --2- 172.21.15.27:0/3928870918 >> v2:172.21.15.137:3300/0 conn(0x7fca98153420 0x7fca98159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2241420467 0 0) 0x7fca980caea0 con 0x7fca9815c8f0 2024-04-20T10:55:35.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca80003520 con 0x7fca9815c8f0 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321863757 0 0) 0x7fca980ca8e0 con 0x7fca980c2ea0 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca980caea0 con 0x7fca980c2ea0 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2582420426 0 0) 0x7fca80003520 con 0x7fca9815c8f0 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 >> v2:172.21.15.137:3300/0 conn(0x7fca98153420 msgr2=0x7fca98159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 --2- 172.21.15.27:0/3928870918 >> v2:172.21.15.137:3300/0 conn(0x7fca98153420 0x7fca98159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 >> v1:172.21.15.137:6789/0 conn(0x7fca980c2ea0 legacy=0x7fca980c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca980cabc0 con 0x7fca9815c8f0 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7fca900019f0 con 0x7fca9815c8f0 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 >> v1:172.21.15.27:6789/0 conn(0x7fca9815c8f0 legacy=0x7fca980c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca80003b40 0x7fca80003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80004800 0x7fca80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca80005230 0x7fca8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca9815cd60 con 0x7fca80003b40 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fca8000c190 con 0x7fca80004800 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fca8000c390 con 0x7fca80005230 2024-04-20T10:55:35.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca977fe640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca80003b40 0x7fca80003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca96ffd640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80004800 0x7fca80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca97fff640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca80005230 0x7fca8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca977fe640 1 -- 172.21.15.27:0/3928870918 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca80005230 msgr2=0x7fca8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca977fe640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca80005230 0x7fca8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca977fe640 1 -- 172.21.15.27:0/3928870918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80004800 msgr2=0x7fca80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca977fe640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80004800 0x7fca80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca977fe640 1 -- 172.21.15.27:0/3928870918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca80003520 con 0x7fca80003b40 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.352+0000 7fca97fff640 1 -- 172.21.15.27:0/3928870918 reap_dead start 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.353+0000 7fca977fe640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca80003b40 0x7fca80003f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca88009d20 tx=0x7fca80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9933ce26cb79f4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.353+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88039070 con 0x7fca80003b40 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.353+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca8802c920 con 0x7fca80003b40 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.353+0000 7fca967fc640 1 -- 172.21.15.27:0/3928870918 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88009690 con 0x7fca80003b40 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.354+0000 7fca9e196640 1 -- 172.21.15.27:0/3928870918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca80003b40 msgr2=0x7fca80003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.354+0000 7fca9e196640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca80003b40 0x7fca80003f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca88009d20 tx=0x7fca80005830 comp rx=0 tx=0).stop 2024-04-20T10:55:35.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.354+0000 7fca9e196640 1 -- 172.21.15.27:0/3928870918 shutdown_connections 2024-04-20T10:55:35.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.354+0000 7fca9e196640 1 --2- 172.21.15.27:0/3928870918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca80003b40 0x7fca80003f40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.354+0000 7fca9e196640 1 -- 172.21.15.27:0/3928870918 >> 172.21.15.27:0/3928870918 conn(0x7fca9806fb50 msgr2=0x7fca9806ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:35.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.354+0000 7fca9e196640 1 -- 172.21.15.27:0/3928870918 shutdown_connections 2024-04-20T10:55:35.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.354+0000 7fca9e196640 1 -- 172.21.15.27:0/3928870918 wait complete. 2024-04-20T10:55:35.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.355+0000 7fca9e196640 1 Processor -- start 2024-04-20T10:55:35.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.355+0000 7fca9e196640 1 -- start start 2024-04-20T10:55:35.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca9e196640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80003b40 0x7fca980c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca9e196640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca981f2540 0x7fca981e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca9e196640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca980c6dd0 0x7fca981e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca9e196640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca981596f0 con 0x7fca981f2540 2024-04-20T10:55:35.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca9e196640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fca98153800 con 0x7fca80003b40 2024-04-20T10:55:35.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca9e196640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fca981599d0 con 0x7fca980c6dd0 2024-04-20T10:55:35.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca981f2540 0x7fca981e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca981f2540 0x7fca981e22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40434/0 (socket says 172.21.15.27:40434) 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca96ffd640 1 -- 172.21.15.27:0/3942059893 learned_addr learned my addr 172.21.15.27:0/3942059893 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca977fe640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80003b40 0x7fca980c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca97fff640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca980c6dd0 0x7fca981e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca96ffd640 1 -- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca980c6dd0 msgr2=0x7fca981e44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca96ffd640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca980c6dd0 0x7fca981e44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.356+0000 7fca96ffd640 1 -- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80003b40 msgr2=0x7fca980c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca96ffd640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80003b40 0x7fca980c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca96ffd640 1 -- 172.21.15.27:0/3942059893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca980c7350 con 0x7fca981f2540 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca97fff640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca980c6dd0 0x7fca981e44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca977fe640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80003b40 0x7fca980c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca96ffd640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca981f2540 0x7fca981e22f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0079a0 tx=0x7fca8c00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca7ffff640 1 -- 172.21.15.27:0/3942059893 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0050d0 con 0x7fca981f2540 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9814ede0 con 0x7fca981f2540 2024-04-20T10:55:35.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9814f7a0 con 0x7fca981f2540 2024-04-20T10:55:35.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca7ffff640 1 -- 172.21.15.27:0/3942059893 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca8c018070 con 0x7fca981f2540 2024-04-20T10:55:35.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.357+0000 7fca7ffff640 1 -- 172.21.15.27:0/3942059893 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0050d0 con 0x7fca981f2540 2024-04-20T10:55:35.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.359+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca78005290 con 0x7fca981f2540 2024-04-20T10:55:35.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.359+0000 7fca7ffff640 1 -- 172.21.15.27:0/3942059893 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca8c005270 con 0x7fca981f2540 2024-04-20T10:55:35.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.359+0000 7fca7ffff640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca8406ccf0 0x7fca8406f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.360+0000 7fca7ffff640 1 -- 172.21.15.27:0/3942059893 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fca8c010040 con 0x7fca981f2540 2024-04-20T10:55:35.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.360+0000 7fca977fe640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca8406ccf0 0x7fca8406f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.360+0000 7fca977fe640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca8406ccf0 0x7fca8406f1b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca88036000 tx=0x7fca88030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.364+0000 7fca7ffff640 1 -- 172.21.15.27:0/3942059893 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fca8c059670 con 0x7fca981f2540 2024-04-20T10:55:35.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.549+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca78005850 con 0x7fca981f2540 2024-04-20T10:55:35.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.550+0000 7fca7ffff640 1 -- 172.21.15.27:0/3942059893 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12848 (secure 0 0 0) 0x7fca8c059670 con 0x7fca981f2540 2024-04-20T10:55:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:35.550 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:31.776479+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca8406ccf0 msgr2=0x7fca8406f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca8406ccf0 0x7fca8406f1b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca88036000 tx=0x7fca88030000 comp rx=0 tx=0).stop 2024-04-20T10:55:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca981f2540 msgr2=0x7fca981e22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca981f2540 0x7fca981e22f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0079a0 tx=0x7fca8c00fc00 comp rx=0 tx=0).stop 2024-04-20T10:55:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 shutdown_connections 2024-04-20T10:55:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca8406ccf0 0x7fca8406f1b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca980c6dd0 0x7fca981e44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.552 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca981f2540 0x7fca981e22f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 --2- 172.21.15.27:0/3942059893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca80003b40 0x7fca980c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 >> 172.21.15.27:0/3942059893 conn(0x7fca9806fb50 msgr2=0x7fca980cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 shutdown_connections 2024-04-20T10:55:35.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.552+0000 7fca9e196640 1 -- 172.21.15.27:0/3942059893 wait complete. 2024-04-20T10:55:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:55:29.717620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:55:35.563 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:55:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.668+0000 7f507d7d0640 1 Processor -- start 2024-04-20T10:55:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.668+0000 7f507d7d0640 1 -- start start 2024-04-20T10:55:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f507d7d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f507815ee50 0x7f507815c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f507d7d0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 0x7f507806cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f507d7d0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f507815f9a0 con 0x7f50780ccbd0 2024-04-20T10:55:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f507d7d0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f50780cb870 con 0x7f507815ee50 2024-04-20T10:55:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f507d7d0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f507806d490 con 0x7f507815f230 2024-04-20T10:55:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f5076ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f507815ee50 0x7f507815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f50777fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f50780ccbd0 0x7f5078155410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33478/0 (socket says 172.21.15.27:33478) 2024-04-20T10:55:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f50777fe640 1 -- 172.21.15.27:0/4201543540 learned_addr learned my addr 172.21.15.27:0/4201543540 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f5076ffd640 1 -- 172.21.15.27:0/4201543540 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 msgr2=0x7f507806cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f506ffff640 1 --2- 172.21.15.27:0/4201543540 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 0x7f507806cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f50767fc640 1 -- 172.21.15.27:0/4201543540 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691973990 0 0) 0x7f5070001490 con 0x7f50780ccbd0 2024-04-20T10:55:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f5076ffd640 1 --2- 172.21.15.27:0/4201543540 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 0x7f507806cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f5076ffd640 1 -- 172.21.15.27:0/4201543540 >> v1:172.21.15.27:6789/0 conn(0x7f50780ccbd0 legacy=0x7f5078155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f5076ffd640 1 -- 172.21.15.27:0/4201543540 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50780cbb30 con 0x7f507815ee50 2024-04-20T10:55:35.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.669+0000 7f506ffff640 1 --2- 172.21.15.27:0/4201543540 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 0x7f507806cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.670+0000 7f5076ffd640 1 --2- 172.21.15.27:0/4201543540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 0x7f507815c180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f50680097a0 tx=0x7f506802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6ddf2c6ef6fc50 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.670+0000 7f50767fc640 1 -- 172.21.15.27:0/4201543540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068034070 con 0x7f507815ee50 2024-04-20T10:55:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.670+0000 7f50767fc640 1 -- 172.21.15.27:0/4201543540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5068030610 con 0x7f507815ee50 2024-04-20T10:55:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.670+0000 7f50767fc640 1 -- 172.21.15.27:0/4201543540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50680309e0 con 0x7f507815ee50 2024-04-20T10:55:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.671+0000 7f507d7d0640 1 -- 172.21.15.27:0/4201543540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 msgr2=0x7f507815c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.671+0000 7f507d7d0640 1 --2- 172.21.15.27:0/4201543540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 0x7f507815c180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f50680097a0 tx=0x7f506802b960 comp rx=0 tx=0).stop 2024-04-20T10:55:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.671+0000 7f507d7d0640 1 -- 172.21.15.27:0/4201543540 shutdown_connections 2024-04-20T10:55:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.671+0000 7f507d7d0640 1 --2- 172.21.15.27:0/4201543540 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 0x7f507806cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.671+0000 7f507d7d0640 1 --2- 172.21.15.27:0/4201543540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 0x7f507815c180 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.671+0000 7f507d7d0640 1 -- 172.21.15.27:0/4201543540 >> 172.21.15.27:0/4201543540 conn(0x7f507806e010 msgr2=0x7f50780d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.671+0000 7f507d7d0640 1 -- 172.21.15.27:0/4201543540 shutdown_connections 2024-04-20T10:55:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.671+0000 7f507d7d0640 1 -- 172.21.15.27:0/4201543540 wait complete. 2024-04-20T10:55:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.672+0000 7f507d7d0640 1 Processor -- start 2024-04-20T10:55:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.672+0000 7f507d7d0640 1 -- start start 2024-04-20T10:55:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.672+0000 7f507d7d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50780ccbd0 0x7f507815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.672+0000 7f507d7d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 0x7f50781f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f507d7d0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 0x7f50780ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f507d7d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50780cc3d0 con 0x7f507815ee50 2024-04-20T10:55:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f507d7d0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f50780cc0f0 con 0x7f50780ccbd0 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f507d7d0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f50780cc6b0 con 0x7f507815f230 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 0x7f50781f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 0x7f50781f1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40444/0 (socket says 172.21.15.27:40444) 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506ffff640 1 -- 172.21.15.27:0/4180081079 learned_addr learned my addr 172.21.15.27:0/4180081079 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f50777fe640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 0x7f50780ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f5076ffd640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50780ccbd0 0x7f507815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506ffff640 1 -- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 msgr2=0x7f50780ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506ffff640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 0x7f50780ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506ffff640 1 -- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50780ccbd0 msgr2=0x7f507815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506ffff640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50780ccbd0 0x7f507815c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506ffff640 1 -- 172.21.15.27:0/4180081079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507814ce40 con 0x7f507815ee50 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506ffff640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 0x7f50781f1910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f506000c820 tx=0x7f506000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506f7fe640 1 -- 172.21.15.27:0/4180081079 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060015070 con 0x7f507815ee50 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506f7fe640 1 -- 172.21.15.27:0/4180081079 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5060007a30 con 0x7f507815ee50 2024-04-20T10:55:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507814d120 con 0x7f507815ee50 2024-04-20T10:55:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f506f7fe640 1 -- 172.21.15.27:0/4180081079 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060007e00 con 0x7f507815ee50 2024-04-20T10:55:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.673+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507814d6e0 con 0x7f507815ee50 2024-04-20T10:55:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.674+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50780688e0 con 0x7f507815ee50 2024-04-20T10:55:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.675+0000 7f506f7fe640 1 -- 172.21.15.27:0/4180081079 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5060018070 con 0x7f507815ee50 2024-04-20T10:55:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.675+0000 7f506f7fe640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f505806c920 0x7f505806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.675+0000 7f506f7fe640 1 -- 172.21.15.27:0/4180081079 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f506008b740 con 0x7f507815ee50 2024-04-20T10:55:35.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.675+0000 7f5076ffd640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f505806c920 0x7f505806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.676+0000 7f5076ffd640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f505806c920 0x7f505806ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5068036000 tx=0x7f5068008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.679+0000 7f506f7fe640 1 -- 172.21.15.27:0/4180081079 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5060059ec0 con 0x7f507815ee50 2024-04-20T10:55:35.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.864+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f507804fcf0 con 0x7f507815ee50 2024-04-20T10:55:35.864 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.864+0000 7f506f7fe640 1 -- 172.21.15.27:0/4180081079 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f50600597c0 con 0x7f507815ee50 2024-04-20T10:55:35.864 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-20T10:55:35.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f505806c920 msgr2=0x7f505806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f505806c920 0x7f505806ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5068036000 tx=0x7f5068008030 comp rx=0 tx=0).stop 2024-04-20T10:55:35.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 msgr2=0x7f50781f1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 0x7f50781f1910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f506000c820 tx=0x7f506000cc20 comp rx=0 tx=0).stop 2024-04-20T10:55:35.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 shutdown_connections 2024-04-20T10:55:35.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f505806c920 0x7f505806ede0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f507815f230 0x7f50780ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f507815ee50 0x7f50781f1910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 --2- 172.21.15.27:0/4180081079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f50780ccbd0 0x7f507815c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 >> 172.21.15.27:0/4180081079 conn(0x7f507806e010 msgr2=0x7f50780cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:35.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 shutdown_connections 2024-04-20T10:55:35.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.866+0000 7f507d7d0640 1 -- 172.21.15.27:0/4180081079 wait complete. 2024-04-20T10:55:35.876 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:55:35.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.977+0000 7f46d5367640 1 Processor -- start 2024-04-20T10:55:35.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.977+0000 7f46d5367640 1 -- start start 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.977+0000 7f46d5367640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f46d00b54f0 0x7f46d00b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.977+0000 7f46d5367640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d00b5e70 con 0x7f46d00d0c90 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.977+0000 7f46d5367640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d015b9c0 con 0x7f46d0155950 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.977+0000 7f46d5367640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f46d015bca0 con 0x7f46d00b54f0 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.978+0000 7f46ce7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f46d00b54f0 0x7f46d00b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.978+0000 7f46cf7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f46d00d0c90 0x7f46d00d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33490/0 (socket says 172.21.15.27:33490) 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.978+0000 7f46ce7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f46d00b54f0 0x7f46d00b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40450/0 (socket says 172.21.15.27:40450) 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.978+0000 7f46cf7fe640 1 -- 172.21.15.27:0/1979037842 learned_addr learned my addr 172.21.15.27:0/1979037842 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.978+0000 7f46ce7fc640 1 -- 172.21.15.27:0/1979037842 >> v1:172.21.15.137:6789/0 conn(0x7f46d0155950 legacy=0x7f46d00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.978+0000 7f46ce7fc640 1 -- 172.21.15.27:0/1979037842 >> v1:172.21.15.27:6789/0 conn(0x7f46d00d0c90 legacy=0x7f46d00d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.978+0000 7f46ce7fc640 1 -- 172.21.15.27:0/1979037842 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d015bf80 con 0x7f46d00b54f0 2024-04-20T10:55:35.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.978+0000 7f46cdffb640 1 -- 172.21.15.27:0/1979037842 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2747127600 0 0) 0x7f46d00b5e70 con 0x7f46d00d0c90 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46ce7fc640 1 --2- 172.21.15.27:0/1979037842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d00b54f0 0x7f46d00b5930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f46c40097a0 tx=0x7f46c402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccb5cb48339daec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46cdffb640 1 -- 172.21.15.27:0/1979037842 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c402f810 con 0x7f46d00b54f0 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46cdffb640 1 -- 172.21.15.27:0/1979037842 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c402f9b0 con 0x7f46d00b54f0 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46cdffb640 1 -- 172.21.15.27:0/1979037842 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c402fd80 con 0x7f46d00b54f0 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46d5367640 1 -- 172.21.15.27:0/1979037842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d00b54f0 msgr2=0x7f46d00b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46d5367640 1 --2- 172.21.15.27:0/1979037842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d00b54f0 0x7f46d00b5930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f46c40097a0 tx=0x7f46c402b9f0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46d5367640 1 -- 172.21.15.27:0/1979037842 shutdown_connections 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46d5367640 1 --2- 172.21.15.27:0/1979037842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d00b54f0 0x7f46d00b5930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46d5367640 1 -- 172.21.15.27:0/1979037842 >> 172.21.15.27:0/1979037842 conn(0x7f46d00676c0 msgr2=0x7f46d0067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46d5367640 1 -- 172.21.15.27:0/1979037842 shutdown_connections 2024-04-20T10:55:35.980 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.979+0000 7f46d5367640 1 -- 172.21.15.27:0/1979037842 wait complete. 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.981+0000 7f46d5367640 1 Processor -- start 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.981+0000 7f46d5367640 1 -- start start 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.981+0000 7f46d5367640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f46d00b54f0 0x7f46d015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.981+0000 7f46d5367640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d00d0c90 0x7f46d01f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.981+0000 7f46d5367640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d0155950 0x7f46d01e7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.981+0000 7f46d5367640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f46d015fb90 con 0x7f46d0155950 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.981+0000 7f46d5367640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f46d015c540 con 0x7f46d00d0c90 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.981+0000 7f46d5367640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f46d015fd10 con 0x7f46d00b54f0 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46cf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d0155950 0x7f46d01e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.983 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46cf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d0155950 0x7f46d01e7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40464/0 (socket says 172.21.15.27:40464) 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46cf7fe640 1 -- 172.21.15.27:0/4190462169 learned_addr learned my addr 172.21.15.27:0/4190462169 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46ce7fc640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d00d0c90 0x7f46d01f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46cf7fe640 1 -- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f46d00b54f0 msgr2=0x7f46d015e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46ceffd640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f46d00b54f0 0x7f46d015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46cf7fe640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f46d00b54f0 0x7f46d015e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46cf7fe640 1 -- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d00d0c90 msgr2=0x7f46d01f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46cf7fe640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d00d0c90 0x7f46d01f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46cf7fe640 1 -- 172.21.15.27:0/4190462169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d01f57a0 con 0x7f46d0155950 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46ce7fc640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d00d0c90 0x7f46d01f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46cf7fe640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d0155950 0x7f46d01e7890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46c000c830 tx=0x7f46c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46b77fe640 1 -- 172.21.15.27:0/4190462169 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0018070 con 0x7f46d0155950 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46b77fe640 1 -- 172.21.15.27:0/4190462169 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c000ce20 con 0x7f46d0155950 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d015d180 con 0x7f46d0155950 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46b77fe640 1 -- 172.21.15.27:0/4190462169 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c000a880 con 0x7f46d0155950 2024-04-20T10:55:35.984 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.982+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d015cea0 con 0x7f46d0155950 2024-04-20T10:55:35.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.983+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d004fcf0 con 0x7f46d0155950 2024-04-20T10:55:35.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.984+0000 7f46b77fe640 1 -- 172.21.15.27:0/4190462169 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46c001b070 con 0x7f46d0155950 2024-04-20T10:55:35.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.984+0000 7f46b77fe640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f46bc06c8f0 0x7f46bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:35.985 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.984+0000 7f46b77fe640 1 -- 172.21.15.27:0/4190462169 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f46c008ac80 con 0x7f46d0155950 2024-04-20T10:55:35.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.984+0000 7f46ceffd640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f46bc06c8f0 0x7f46bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:35.986 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.985+0000 7f46ceffd640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f46bc06c8f0 0x7f46bc06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46c8000f60 tx=0x7f46c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:35.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:35.988+0000 7f46b77fe640 1 -- 172.21.15.27:0/4190462169 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f46c0059550 con 0x7f46d0155950 2024-04-20T10:55:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.210+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46d00cd0a0 con 0x7f46d0155950 2024-04-20T10:55:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.211+0000 7f46b77fe640 1 -- 172.21.15.27:0/4190462169 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f46c0058e50 con 0x7f46d0155950 2024-04-20T10:55:36.211 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:36.211 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T10:55:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 2 2024-04-20T10:55:36.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f46bc06c8f0 msgr2=0x7f46bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:36.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f46bc06c8f0 0x7f46bc06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46c8000f60 tx=0x7f46c8009040 comp rx=0 tx=0).stop 2024-04-20T10:55:36.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d0155950 msgr2=0x7f46d01e7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d0155950 0x7f46d01e7890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46c000c830 tx=0x7f46c000ccf0 comp rx=0 tx=0).stop 2024-04-20T10:55:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 shutdown_connections 2024-04-20T10:55:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f46bc06c8f0 0x7f46bc06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d0155950 0x7f46d01e7890 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d00d0c90 0x7f46d01f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 --2- 172.21.15.27:0/4190462169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f46d00b54f0 0x7f46d015e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 >> 172.21.15.27:0/4190462169 conn(0x7f46d00676c0 msgr2=0x7f46d0067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.212+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 shutdown_connections 2024-04-20T10:55:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:36.213+0000 7f46d5367640 1 -- 172.21.15.27:0/4190462169 wait complete. 2024-04-20T10:55:38.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:55:38.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.336+0000 7f410ae31640 1 Processor -- start 2024-04-20T10:55:38.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.336+0000 7f410ae31640 1 -- start start 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.337+0000 7f410ae31640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4104155950 0x7f41040cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.337+0000 7f410ae31640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 0x7f41040b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.337+0000 7f410ae31640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41040b5ea0 con 0x7f41040b5a30 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.337+0000 7f410ae31640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f410415b9c0 con 0x7f4104155950 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.337+0000 7f410ae31640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f410415bca0 con 0x7f41040cc330 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.337+0000 7f4108bcd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4104155950 0x7f41040cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.337+0000 7f4108bcd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4104155950 0x7f41040cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40474/0 (socket says 172.21.15.27:40474) 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.337+0000 7f4108bcd640 1 -- 172.21.15.27:0/447586371 learned_addr learned my addr 172.21.15.27:0/447586371 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.338+0000 7f4103fff640 1 --2- 172.21.15.27:0/447586371 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 0x7f41040b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.338+0000 7f4108bcd640 1 -- 172.21.15.27:0/447586371 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 msgr2=0x7f41040b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.338+0000 7f4108bcd640 1 --2- 172.21.15.27:0/447586371 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 0x7f41040b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.338+0000 7f4108bcd640 1 -- 172.21.15.27:0/447586371 >> v1:172.21.15.137:6789/0 conn(0x7f41040b5a30 legacy=0x7f41040d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.338+0000 7f4108bcd640 1 -- 172.21.15.27:0/447586371 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410415bf80 con 0x7f4104155950 2024-04-20T10:55:38.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.338+0000 7f4103fff640 1 --2- 172.21.15.27:0/447586371 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 0x7f41040b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f4108bcd640 1 --2- 172.21.15.27:0/447586371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 0x7f41040cbdf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009e50 tx=0x7f40fc02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f3d890d44f378c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f41037fe640 1 -- 172.21.15.27:0/447586371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc02cb90 con 0x7f4104155950 2024-04-20T10:55:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f41037fe640 1 -- 172.21.15.27:0/447586371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40fc02cd30 con 0x7f4104155950 2024-04-20T10:55:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f41037fe640 1 -- 172.21.15.27:0/447586371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc034610 con 0x7f4104155950 2024-04-20T10:55:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f410ae31640 1 -- 172.21.15.27:0/447586371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 msgr2=0x7f41040cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f410ae31640 1 --2- 172.21.15.27:0/447586371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 0x7f41040cbdf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009e50 tx=0x7f40fc02bff0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f410ae31640 1 -- 172.21.15.27:0/447586371 shutdown_connections 2024-04-20T10:55:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f410ae31640 1 --2- 172.21.15.27:0/447586371 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 0x7f41040b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f410ae31640 1 --2- 172.21.15.27:0/447586371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 0x7f41040cbdf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f410ae31640 1 -- 172.21.15.27:0/447586371 >> 172.21.15.27:0/447586371 conn(0x7f41040676c0 msgr2=0x7f4104067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f410ae31640 1 -- 172.21.15.27:0/447586371 shutdown_connections 2024-04-20T10:55:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.339+0000 7f410ae31640 1 -- 172.21.15.27:0/447586371 wait complete. 2024-04-20T10:55:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.341+0000 7f410ae31640 1 Processor -- start 2024-04-20T10:55:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.341+0000 7f410ae31640 1 -- start start 2024-04-20T10:55:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.341+0000 7f410ae31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41040b5a30 0x7f41040cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.341+0000 7f410ae31640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 0x7f41041f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.341+0000 7f410ae31640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 0x7f41041f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.341+0000 7f410ae31640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f410415fb90 con 0x7f4104155950 2024-04-20T10:55:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.341+0000 7f410ae31640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f410415c540 con 0x7f41040b5a30 2024-04-20T10:55:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.341+0000 7f410ae31640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f410415fd10 con 0x7f41040cc330 2024-04-20T10:55:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41093ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 0x7f41041f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41093ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 0x7f41041f5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40490/0 (socket says 172.21.15.27:40490) 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41093ce640 1 -- 172.21.15.27:0/1980553151 learned_addr learned my addr 172.21.15.27:0/1980553151 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f4103fff640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 0x7f41041f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f4108bcd640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41040b5a30 0x7f41040cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41093ce640 1 -- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 msgr2=0x7f41041f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41093ce640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 0x7f41041f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41093ce640 1 -- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41040b5a30 msgr2=0x7f41040cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41093ce640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41040b5a30 0x7f41040cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41093ce640 1 -- 172.21.15.27:0/1980553151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040c7ae0 con 0x7f4104155950 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41093ce640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 0x7f41041f5100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00c3f0 tx=0x7f40ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41017fa640 1 -- 172.21.15.27:0/1980553151 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec005220 con 0x7f4104155950 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41017fa640 1 -- 172.21.15.27:0/1980553151 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ec0053c0 con 0x7f4104155950 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410415d390 con 0x7f4104155950 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f41017fa640 1 -- 172.21.15.27:0/1980553151 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec007270 con 0x7f4104155950 2024-04-20T10:55:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.342+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f410415d950 con 0x7f4104155950 2024-04-20T10:55:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.343+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410404fcf0 con 0x7f4104155950 2024-04-20T10:55:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.344+0000 7f41017fa640 1 -- 172.21.15.27:0/1980553151 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40ec019020 con 0x7f4104155950 2024-04-20T10:55:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.344+0000 7f41017fa640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40e406c920 0x7f40e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.344+0000 7f41017fa640 1 -- 172.21.15.27:0/1980553151 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f40ec08a8c0 con 0x7f4104155950 2024-04-20T10:55:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.345+0000 7f4108bcd640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40e406c920 0x7f40e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.345+0000 7f4108bcd640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40e406c920 0x7f40e406ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40fc035000 tx=0x7f40fc030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.349+0000 7f41017fa640 1 -- 172.21.15.27:0/1980553151 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f40ec0590d0 con 0x7f4104155950 2024-04-20T10:55:38.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.573+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41040cd0a0 con 0x7f4104155950 2024-04-20T10:55:38.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.573+0000 7f41017fa640 1 -- 172.21.15.27:0/1980553151 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5518 (secure 0 0 0) 0x7f40ec0589d0 con 0x7f4104155950 2024-04-20T10:55:38.573 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:38.573 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-20T10:55:38.573 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2024-04-20T10:55:38.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.575+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40e406c920 msgr2=0x7f40e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.575+0000 7f410ae31640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40e406c920 0x7f40e406ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40fc035000 tx=0x7f40fc030000 comp rx=0 tx=0).stop 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.575+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 msgr2=0x7f41041f5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.575+0000 7f410ae31640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 0x7f41041f5100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00c3f0 tx=0x7f40ec00c8b0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.576+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 shutdown_connections 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.576+0000 7f410ae31640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40e406c920 0x7f40e406ede0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.576+0000 7f410ae31640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4104155950 0x7f41041f5100 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.576+0000 7f410ae31640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f41040cc330 0x7f41041f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.576+0000 7f410ae31640 1 --2- 172.21.15.27:0/1980553151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41040b5a30 0x7f41040cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.576+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 >> 172.21.15.27:0/1980553151 conn(0x7f41040676c0 msgr2=0x7f41040b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.576+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 shutdown_connections 2024-04-20T10:55:38.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.576+0000 7f410ae31640 1 -- 172.21.15.27:0/1980553151 wait complete. 2024-04-20T10:55:38.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:38.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.687+0000 7f1129cd5640 1 Processor -- start 2024-04-20T10:55:38.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.687+0000 7f1129cd5640 1 -- start start 2024-04-20T10:55:38.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.687+0000 7f1129cd5640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f11240c6fe0 0x7f11240b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.687+0000 7f1129cd5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f112415eb70 0x7f112415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.687+0000 7f1129cd5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11240cba80 con 0x7f1124155900 2024-04-20T10:55:38.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.687+0000 7f1129cd5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1124155ce0 con 0x7f112415eb70 2024-04-20T10:55:38.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.687+0000 7f1129cd5640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f11240cbd60 con 0x7f11240c6fe0 2024-04-20T10:55:38.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f1123fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f112415eb70 0x7f112415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f11237fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1124155900 0x7f112406c6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33504/0 (socket says 172.21.15.27:33504) 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f11237fe640 1 -- 172.21.15.27:0/578720882 learned_addr learned my addr 172.21.15.27:0/578720882 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f1122ffd640 1 --2- 172.21.15.27:0/578720882 >> v2:172.21.15.137:3300/0 conn(0x7f11240c6fe0 0x7f11240b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f1123fff640 1 -- 172.21.15.27:0/578720882 >> v2:172.21.15.137:3300/0 conn(0x7f11240c6fe0 msgr2=0x7f11240b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f1123fff640 1 --2- 172.21.15.27:0/578720882 >> v2:172.21.15.137:3300/0 conn(0x7f11240c6fe0 0x7f11240b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f1123fff640 1 -- 172.21.15.27:0/578720882 >> v1:172.21.15.27:6789/0 conn(0x7f1124155900 legacy=0x7f112406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f1123fff640 1 -- 172.21.15.27:0/578720882 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11240c6220 con 0x7f112415eb70 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f11227fc640 1 -- 172.21.15.27:0/578720882 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585254731 0 0) 0x7f11240cba80 con 0x7f1124155900 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.688+0000 7f1122ffd640 1 --2- 172.21.15.27:0/578720882 >> v2:172.21.15.137:3300/0 conn(0x7f11240c6fe0 0x7f11240b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f1123fff640 1 --2- 172.21.15.27:0/578720882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112415eb70 0x7f112415efc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11240ce3b0 tx=0x7f111c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9562dd44c5d0682 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f11227fc640 1 -- 172.21.15.27:0/578720882 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c039070 con 0x7f112415eb70 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f11227fc640 1 -- 172.21.15.27:0/578720882 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f111c02cc80 con 0x7f112415eb70 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f11227fc640 1 -- 172.21.15.27:0/578720882 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c034660 con 0x7f112415eb70 2024-04-20T10:55:38.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f1129cd5640 1 -- 172.21.15.27:0/578720882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112415eb70 msgr2=0x7f112415efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f1129cd5640 1 --2- 172.21.15.27:0/578720882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112415eb70 0x7f112415efc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11240ce3b0 tx=0x7f111c02bed0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f1129cd5640 1 -- 172.21.15.27:0/578720882 shutdown_connections 2024-04-20T10:55:38.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f1129cd5640 1 --2- 172.21.15.27:0/578720882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f112415eb70 0x7f112415efc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f1129cd5640 1 --2- 172.21.15.27:0/578720882 >> v2:172.21.15.137:3300/0 conn(0x7f11240c6fe0 0x7f11240b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f1129cd5640 1 -- 172.21.15.27:0/578720882 >> 172.21.15.27:0/578720882 conn(0x7f1124067cf0 msgr2=0x7f112406d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:38.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f1129cd5640 1 -- 172.21.15.27:0/578720882 shutdown_connections 2024-04-20T10:55:38.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.689+0000 7f1129cd5640 1 -- 172.21.15.27:0/578720882 wait complete. 2024-04-20T10:55:38.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.691+0000 7f1129cd5640 1 Processor -- start 2024-04-20T10:55:38.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.691+0000 7f1129cd5640 1 -- start start 2024-04-20T10:55:38.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.691+0000 7f1129cd5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11240c6fe0 0x7f1124156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.691+0000 7f1129cd5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1124155900 0x7f1124157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.691+0000 7f1129cd5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f112415eb70 0x7f1124157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.691+0000 7f1129cd5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f11240c67e0 con 0x7f1124155900 2024-04-20T10:55:38.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.691+0000 7f1129cd5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11240c6b50 con 0x7f112415eb70 2024-04-20T10:55:38.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.691+0000 7f1129cd5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f11240cb680 con 0x7f11240c6fe0 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.691+0000 7f1122ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1124155900 0x7f1124157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1122ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1124155900 0x7f1124157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40506/0 (socket says 172.21.15.27:40506) 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1122ffd640 1 -- 172.21.15.27:0/1512163297 learned_addr learned my addr 172.21.15.27:0/1512163297 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1122ffd640 1 -- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11240c6fe0 msgr2=0x7f1124156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1122ffd640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11240c6fe0 0x7f1124156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1123fff640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f112415eb70 0x7f1124157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1122ffd640 1 -- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f112415eb70 msgr2=0x7f1124157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1122ffd640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f112415eb70 0x7f1124157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1122ffd640 1 -- 172.21.15.27:0/1512163297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11240cd870 con 0x7f1124155900 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1122ffd640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1124155900 0x7f1124157880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f111800e810 tx=0x7f111800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f110bfff640 1 -- 172.21.15.27:0/1512163297 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111800f040 con 0x7f1124155900 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f110bfff640 1 -- 172.21.15.27:0/1512163297 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1118004230 con 0x7f1124155900 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11240d1100 con 0x7f1124155900 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f110bfff640 1 -- 172.21.15.27:0/1512163297 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111800c6d0 con 0x7f1124155900 2024-04-20T10:55:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.692+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11240d13a0 con 0x7f1124155900 2024-04-20T10:55:38.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.693+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f112404fcf0 con 0x7f1124155900 2024-04-20T10:55:38.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.694+0000 7f110bfff640 1 -- 172.21.15.27:0/1512163297 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f111800c870 con 0x7f1124155900 2024-04-20T10:55:38.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.694+0000 7f110bfff640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f111006c920 0x7f111006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:38.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.694+0000 7f110bfff640 1 -- 172.21.15.27:0/1512163297 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f111808b4a0 con 0x7f1124155900 2024-04-20T10:55:38.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.694+0000 7f11237fe640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f111006c920 0x7f111006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:38.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.695+0000 7f11237fe640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f111006c920 0x7f111006ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1114007fd0 tx=0x7f1114007920 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:38.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.698+0000 7f110bfff640 1 -- 172.21.15.27:0/1512163297 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1118017020 con 0x7f1124155900 2024-04-20T10:55:38.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.889+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11240cbf10 con 0x7f1124155900 2024-04-20T10:55:38.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.890+0000 7f110bfff640 1 -- 172.21.15.27:0/1512163297 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12848 (secure 0 0 0) 0x7f1118059c20 con 0x7f1124155900 2024-04-20T10:55:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:38.890 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:31.776479+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f111006c920 msgr2=0x7f111006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f111006c920 0x7f111006ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1114007fd0 tx=0x7f1114007920 comp rx=0 tx=0).stop 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1124155900 msgr2=0x7f1124157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1124155900 0x7f1124157880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f111800e810 tx=0x7f111800ecd0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 shutdown_connections 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f111006c920 0x7f111006ede0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f112415eb70 0x7f1124157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1124155900 0x7f1124157880 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 --2- 172.21.15.27:0/1512163297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11240c6fe0 0x7f1124156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:38.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 >> 172.21.15.27:0/1512163297 conn(0x7f1124067cf0 msgr2=0x7f112414fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:38.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 shutdown_connections 2024-04-20T10:55:38.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:38.892+0000 7f1129cd5640 1 -- 172.21.15.27:0/1512163297 wait complete. 2024-04-20T10:55:38.902 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:55:29.717620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:55:38.903 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:55:39.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.006+0000 7f25de752640 1 Processor -- start 2024-04-20T10:55:39.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.006+0000 7f25de752640 1 -- start start 2024-04-20T10:55:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.007+0000 7f25de752640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f25d815ee50 0x7f25d815c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.007+0000 7f25de752640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d815f9a0 con 0x7f25d815f230 2024-04-20T10:55:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.007+0000 7f25de752640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25d80cb870 con 0x7f25d80ccbd0 2024-04-20T10:55:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.007+0000 7f25de752640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f25d80cbb30 con 0x7f25d815ee50 2024-04-20T10:55:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.007+0000 7f25d77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f25d815f230 0x7f25d806cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33512/0 (socket says 172.21.15.27:33512) 2024-04-20T10:55:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.007+0000 7f25d77fe640 1 -- 172.21.15.27:0/626263887 learned_addr learned my addr 172.21.15.27:0/626263887 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.007+0000 7f25d7fff640 1 --2- 172.21.15.27:0/626263887 >> v2:172.21.15.137:3300/0 conn(0x7f25d815ee50 0x7f25d815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.007+0000 7f25d6ffd640 1 -- 172.21.15.27:0/626263887 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19363527 0 0) 0x7f25c8001110 con 0x7f25d815f230 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.007+0000 7f25d6ffd640 1 -- 172.21.15.27:0/626263887 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25c0003520 con 0x7f25d815f230 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.008+0000 7f25d6ffd640 1 -- 172.21.15.27:0/626263887 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 778054412 0 0) 0x7f25d80cb870 con 0x7f25d80ccbd0 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.008+0000 7f25d6ffd640 1 -- 172.21.15.27:0/626263887 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25c8001110 con 0x7f25d80ccbd0 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.008+0000 7f25d6ffd640 1 -- 172.21.15.27:0/626263887 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 462081051 0 0) 0x7f25c0003520 con 0x7f25d815f230 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.008+0000 7f25d7fff640 1 -- 172.21.15.27:0/626263887 >> v1:172.21.15.137:6789/0 conn(0x7f25d80ccbd0 legacy=0x7f25d8155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.008+0000 7f25d7fff640 1 -- 172.21.15.27:0/626263887 >> v1:172.21.15.27:6789/0 conn(0x7f25d815f230 legacy=0x7f25d806cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.008+0000 7f25d7fff640 1 -- 172.21.15.27:0/626263887 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d80cbe10 con 0x7f25d815ee50 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.008+0000 7f25d7fff640 1 --2- 172.21.15.27:0/626263887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 0x7f25d815c180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0097a0 tx=0x7f25cc02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef6d521ba6a1962d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25d6ffd640 1 -- 172.21.15.27:0/626263887 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8001840 con 0x7f25d815ee50 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25d6ffd640 1 -- 172.21.15.27:0/626263887 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25cc037070 con 0x7f25d815ee50 2024-04-20T10:55:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25d6ffd640 1 -- 172.21.15.27:0/626263887 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0317b0 con 0x7f25d815ee50 2024-04-20T10:55:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25de752640 1 -- 172.21.15.27:0/626263887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 msgr2=0x7f25d815c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25de752640 1 --2- 172.21.15.27:0/626263887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 0x7f25d815c180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0097a0 tx=0x7f25cc02b8b0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25de752640 1 -- 172.21.15.27:0/626263887 shutdown_connections 2024-04-20T10:55:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25de752640 1 --2- 172.21.15.27:0/626263887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 0x7f25d815c180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25de752640 1 -- 172.21.15.27:0/626263887 >> 172.21.15.27:0/626263887 conn(0x7f25d806e010 msgr2=0x7f25d80d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25de752640 1 -- 172.21.15.27:0/626263887 shutdown_connections 2024-04-20T10:55:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.009+0000 7f25de752640 1 -- 172.21.15.27:0/626263887 wait complete. 2024-04-20T10:55:39.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.010+0000 7f25de752640 1 Processor -- start 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25de752640 1 -- start start 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25de752640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25d80ccbd0 0x7f25d81f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25de752640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 0x7f25d8155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25de752640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25d815f230 0x7f25d815bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25de752640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25d80cc6b0 con 0x7f25d80ccbd0 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25de752640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25d80cc3d0 con 0x7f25d815ee50 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25de752640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f25d815cae0 con 0x7f25d815f230 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25d80ccbd0 0x7f25d81f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25d80ccbd0 0x7f25d81f0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40516/0 (socket says 172.21.15.27:40516) 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.011+0000 7f25d7fff640 1 -- 172.21.15.27:0/3522570147 learned_addr learned my addr 172.21.15.27:0/3522570147 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d77fe640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 0x7f25d8155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25dccef640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25d815f230 0x7f25d815bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d7fff640 1 -- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25d815f230 msgr2=0x7f25d815bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d7fff640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25d815f230 0x7f25d815bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d7fff640 1 -- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 msgr2=0x7f25d8155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d7fff640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 0x7f25d8155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d7fff640 1 -- 172.21.15.27:0/3522570147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d81f1080 con 0x7f25d80ccbd0 2024-04-20T10:55:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d77fe640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 0x7f25d8155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d7fff640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25d80ccbd0 0x7f25d81f0b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0098c0 tx=0x7f25cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d4ff9640 1 -- 172.21.15.27:0/3522570147 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc031ed0 con 0x7f25d80ccbd0 2024-04-20T10:55:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d4ff9640 1 -- 172.21.15.27:0/3522570147 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25cc032640 con 0x7f25d80ccbd0 2024-04-20T10:55:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25d4ff9640 1 -- 172.21.15.27:0/3522570147 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc032a30 con 0x7f25d80ccbd0 2024-04-20T10:55:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d81f1310 con 0x7f25d80ccbd0 2024-04-20T10:55:39.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.012+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d81f18d0 con 0x7f25d80ccbd0 2024-04-20T10:55:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.013+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d80688e0 con 0x7f25d80ccbd0 2024-04-20T10:55:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.013+0000 7f25d4ff9640 1 -- 172.21.15.27:0/3522570147 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25cc04c060 con 0x7f25d80ccbd0 2024-04-20T10:55:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.014+0000 7f25d4ff9640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f25c406c8f0 0x7f25c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.014+0000 7f25d4ff9640 1 -- 172.21.15.27:0/3522570147 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f25cc0acf10 con 0x7f25d80ccbd0 2024-04-20T10:55:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.014+0000 7f25d77fe640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f25c406c8f0 0x7f25c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.015+0000 7f25d77fe640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f25c406c8f0 0x7f25c406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25c80019b0 tx=0x7f25c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:39.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.018+0000 7f25d4ff9640 1 -- 172.21.15.27:0/3522570147 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f25cc040070 con 0x7f25d80ccbd0 2024-04-20T10:55:39.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.203+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25d804fcf0 con 0x7f25d80ccbd0 2024-04-20T10:55:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.203+0000 7f25d4ff9640 1 -- 172.21.15.27:0/3522570147 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f25cc07b610 con 0x7f25d80ccbd0 2024-04-20T10:55:39.203 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-20T10:55:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f25c406c8f0 msgr2=0x7f25c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f25c406c8f0 0x7f25c406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25c80019b0 tx=0x7f25c8009040 comp rx=0 tx=0).stop 2024-04-20T10:55:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25d80ccbd0 msgr2=0x7f25d81f0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25d80ccbd0 0x7f25d81f0b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0098c0 tx=0x7f25cc0046c0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 shutdown_connections 2024-04-20T10:55:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f25c406c8f0 0x7f25c406edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f25d815f230 0x7f25d815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d815ee50 0x7f25d8155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 --2- 172.21.15.27:0/3522570147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25d80ccbd0 0x7f25d81f0b40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 >> 172.21.15.27:0/3522570147 conn(0x7f25d806e010 msgr2=0x7f25d815d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 shutdown_connections 2024-04-20T10:55:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.205+0000 7f25de752640 1 -- 172.21.15.27:0/3522570147 wait complete. 2024-04-20T10:55:39.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T10:55:39.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.316+0000 7f42e9bed640 1 Processor -- start 2024-04-20T10:55:39.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.316+0000 7f42e9bed640 1 -- start start 2024-04-20T10:55:39.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.317+0000 7f42e9bed640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f42e4152640 0x7f42e40719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.317+0000 7f42e9bed640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e415ba80 con 0x7f42e4152260 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.317+0000 7f42e9bed640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e415bd60 con 0x7f42e40cd6f0 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.317+0000 7f42e9bed640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f42e415c040 con 0x7f42e4152640 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.317+0000 7f42e37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f42e4152260 0x7f42e40c86e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33516/0 (socket says 172.21.15.27:33516) 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.317+0000 7f42e37fe640 1 -- 172.21.15.27:0/2859985210 learned_addr learned my addr 172.21.15.27:0/2859985210 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.317+0000 7f42e2ffd640 1 --2- 172.21.15.27:0/2859985210 >> v2:172.21.15.137:3300/0 conn(0x7f42e4152640 0x7f42e40719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.317+0000 7f42e27fc640 1 -- 172.21.15.27:0/2859985210 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705666264 0 0) 0x7f42e415ba80 con 0x7f42e4152260 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.317+0000 7f42e27fc640 1 -- 172.21.15.27:0/2859985210 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42cc003520 con 0x7f42e4152260 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.318+0000 7f42e27fc640 1 -- 172.21.15.27:0/2859985210 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2504149921 0 0) 0x7f42cc003520 con 0x7f42e4152260 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.318+0000 7f42e2ffd640 1 -- 172.21.15.27:0/2859985210 >> v1:172.21.15.137:6789/0 conn(0x7f42e40cd6f0 legacy=0x7f42e4158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.318+0000 7f42e2ffd640 1 -- 172.21.15.27:0/2859985210 >> v1:172.21.15.27:6789/0 conn(0x7f42e4152260 legacy=0x7f42e40c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.318+0000 7f42e2ffd640 1 -- 172.21.15.27:0/2859985210 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e415c320 con 0x7f42e4152640 2024-04-20T10:55:39.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.318+0000 7f42e2ffd640 1 --2- 172.21.15.27:0/2859985210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42e4152640 0x7f42e40719c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f42d80097a0 tx=0x7f42d802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f344210aae9db833 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:39.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e27fc640 1 -- 172.21.15.27:0/2859985210 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8038070 con 0x7f42e4152640 2024-04-20T10:55:39.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e27fc640 1 -- 172.21.15.27:0/2859985210 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d8031500 con 0x7f42e4152640 2024-04-20T10:55:39.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e27fc640 1 -- 172.21.15.27:0/2859985210 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4001670 con 0x7f42e4152640 2024-04-20T10:55:39.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e9bed640 1 -- 172.21.15.27:0/2859985210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42e4152640 msgr2=0x7f42e40719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e9bed640 1 --2- 172.21.15.27:0/2859985210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42e4152640 0x7f42e40719c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f42d80097a0 tx=0x7f42d802b820 comp rx=0 tx=0).stop 2024-04-20T10:55:39.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e9bed640 1 -- 172.21.15.27:0/2859985210 shutdown_connections 2024-04-20T10:55:39.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e9bed640 1 --2- 172.21.15.27:0/2859985210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42e4152640 0x7f42e40719c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e9bed640 1 -- 172.21.15.27:0/2859985210 >> 172.21.15.27:0/2859985210 conn(0x7f42e40ccbd0 msgr2=0x7f42e40ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:39.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e9bed640 1 -- 172.21.15.27:0/2859985210 shutdown_connections 2024-04-20T10:55:39.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.319+0000 7f42e9bed640 1 -- 172.21.15.27:0/2859985210 wait complete. 2024-04-20T10:55:39.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.320+0000 7f42e9bed640 1 Processor -- start 2024-04-20T10:55:39.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e9bed640 1 -- start start 2024-04-20T10:55:39.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e9bed640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e40cd6f0 0x7f42e4143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e9bed640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f42e4152260 0x7f42e413e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e9bed640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42e4152640 0x7f42e413ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e9bed640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f42e415c600 con 0x7f42e40cd6f0 2024-04-20T10:55:39.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e9bed640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4074260 con 0x7f42e4152640 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e9bed640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f42e415b720 con 0x7f42e4152260 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e40cd6f0 0x7f42e4143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e40cd6f0 0x7f42e4143d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40530/0 (socket says 172.21.15.27:40530) 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.321+0000 7f42e37fe640 1 -- 172.21.15.27:0/3966588804 learned_addr learned my addr 172.21.15.27:0/3966588804 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e2ffd640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f42e4152260 0x7f42e413e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e3fff640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42e4152640 0x7f42e413ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e37fe640 1 -- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f42e4152260 msgr2=0x7f42e413e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e37fe640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f42e4152260 0x7f42e413e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e37fe640 1 -- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42e4152640 msgr2=0x7f42e413ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e37fe640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42e4152640 0x7f42e413ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e37fe640 1 -- 172.21.15.27:0/3966588804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e413fea0 con 0x7f42e40cd6f0 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e37fe640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e40cd6f0 0x7f42e4143d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c730 tx=0x7f42d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42cbfff640 1 -- 172.21.15.27:0/3966588804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4017070 con 0x7f42e40cd6f0 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42cbfff640 1 -- 172.21.15.27:0/3966588804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d4005af0 con 0x7f42e40cd6f0 2024-04-20T10:55:39.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e415a3e0 con 0x7f42e40cd6f0 2024-04-20T10:55:39.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42cbfff640 1 -- 172.21.15.27:0/3966588804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4007070 con 0x7f42e40cd6f0 2024-04-20T10:55:39.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.322+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e41f1a10 con 0x7f42e40cd6f0 2024-04-20T10:55:39.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.323+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e41f1c50 con 0x7f42e40cd6f0 2024-04-20T10:55:39.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.324+0000 7f42cbfff640 1 -- 172.21.15.27:0/3966588804 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42d401a070 con 0x7f42e40cd6f0 2024-04-20T10:55:39.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.324+0000 7f42cbfff640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f42d006c920 0x7f42d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.324+0000 7f42cbfff640 1 -- 172.21.15.27:0/3966588804 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f42d408ac40 con 0x7f42e40cd6f0 2024-04-20T10:55:39.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.324+0000 7f42e2ffd640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f42d006c920 0x7f42d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.325+0000 7f42e2ffd640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f42d006c920 0x7f42d006ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f42d8035000 tx=0x7f42d8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:39.328 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.329+0000 7f42cbfff640 1 -- 172.21.15.27:0/3966588804 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f42e413fea0 con 0x7f42e40cd6f0 2024-04-20T10:55:39.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.513+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f42e404fcf0 con 0x7f42e40cd6f0 2024-04-20T10:55:39.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.513+0000 7f42cbfff640 1 -- 172.21.15.27:0/3966588804 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f42d4059450 con 0x7f42e40cd6f0 2024-04-20T10:55:39.513 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:39.513 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2024-04-20T10:55:39.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2024-04-20T10:55:39.513 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2024-04-20T10:55:39.513 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2024-04-20T10:55:39.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-04-20T10:55:39.514 INFO:teuthology.orchestra.run.smithi027.stdout:] 2024-04-20T10:55:39.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f42d006c920 msgr2=0x7f42d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f42d006c920 0x7f42d006ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f42d8035000 tx=0x7f42d8008030 comp rx=0 tx=0).stop 2024-04-20T10:55:39.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e40cd6f0 msgr2=0x7f42e4143d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e40cd6f0 0x7f42e4143d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c730 tx=0x7f42d400cbf0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 shutdown_connections 2024-04-20T10:55:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f42d006c920 0x7f42d006ede0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42e4152640 0x7f42e413ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f42e4152260 0x7f42e413e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 --2- 172.21.15.27:0/3966588804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42e40cd6f0 0x7f42e4143d10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 >> 172.21.15.27:0/3966588804 conn(0x7f42e40ccbd0 msgr2=0x7f42e40c9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 shutdown_connections 2024-04-20T10:55:39.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.515+0000 7f42e9bed640 1 -- 172.21.15.27:0/3966588804 wait complete. 2024-04-20T10:55:39.526 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T10:55:39.526 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T10:55:39.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.630+0000 7f5886966640 1 Processor -- start 2024-04-20T10:55:39.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.630+0000 7f5886966640 1 -- start start 2024-04-20T10:55:39.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f5886966640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f58800c89e0 0x7f5880158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f5886966640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58800c9120 con 0x7f58800cd370 2024-04-20T10:55:39.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f5886966640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588006bad0 con 0x7f58800c8600 2024-04-20T10:55:39.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f5886966640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f588006bdb0 con 0x7f58800c89e0 2024-04-20T10:55:39.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f5884f03640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f58800cd370 0x7f58800c5480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33518/0 (socket says 172.21.15.27:33518) 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f5884f03640 1 -- 172.21.15.27:0/4026532919 learned_addr learned my addr 172.21.15.27:0/4026532919 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f587f7fe640 1 --2- 172.21.15.27:0/4026532919 >> v2:172.21.15.137:3300/0 conn(0x7f58800c89e0 0x7f5880158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937291162 0 0) 0x7f58800c9120 con 0x7f58800cd370 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5868003520 con 0x7f58800cd370 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 356785893 0 0) 0x7f5868003520 con 0x7f58800cd370 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 >> v2:172.21.15.137:3300/0 conn(0x7f58800c89e0 msgr2=0x7f5880158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f587effd640 1 --2- 172.21.15.27:0/4026532919 >> v2:172.21.15.137:3300/0 conn(0x7f58800c89e0 0x7f5880158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.631+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 >> v1:172.21.15.137:6789/0 conn(0x7f58800c8600 legacy=0x7f5880151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588006c090 con 0x7f58800cd370 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f58780019f0 con 0x7f58800cd370 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 >> v1:172.21.15.27:6789/0 conn(0x7f58800cd370 legacy=0x7f58800c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587effd640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58680039b0 0x7f5868003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587effd640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5868004800 0x7f5868004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587effd640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58680051c0 0x7f586800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f588006bdb0 con 0x7f58680039b0 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f586800c070 con 0x7f5868004800 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f586800c270 con 0x7f58680051c0 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587ffff640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58680039b0 0x7f5868003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587f7fe640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5868004800 0x7f5868004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f5884f03640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58680051c0 0x7f586800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587ffff640 1 -- 172.21.15.27:0/4026532919 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58680051c0 msgr2=0x7f586800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587ffff640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58680051c0 0x7f586800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587ffff640 1 -- 172.21.15.27:0/4026532919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5868004800 msgr2=0x7f5868004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587ffff640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5868004800 0x7f5868004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587ffff640 1 -- 172.21.15.27:0/4026532919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5868003520 con 0x7f58680039b0 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f5884f03640 1 -- 172.21.15.27:0/4026532919 reap_dead start 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.632+0000 7f587f7fe640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5868004800 0x7f5868004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f587ffff640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58680039b0 0x7f5868003dd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5874001750 tx=0x7f5868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1287bce2d259b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874001370 con 0x7f58680039b0 2024-04-20T10:55:39.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58740052d0 con 0x7f58680039b0 2024-04-20T10:55:39.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f587effd640 1 -- 172.21.15.27:0/4026532919 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874003830 con 0x7f58680039b0 2024-04-20T10:55:39.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f5886966640 1 -- 172.21.15.27:0/4026532919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58680039b0 msgr2=0x7f5868003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f5886966640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58680039b0 0x7f5868003dd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5874001750 tx=0x7f5868005830 comp rx=0 tx=0).stop 2024-04-20T10:55:39.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f5886966640 1 -- 172.21.15.27:0/4026532919 shutdown_connections 2024-04-20T10:55:39.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f5886966640 1 --2- 172.21.15.27:0/4026532919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58680039b0 0x7f5868003dd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f5886966640 1 -- 172.21.15.27:0/4026532919 >> 172.21.15.27:0/4026532919 conn(0x7f58800cc880 msgr2=0x7f58800ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:39.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.633+0000 7f5886966640 1 -- 172.21.15.27:0/4026532919 shutdown_connections 2024-04-20T10:55:39.636 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.634+0000 7f5886966640 1 -- 172.21.15.27:0/4026532919 wait complete. 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.635+0000 7f5886966640 1 Processor -- start 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.635+0000 7f5886966640 1 -- start start 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5886966640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58680039b0 0x7f5880157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5886966640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5880073720 0x7f58801f0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5886966640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58801f0bb0 0x7f5880072180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5886966640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5880070e50 con 0x7f5880073720 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5886966640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f588006c650 con 0x7f58680039b0 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5886966640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5880070fd0 con 0x7f58801f0bb0 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5880073720 0x7f58801f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5880073720 0x7f58801f0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40548/0 (socket says 172.21.15.27:40548) 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587f7fe640 1 -- 172.21.15.27:0/2745008277 learned_addr learned my addr 172.21.15.27:0/2745008277 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587ffff640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58680039b0 0x7f5880157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5884f03640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58801f0bb0 0x7f5880072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587f7fe640 1 -- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58801f0bb0 msgr2=0x7f5880072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587f7fe640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58801f0bb0 0x7f5880072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587f7fe640 1 -- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58680039b0 msgr2=0x7f5880157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587f7fe640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58680039b0 0x7f5880157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587f7fe640 1 -- 172.21.15.27:0/2745008277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58801ed080 con 0x7f5880073720 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5884f03640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58801f0bb0 0x7f5880072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587f7fe640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5880073720 0x7f58801f0670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58700114c0 tx=0x7f5870011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587cff9640 1 -- 172.21.15.27:0/2745008277 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870015870 con 0x7f5880073720 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587cff9640 1 -- 172.21.15.27:0/2745008277 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5870015a10 con 0x7f5880073720 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f587cff9640 1 -- 172.21.15.27:0/2745008277 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870015dc0 con 0x7f5880073720 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5880072dc0 con 0x7f5880073720 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.636+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58801ed620 con 0x7f5880073720 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.637+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f588004fcf0 con 0x7f5880073720 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.638+0000 7f587cff9640 1 -- 172.21.15.27:0/2745008277 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5870013400 con 0x7f5880073720 2024-04-20T10:55:39.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.638+0000 7f587cff9640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f586c06c8f0 0x7f586c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:39.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.638+0000 7f587cff9640 1 -- 172.21.15.27:0/2745008277 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f587008bf70 con 0x7f5880073720 2024-04-20T10:55:39.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.638+0000 7f587ffff640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f586c06c8f0 0x7f586c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:39.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.639+0000 7f587ffff640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f586c06c8f0 0x7f586c06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5874033000 tx=0x7f5874009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:39.642 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.642+0000 7f587cff9640 1 -- 172.21.15.27:0/2745008277 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5870019030 con 0x7f5880073720 2024-04-20T10:55:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:39.828+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f58800c3780 con 0x7f5880073720 2024-04-20T10:55:40.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.770+0000 7f587cff9640 1 -- 172.21.15.27:0/2745008277 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f587005a6f0 con 0x7f5880073720 2024-04-20T10:55:40.770 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2024-04-20T10:55:40.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.773+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f586c06c8f0 msgr2=0x7f586c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:40.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.773+0000 7f5886966640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f586c06c8f0 0x7f586c06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5874033000 tx=0x7f5874009030 comp rx=0 tx=0).stop 2024-04-20T10:55:40.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.773+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5880073720 msgr2=0x7f58801f0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:40.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.773+0000 7f5886966640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5880073720 0x7f58801f0670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f58700114c0 tx=0x7f5870011980 comp rx=0 tx=0).stop 2024-04-20T10:55:40.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.774+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 shutdown_connections 2024-04-20T10:55:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.774+0000 7f5886966640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f586c06c8f0 0x7f586c06edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.774+0000 7f5886966640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58801f0bb0 0x7f5880072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.774+0000 7f5886966640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5880073720 0x7f58801f0670 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.774+0000 7f5886966640 1 --2- 172.21.15.27:0/2745008277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58680039b0 0x7f5880157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.774+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 >> 172.21.15.27:0/2745008277 conn(0x7f58800cc880 msgr2=0x7f58800ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.774+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 shutdown_connections 2024-04-20T10:55:40.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.774+0000 7f5886966640 1 -- 172.21.15.27:0/2745008277 wait complete. 2024-04-20T10:55:40.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T10:55:40.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.904+0000 7ff7d8a11640 1 Processor -- start 2024-04-20T10:55:40.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.904+0000 7ff7d8a11640 1 -- start start 2024-04-20T10:55:40.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d8a11640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff7d4155970 0x7ff7d40c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d8a11640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d40cc5a0 con 0x7ff7d415edc0 2024-04-20T10:55:40.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d8a11640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7d40cc880 con 0x7ff7d415f8b0 2024-04-20T10:55:40.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d8a11640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d40cc350 con 0x7ff7d4155970 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff7d415edc0 0x7ff7d415f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33526/0 (socket says 172.21.15.27:33526) 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d259c640 1 -- 172.21.15.27:0/1076786599 learned_addr learned my addr 172.21.15.27:0/1076786599 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565785899 0 0) 0x7ff7d40cc5a0 con 0x7ff7d415edc0 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d2d9d640 1 --2- 172.21.15.27:0/1076786599 >> v2:172.21.15.137:3300/0 conn(0x7ff7d4155970 0x7ff7d40c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7c0003520 con 0x7ff7d415edc0 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 606626686 0 0) 0x7ff7c0003520 con 0x7ff7d415edc0 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 >> v2:172.21.15.137:3300/0 conn(0x7ff7d4155970 msgr2=0x7ff7d40c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 --2- 172.21.15.27:0/1076786599 >> v2:172.21.15.137:3300/0 conn(0x7ff7d4155970 0x7ff7d40c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 >> v1:172.21.15.137:6789/0 conn(0x7ff7d415f8b0 legacy=0x7ff7d4155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d40c30b0 con 0x7ff7d415edc0 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7ff7c4001670 con 0x7ff7d415edc0 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 >> v1:172.21.15.27:6789/0 conn(0x7ff7d415edc0 legacy=0x7ff7d415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c00039b0 0x7ff7c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7c0004800 0x7ff7c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7c00051c0 0x7ff7c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d40cc350 con 0x7ff7c0004800 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c000c070 con 0x7ff7c00039b0 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c000c270 con 0x7ff7c00051c0 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.905+0000 7ff7c9d9b640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7c0004800 0x7ff7c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.906+0000 7ff7d259c640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c00039b0 0x7ff7c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.906+0000 7ff7d2d9d640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7c00051c0 0x7ff7c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.906+0000 7ff7c9d9b640 1 -- 172.21.15.27:0/1076786599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7c00051c0 msgr2=0x7ff7c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.906+0000 7ff7c9d9b640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7c00051c0 0x7ff7c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.906+0000 7ff7c9d9b640 1 -- 172.21.15.27:0/1076786599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c00039b0 msgr2=0x7ff7c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.906+0000 7ff7c9d9b640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c00039b0 0x7ff7c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.906+0000 7ff7d2d9d640 1 -- 172.21.15.27:0/1076786599 reap_dead start 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.906+0000 7ff7c9d9b640 1 -- 172.21.15.27:0/1076786599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c0003520 con 0x7ff7c0004800 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7c9d9b640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7c0004800 0x7ff7c0004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc001750 tx=0x7ff7c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a17513cb956992 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:40.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc001370 con 0x7ff7c0004800 2024-04-20T10:55:40.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7bc0052d0 con 0x7ff7c0004800 2024-04-20T10:55:40.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d1d9b640 1 -- 172.21.15.27:0/1076786599 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc003830 con 0x7ff7c0004800 2024-04-20T10:55:40.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d8a11640 1 -- 172.21.15.27:0/1076786599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7c0004800 msgr2=0x7ff7c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:40.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d8a11640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7c0004800 0x7ff7c0004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc001750 tx=0x7ff7c0005830 comp rx=0 tx=0).stop 2024-04-20T10:55:40.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d8a11640 1 -- 172.21.15.27:0/1076786599 shutdown_connections 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d8a11640 1 --2- 172.21.15.27:0/1076786599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7c0004800 0x7ff7c0004c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d8a11640 1 -- 172.21.15.27:0/1076786599 >> 172.21.15.27:0/1076786599 conn(0x7ff7d406fc90 msgr2=0x7ff7d4070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d8a11640 1 -- 172.21.15.27:0/1076786599 shutdown_connections 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.907+0000 7ff7d8a11640 1 -- 172.21.15.27:0/1076786599 wait complete. 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.909+0000 7ff7d8a11640 1 Processor -- start 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.909+0000 7ff7d8a11640 1 -- start start 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.909+0000 7ff7d8a11640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7c0004800 0x7ff7d40c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.909+0000 7ff7d8a11640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7d415c720 0x7ff7d40ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.909+0000 7ff7d8a11640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7d41f0710 0x7ff7d415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.909+0000 7ff7d8a11640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d40c8b70 con 0x7ff7d41f0710 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.909+0000 7ff7d8a11640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d40c3670 con 0x7ff7d415c720 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.909+0000 7ff7d8a11640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d40c9620 con 0x7ff7c0004800 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7d41f0710 0x7ff7d415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7d41f0710 0x7ff7d415bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40554/0 (socket says 172.21.15.27:40554) 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d2d9d640 1 -- 172.21.15.27:0/3208801288 learned_addr learned my addr 172.21.15.27:0/3208801288 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d259c640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7c0004800 0x7ff7d40c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7c9d9b640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7d415c720 0x7ff7d40ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d2d9d640 1 -- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7c0004800 msgr2=0x7ff7d40c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d2d9d640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7c0004800 0x7ff7d40c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d2d9d640 1 -- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7d415c720 msgr2=0x7ff7d40ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d2d9d640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7d415c720 0x7ff7d40ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d2d9d640 1 -- 172.21.15.27:0/3208801288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d41f4ae0 con 0x7ff7d41f0710 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d2d9d640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7d41f0710 0x7ff7d415bab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff7d406bef0 tx=0x7ff7cc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7cb7fe640 1 -- 172.21.15.27:0/3208801288 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00fdb0 con 0x7ff7d41f0710 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7cb7fe640 1 -- 172.21.15.27:0/3208801288 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7cc01d070 con 0x7ff7d41f0710 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d415d810 con 0x7ff7d41f0710 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7cb7fe640 1 -- 172.21.15.27:0/3208801288 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc015670 con 0x7ff7d41f0710 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.910+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d41f4000 con 0x7ff7d41f0710 2024-04-20T10:55:40.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.911+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d404fcf0 con 0x7ff7d41f0710 2024-04-20T10:55:40.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.912+0000 7ff7cb7fe640 1 -- 172.21.15.27:0/3208801288 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7cc015810 con 0x7ff7d41f0710 2024-04-20T10:55:40.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.912+0000 7ff7cb7fe640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff7b406c920 0x7ff7b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:40.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.912+0000 7ff7cb7fe640 1 -- 172.21.15.27:0/3208801288 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7ff7cc08c1a0 con 0x7ff7d41f0710 2024-04-20T10:55:40.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.913+0000 7ff7d259c640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff7b406c920 0x7ff7b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:40.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.913+0000 7ff7d259c640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff7b406c920 0x7ff7b406ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4050f60 tx=0x7ff7c40078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:40.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:40.917+0000 7ff7cb7fe640 1 -- 172.21.15.27:0/3208801288 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff7cc010040 con 0x7ff7d41f0710 2024-04-20T10:55:41.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.100+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff7d40c5480 con 0x7ff7d41f0710 2024-04-20T10:55:41.543 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:55:41.545 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:55:41.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.776+0000 7ff7cb7fe640 1 -- 172.21.15.27:0/3208801288 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7ff7cc05a160 con 0x7ff7d41f0710 2024-04-20T10:55:41.776 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2024-04-20T10:55:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.781+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff7b406c920 msgr2=0x7ff7b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.781+0000 7ff7d8a11640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff7b406c920 0x7ff7b406ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4050f60 tx=0x7ff7c40078f0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.781+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7d41f0710 msgr2=0x7ff7d415bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:41.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.781+0000 7ff7d8a11640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7d41f0710 0x7ff7d415bab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff7d406bef0 tx=0x7ff7cc00f580 comp rx=0 tx=0).stop 2024-04-20T10:55:41.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.786+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 shutdown_connections 2024-04-20T10:55:41.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.786+0000 7ff7d8a11640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff7b406c920 0x7ff7b406ede0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.786+0000 7ff7d8a11640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7d41f0710 0x7ff7d415bab0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.786+0000 7ff7d8a11640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7d415c720 0x7ff7d40ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.786+0000 7ff7d8a11640 1 --2- 172.21.15.27:0/3208801288 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7c0004800 0x7ff7d40c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.786+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 >> 172.21.15.27:0/3208801288 conn(0x7ff7d406fc90 msgr2=0x7ff7d4153980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:41.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.786+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 shutdown_connections 2024-04-20T10:55:41.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.790+0000 7ff7d8a11640 1 -- 172.21.15.27:0/3208801288 wait complete. 2024-04-20T10:55:41.807 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T10:55:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.913+0000 7fe98eb32640 1 Processor -- start 2024-04-20T10:55:41.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.913+0000 7fe98eb32640 1 -- start start 2024-04-20T10:55:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98eb32640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe988155a20 0x7fe9880c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98eb32640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe98815ec40 0x7fe9880cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98eb32640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe98815f210 con 0x7fe9880c62e0 2024-04-20T10:55:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98eb32640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe988155e00 con 0x7fe988155a20 2024-04-20T10:55:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98eb32640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fe98815ba30 con 0x7fe98815ec40 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98c8ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe988155a20 0x7fe9880c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe987fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe9880c62e0 0x7fe98815bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33540/0 (socket says 172.21.15.27:33540) 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98c8ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe988155a20 0x7fe9880c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40560/0 (socket says 172.21.15.27:40560) 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe987fff640 1 -- 172.21.15.27:0/3678688318 learned_addr learned my addr 172.21.15.27:0/3678688318 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98d0cf640 1 --2- 172.21.15.27:0/3678688318 >> v2:172.21.15.137:3300/0 conn(0x7fe98815ec40 0x7fe9880cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98c8ce640 1 -- 172.21.15.27:0/3678688318 >> v2:172.21.15.137:3300/0 conn(0x7fe98815ec40 msgr2=0x7fe9880cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98c8ce640 1 --2- 172.21.15.27:0/3678688318 >> v2:172.21.15.137:3300/0 conn(0x7fe98815ec40 0x7fe9880cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98c8ce640 1 -- 172.21.15.27:0/3678688318 >> v1:172.21.15.27:6789/0 conn(0x7fe9880c62e0 legacy=0x7fe98815bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe9877fe640 1 -- 172.21.15.27:0/3678688318 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123985408 0 0) 0x7fe98815f210 con 0x7fe9880c62e0 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.914+0000 7fe98c8ce640 1 -- 172.21.15.27:0/3678688318 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98815f4f0 con 0x7fe988155a20 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.915+0000 7fe98c8ce640 1 --2- 172.21.15.27:0/3678688318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe988155a20 0x7fe9880c5bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9880b5060 tx=0x7fe97c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418b659dc74c41db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.915+0000 7fe9877fe640 1 -- 172.21.15.27:0/3678688318 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c034070 con 0x7fe988155a20 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.915+0000 7fe9877fe640 1 -- 172.21.15.27:0/3678688318 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c030610 con 0x7fe988155a20 2024-04-20T10:55:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.915+0000 7fe9877fe640 1 -- 172.21.15.27:0/3678688318 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c0309e0 con 0x7fe988155a20 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.916+0000 7fe98eb32640 1 -- 172.21.15.27:0/3678688318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe988155a20 msgr2=0x7fe9880c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.916+0000 7fe98eb32640 1 --2- 172.21.15.27:0/3678688318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe988155a20 0x7fe9880c5bb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9880b5060 tx=0x7fe97c02b960 comp rx=0 tx=0).stop 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.916+0000 7fe98eb32640 1 -- 172.21.15.27:0/3678688318 shutdown_connections 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.916+0000 7fe98eb32640 1 --2- 172.21.15.27:0/3678688318 >> v2:172.21.15.137:3300/0 conn(0x7fe98815ec40 0x7fe9880cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.916+0000 7fe98eb32640 1 --2- 172.21.15.27:0/3678688318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe988155a20 0x7fe9880c5bb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.916+0000 7fe98eb32640 1 -- 172.21.15.27:0/3678688318 >> 172.21.15.27:0/3678688318 conn(0x7fe9880678a0 msgr2=0x7fe988067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.916+0000 7fe98eb32640 1 -- 172.21.15.27:0/3678688318 shutdown_connections 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.916+0000 7fe98eb32640 1 -- 172.21.15.27:0/3678688318 wait complete. 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.917+0000 7fe98eb32640 1 Processor -- start 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.917+0000 7fe98eb32640 1 -- start start 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98eb32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9880c62e0 0x7fe9880b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98eb32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe988155a20 0x7fe98815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98eb32640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98815ec40 0x7fe98806df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98eb32640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe98815c5d0 con 0x7fe9880c62e0 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98eb32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe98815fab0 con 0x7fe988155a20 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98eb32640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe9880c67c0 con 0x7fe98815ec40 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98c8ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9880c62e0 0x7fe9880b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98c8ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9880c62e0 0x7fe9880b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40562/0 (socket says 172.21.15.27:40562) 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98c8ce640 1 -- 172.21.15.27:0/1621508128 learned_addr learned my addr 172.21.15.27:0/1621508128 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe987fff640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe988155a20 0x7fe98815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98d0cf640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98815ec40 0x7fe98806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98c8ce640 1 -- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98815ec40 msgr2=0x7fe98806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98c8ce640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98815ec40 0x7fe98806df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98c8ce640 1 -- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe988155a20 msgr2=0x7fe98815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98c8ce640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe988155a20 0x7fe98815d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98c8ce640 1 -- 172.21.15.27:0/1621508128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9880cc710 con 0x7fe9880c62e0 2024-04-20T10:55:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98d0cf640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98815ec40 0x7fe98806df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe987fff640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe988155a20 0x7fe98815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.918+0000 7fe98c8ce640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9880c62e0 0x7fe9880b3640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009890 tx=0x7fe97c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.919+0000 7fe9857fa640 1 -- 172.21.15.27:0/1621508128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c034040 con 0x7fe9880c62e0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.919+0000 7fe9857fa640 1 -- 172.21.15.27:0/1621508128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97c039a10 con 0x7fe9880c62e0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.919+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9881f4770 con 0x7fe9880c62e0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.919+0000 7fe9857fa640 1 -- 172.21.15.27:0/1621508128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c039e20 con 0x7fe9880c62e0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.919+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9881f4d30 con 0x7fe9880c62e0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.920+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98804fcf0 con 0x7fe9880c62e0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.920+0000 7fe9857fa640 1 -- 172.21.15.27:0/1621508128 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe97c037070 con 0x7fe9880c62e0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.921+0000 7fe9857fa640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe96c06c920 0x7fe96c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.921+0000 7fe9857fa640 1 -- 172.21.15.27:0/1621508128 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fe97c0ad420 con 0x7fe9880c62e0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.921+0000 7fe987fff640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe96c06c920 0x7fe96c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.922+0000 7fe987fff640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe96c06c920 0x7fe96c06ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9880b6bb0 tx=0x7fe974007c30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:41.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:41.926+0000 7fe9857fa640 1 -- 172.21.15.27:0/1621508128 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe97c07b820 con 0x7fe9880c62e0 2024-04-20T10:55:42.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.148+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe9880cd1c0 con 0x7fe9880c62e0 2024-04-20T10:55:42.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.776+0000 7fe9857fa640 1 -- 172.21.15.27:0/1621508128 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fe97c07b120 con 0x7fe9880c62e0 2024-04-20T10:55:42.777 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T10:55:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.778+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe96c06c920 msgr2=0x7fe96c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.778+0000 7fe98eb32640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe96c06c920 0x7fe96c06ede0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9880b6bb0 tx=0x7fe974007c30 comp rx=0 tx=0).stop 2024-04-20T10:55:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.778+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9880c62e0 msgr2=0x7fe9880b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.778+0000 7fe98eb32640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9880c62e0 0x7fe9880b3640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009890 tx=0x7fe97c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.779+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 shutdown_connections 2024-04-20T10:55:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.779+0000 7fe98eb32640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe96c06c920 0x7fe96c06ede0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.779+0000 7fe98eb32640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98815ec40 0x7fe98806df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.779+0000 7fe98eb32640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe988155a20 0x7fe98815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.779+0000 7fe98eb32640 1 --2- 172.21.15.27:0/1621508128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9880c62e0 0x7fe9880b3640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.779+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 >> 172.21.15.27:0/1621508128 conn(0x7fe9880678a0 msgr2=0x7fe9880d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.779+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 shutdown_connections 2024-04-20T10:55:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.779+0000 7fe98eb32640 1 -- 172.21.15.27:0/1621508128 wait complete. 2024-04-20T10:55:42.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:42.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.904+0000 7faca6d15640 1 Processor -- start 2024-04-20T10:55:42.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.904+0000 7faca6d15640 1 -- start start 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca6d15640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faca0155800 0x7faca015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca6d15640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7faca00cbf50 0x7faca00c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca6d15640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00c6500 0x7faca00d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca6d15640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7faca00ce610 con 0x7faca0155800 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca6d15640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7faca00ce180 con 0x7faca00cbf50 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca6d15640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7faca015c4d0 con 0x7faca00c6500 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca4ab1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faca0155800 0x7faca015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca4ab1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faca0155800 0x7faca015bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40568/0 (socket says 172.21.15.27:40568) 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca4ab1640 1 -- 172.21.15.27:0/1706735658 learned_addr learned my addr 172.21.15.27:0/1706735658 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7fac9ffff640 1 --2- 172.21.15.27:0/1706735658 >> v2:172.21.15.137:3300/0 conn(0x7faca00cbf50 0x7faca00c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca52b2640 1 --2- 172.21.15.27:0/1706735658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00c6500 0x7faca00d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.905+0000 7faca4ab1640 1 -- 172.21.15.27:0/1706735658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00c6500 msgr2=0x7faca00d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:42.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.906+0000 7faca4ab1640 1 --2- 172.21.15.27:0/1706735658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00c6500 0x7faca00d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.906+0000 7faca4ab1640 1 -- 172.21.15.27:0/1706735658 >> v2:172.21.15.137:3300/0 conn(0x7faca00cbf50 msgr2=0x7faca00c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.906+0000 7faca4ab1640 1 --2- 172.21.15.27:0/1706735658 >> v2:172.21.15.137:3300/0 conn(0x7faca00cbf50 0x7faca00c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.906+0000 7faca4ab1640 1 -- 172.21.15.27:0/1706735658 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca00cd290 con 0x7faca0155800 2024-04-20T10:55:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.906+0000 7fac9ffff640 1 --2- 172.21.15.27:0/1706735658 >> v2:172.21.15.137:3300/0 conn(0x7faca00cbf50 0x7faca00c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:42.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.906+0000 7faca4ab1640 1 --2- 172.21.15.27:0/1706735658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 0x7faca015bf90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faca006ec00 tx=0x7fac9802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2b9ce11fe4d863 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.906+0000 7fac9f7fe640 1 -- 172.21.15.27:0/1706735658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9802c9e0 con 0x7faca0155800 2024-04-20T10:55:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.906+0000 7fac9f7fe640 1 -- 172.21.15.27:0/1706735658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9802cb80 con 0x7faca0155800 2024-04-20T10:55:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7fac9f7fe640 1 -- 172.21.15.27:0/1706735658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac980343f0 con 0x7faca0155800 2024-04-20T10:55:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7faca6d15640 1 -- 172.21.15.27:0/1706735658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 msgr2=0x7faca015bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7faca6d15640 1 --2- 172.21.15.27:0/1706735658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 0x7faca015bf90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faca006ec00 tx=0x7fac9802be40 comp rx=0 tx=0).stop 2024-04-20T10:55:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7faca6d15640 1 -- 172.21.15.27:0/1706735658 shutdown_connections 2024-04-20T10:55:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7faca6d15640 1 --2- 172.21.15.27:0/1706735658 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00c6500 0x7faca00d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7faca6d15640 1 --2- 172.21.15.27:0/1706735658 >> v2:172.21.15.137:3300/0 conn(0x7faca00cbf50 0x7faca00c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7faca6d15640 1 --2- 172.21.15.27:0/1706735658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 0x7faca015bf90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7faca6d15640 1 -- 172.21.15.27:0/1706735658 >> 172.21.15.27:0/1706735658 conn(0x7faca0067ae0 msgr2=0x7faca0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:42.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7faca6d15640 1 -- 172.21.15.27:0/1706735658 shutdown_connections 2024-04-20T10:55:42.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.907+0000 7faca6d15640 1 -- 172.21.15.27:0/1706735658 wait complete. 2024-04-20T10:55:42.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.908+0000 7faca6d15640 1 Processor -- start 2024-04-20T10:55:42.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.908+0000 7faca6d15640 1 -- start start 2024-04-20T10:55:42.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca6d15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faca00c6500 0x7faca0158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:42.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca6d15640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00cbf50 0x7faca01594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca6d15640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 0x7faca0159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca6d15640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faca00cdb30 con 0x7faca0155800 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca6d15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faca00cd850 con 0x7faca00c6500 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca6d15640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7faca00cde10 con 0x7faca00cbf50 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca52b2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 0x7faca0159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca52b2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 0x7faca0159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40576/0 (socket says 172.21.15.27:40576) 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca52b2640 1 -- 172.21.15.27:0/19835704 learned_addr learned my addr 172.21.15.27:0/19835704 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7faca4ab1640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faca00c6500 0x7faca0158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.909+0000 7fac9ffff640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00cbf50 0x7faca01594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7faca52b2640 1 -- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00cbf50 msgr2=0x7faca01594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7faca52b2640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00cbf50 0x7faca01594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7faca52b2640 1 -- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faca00c6500 msgr2=0x7faca0158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7faca52b2640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faca00c6500 0x7faca0158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7faca52b2640 1 -- 172.21.15.27:0/19835704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca015c1c0 con 0x7faca0155800 2024-04-20T10:55:42.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7faca4ab1640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faca00c6500 0x7faca0158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7faca52b2640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 0x7faca0159a20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac8800d830 tx=0x7fac8800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7fac9d7fa640 1 -- 172.21.15.27:0/19835704 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88018020 con 0x7faca0155800 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7fac9d7fa640 1 -- 172.21.15.27:0/19835704 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac880096c0 con 0x7faca0155800 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca01f5040 con 0x7faca0155800 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7fac9d7fa640 1 -- 172.21.15.27:0/19835704 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88009a90 con 0x7faca0155800 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.910+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca01f5600 con 0x7faca0155800 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.911+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca00688e0 con 0x7faca0155800 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.911+0000 7fac9d7fa640 1 -- 172.21.15.27:0/19835704 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac8801b020 con 0x7faca0155800 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.912+0000 7fac9d7fa640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fac8006c8f0 0x7fac8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.912+0000 7fac9d7fa640 1 -- 172.21.15.27:0/19835704 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fac8808c2a0 con 0x7faca0155800 2024-04-20T10:55:42.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.912+0000 7faca4ab1640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fac8006c8f0 0x7fac8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:42.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.913+0000 7faca4ab1640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fac8006c8f0 0x7fac8006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac98035000 tx=0x7fac98030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:42.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:42.916+0000 7fac9d7fa640 1 -- 172.21.15.27:0/19835704 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fac88022020 con 0x7faca0155800 2024-04-20T10:55:43.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.102+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faca004fcf0 con 0x7faca0155800 2024-04-20T10:55:43.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.103+0000 7fac9d7fa640 1 -- 172.21.15.27:0/19835704 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16582 (secure 0 0 0) 0x7fac88011340 con 0x7faca0155800 2024-04-20T10:55:43.104 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:43.104 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":19,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:42.150354+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:43.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fac8006c8f0 msgr2=0x7fac8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fac8006c8f0 0x7fac8006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac98035000 tx=0x7fac98030000 comp rx=0 tx=0).stop 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 msgr2=0x7faca0159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 0x7faca0159a20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac8800d830 tx=0x7fac8800dcf0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 shutdown_connections 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fac8006c8f0 0x7fac8006edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faca0155800 0x7faca0159a20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7faca00cbf50 0x7faca01594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 --2- 172.21.15.27:0/19835704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faca00c6500 0x7faca0158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 >> 172.21.15.27:0/19835704 conn(0x7faca0067ae0 msgr2=0x7faca00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 shutdown_connections 2024-04-20T10:55:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.105+0000 7faca6d15640 1 -- 172.21.15.27:0/19835704 wait complete. 2024-04-20T10:55:43.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:43.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.215+0000 7f96c1b32640 1 Processor -- start 2024-04-20T10:55:43.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.216+0000 7f96c1b32640 1 -- start start 2024-04-20T10:55:43.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.216+0000 7f96c1b32640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96bc0c9b00 0x7f96bc159150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.216+0000 7f96c1b32640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c3eb0 0x7f96bc153010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.216+0000 7f96c1b32640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96bc159850 con 0x7f96bc0c9ee0 2024-04-20T10:55:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.216+0000 7f96c1b32640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f96bc159b30 con 0x7f96bc0c9b00 2024-04-20T10:55:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.216+0000 7f96c1b32640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f96bc159e10 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.216+0000 7f96bb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96bc0c9b00 0x7f96bc159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.217+0000 7f96bb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96bc0c9b00 0x7f96bc159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40592/0 (socket says 172.21.15.27:40592) 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.217+0000 7f96bb7fe640 1 -- 172.21.15.27:0/3018922664 learned_addr learned my addr 172.21.15.27:0/3018922664 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.217+0000 7f96bbfff640 1 --2- 172.21.15.27:0/3018922664 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c3eb0 0x7f96bc153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.217+0000 7f96bb7fe640 1 -- 172.21.15.27:0/3018922664 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c3eb0 msgr2=0x7f96bc153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.217+0000 7f96bb7fe640 1 --2- 172.21.15.27:0/3018922664 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c3eb0 0x7f96bc153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.217+0000 7f96bb7fe640 1 -- 172.21.15.27:0/3018922664 >> v1:172.21.15.137:6789/0 conn(0x7f96bc0c9ee0 legacy=0x7f96bc0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.217+0000 7f96bb7fe640 1 -- 172.21.15.27:0/3018922664 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc0ca720 con 0x7f96bc0c9b00 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.217+0000 7f96bb7fe640 1 --2- 172.21.15.27:0/3018922664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c9b00 0x7f96bc159150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009e50 tx=0x7f96ac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b7e00292c9badb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96ba7fc640 1 -- 172.21.15.27:0/3018922664 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac02ca70 con 0x7f96bc0c9b00 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96ba7fc640 1 -- 172.21.15.27:0/3018922664 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac02cc10 con 0x7f96bc0c9b00 2024-04-20T10:55:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96ba7fc640 1 -- 172.21.15.27:0/3018922664 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac0343f0 con 0x7f96bc0c9b00 2024-04-20T10:55:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96c1b32640 1 -- 172.21.15.27:0/3018922664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c9b00 msgr2=0x7f96bc159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96c1b32640 1 --2- 172.21.15.27:0/3018922664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c9b00 0x7f96bc159150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009e50 tx=0x7f96ac02bed0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96c1b32640 1 -- 172.21.15.27:0/3018922664 shutdown_connections 2024-04-20T10:55:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96c1b32640 1 --2- 172.21.15.27:0/3018922664 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c3eb0 0x7f96bc153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96c1b32640 1 --2- 172.21.15.27:0/3018922664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c9b00 0x7f96bc159150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96c1b32640 1 -- 172.21.15.27:0/3018922664 >> 172.21.15.27:0/3018922664 conn(0x7f96bc0cee90 msgr2=0x7f96bc0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96c1b32640 1 -- 172.21.15.27:0/3018922664 shutdown_connections 2024-04-20T10:55:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.218+0000 7f96c1b32640 1 -- 172.21.15.27:0/3018922664 wait complete. 2024-04-20T10:55:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.219+0000 7f96c1b32640 1 Processor -- start 2024-04-20T10:55:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96c1b32640 1 -- start start 2024-04-20T10:55:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96c1b32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c3eb0 0x7f96bc152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96c1b32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96bc0c9b00 0x7f96bc1f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96c1b32640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c9ee0 0x7f96bc14c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96c1b32640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc15c880 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96c1b32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc15c700 con 0x7f96bc0c9b00 2024-04-20T10:55:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96c1b32640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f96bc0b4e10 con 0x7f96bc0c9ee0 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96bb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c3eb0 0x7f96bc152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96bb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c3eb0 0x7f96bc152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40608/0 (socket says 172.21.15.27:40608) 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.220+0000 7f96bb7fe640 1 -- 172.21.15.27:0/573505955 learned_addr learned my addr 172.21.15.27:0/573505955 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96baffd640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96bc0c9b00 0x7f96bc1f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96bbfff640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c9ee0 0x7f96bc14c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96bb7fe640 1 -- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c9ee0 msgr2=0x7f96bc14c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96bb7fe640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c9ee0 0x7f96bc14c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96bb7fe640 1 -- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96bc0c9b00 msgr2=0x7f96bc1f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96bb7fe640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96bc0c9b00 0x7f96bc1f29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96bb7fe640 1 -- 172.21.15.27:0/573505955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc0ce1e0 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96bb7fe640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c3eb0 0x7f96bc152670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96ac036040 tx=0x7f96ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96a3fff640 1 -- 172.21.15.27:0/573505955 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac043020 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96a3fff640 1 -- 172.21.15.27:0/573505955 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac03e450 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96a3fff640 1 -- 172.21.15.27:0/573505955 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac009510 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc150cf0 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.221+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc15b730 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.222+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96bc04fcf0 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.223+0000 7f96a3fff640 1 -- 172.21.15.27:0/573505955 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96ac046020 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.223+0000 7f96a3fff640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f96a806c8f0 0x7f96a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.223+0000 7f96a3fff640 1 -- 172.21.15.27:0/573505955 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f96ac0ae3d0 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.223+0000 7f96baffd640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f96a806c8f0 0x7f96a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.224+0000 7f96baffd640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f96a806c8f0 0x7f96a806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0c94d0 tx=0x7f96b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.227+0000 7f96a3fff640 1 -- 172.21.15.27:0/573505955 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f96ac04d030 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.412+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96bc0b49e0 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.413+0000 7f96a3fff640 1 -- 172.21.15.27:0/573505955 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16582 (secure 0 0 0) 0x7f96ac07c800 con 0x7f96bc0c3eb0 2024-04-20T10:55:43.414 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":19,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:42.150354+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f96a806c8f0 msgr2=0x7f96a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f96a806c8f0 0x7f96a806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0c94d0 tx=0x7f96b0009040 comp rx=0 tx=0).stop 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c3eb0 msgr2=0x7f96bc152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c3eb0 0x7f96bc152670 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96ac036040 tx=0x7f96ac004a40 comp rx=0 tx=0).stop 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 shutdown_connections 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f96a806c8f0 0x7f96a806edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f96bc0c9ee0 0x7f96bc14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96bc0c9b00 0x7f96bc1f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 --2- 172.21.15.27:0/573505955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0c3eb0 0x7f96bc152670 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 >> 172.21.15.27:0/573505955 conn(0x7f96bc0cee90 msgr2=0x7f96bc158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:43.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 shutdown_connections 2024-04-20T10:55:43.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.415+0000 7f96c1b32640 1 -- 172.21.15.27:0/573505955 wait complete. 2024-04-20T10:55:43.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T10:55:43.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.529+0000 7f30c8704640 1 Processor -- start 2024-04-20T10:55:43.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.530+0000 7f30c8704640 1 -- start start 2024-04-20T10:55:43.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.530+0000 7f30c8704640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30c0155800 0x7f30c015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.530+0000 7f30c8704640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f30c00cbf50 0x7f30c00c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.530+0000 7f30c8704640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00c6500 0x7f30c00d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.530+0000 7f30c8704640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f30c00ce610 con 0x7f30c0155800 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.530+0000 7f30c8704640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f30c00ce180 con 0x7f30c00cbf50 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.530+0000 7f30c8704640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f30c015c4d0 con 0x7f30c00c6500 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c64a0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30c0155800 0x7f30c015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c64a0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30c0155800 0x7f30c015bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40624/0 (socket says 172.21.15.27:40624) 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c64a0640 1 -- 172.21.15.27:0/3807991180 learned_addr learned my addr 172.21.15.27:0/3807991180 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c6ca1640 1 --2- 172.21.15.27:0/3807991180 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00c6500 0x7f30c00d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c5c9f640 1 --2- 172.21.15.27:0/3807991180 >> v2:172.21.15.137:3300/0 conn(0x7f30c00cbf50 0x7f30c00c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c64a0640 1 -- 172.21.15.27:0/3807991180 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00c6500 msgr2=0x7f30c00d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c64a0640 1 --2- 172.21.15.27:0/3807991180 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00c6500 0x7f30c00d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c64a0640 1 -- 172.21.15.27:0/3807991180 >> v2:172.21.15.137:3300/0 conn(0x7f30c00cbf50 msgr2=0x7f30c00c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c64a0640 1 --2- 172.21.15.27:0/3807991180 >> v2:172.21.15.137:3300/0 conn(0x7f30c00cbf50 0x7f30c00c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c64a0640 1 -- 172.21.15.27:0/3807991180 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c00cd290 con 0x7f30c0155800 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.531+0000 7f30c6ca1640 1 --2- 172.21.15.27:0/3807991180 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00c6500 0x7f30c00d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c64a0640 1 --2- 172.21.15.27:0/3807991180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c0155800 0x7f30c015bf90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f30c006ec00 tx=0x7f30b802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333a373699159119 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c549e640 1 -- 172.21.15.27:0/3807991180 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b802c9e0 con 0x7f30c0155800 2024-04-20T10:55:43.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c549e640 1 -- 172.21.15.27:0/3807991180 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b802cb80 con 0x7f30c0155800 2024-04-20T10:55:43.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c549e640 1 -- 172.21.15.27:0/3807991180 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b80343f0 con 0x7f30c0155800 2024-04-20T10:55:43.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c8704640 1 -- 172.21.15.27:0/3807991180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c0155800 msgr2=0x7f30c015bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c8704640 1 --2- 172.21.15.27:0/3807991180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c0155800 0x7f30c015bf90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f30c006ec00 tx=0x7f30b802be40 comp rx=0 tx=0).stop 2024-04-20T10:55:43.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c8704640 1 -- 172.21.15.27:0/3807991180 shutdown_connections 2024-04-20T10:55:43.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c8704640 1 --2- 172.21.15.27:0/3807991180 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00c6500 0x7f30c00d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c8704640 1 --2- 172.21.15.27:0/3807991180 >> v2:172.21.15.137:3300/0 conn(0x7f30c00cbf50 0x7f30c00c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c8704640 1 --2- 172.21.15.27:0/3807991180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c0155800 0x7f30c015bf90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c8704640 1 -- 172.21.15.27:0/3807991180 >> 172.21.15.27:0/3807991180 conn(0x7f30c0067ae0 msgr2=0x7f30c0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:43.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c8704640 1 -- 172.21.15.27:0/3807991180 shutdown_connections 2024-04-20T10:55:43.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.532+0000 7f30c8704640 1 -- 172.21.15.27:0/3807991180 wait complete. 2024-04-20T10:55:43.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.533+0000 7f30c8704640 1 Processor -- start 2024-04-20T10:55:43.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c8704640 1 -- start start 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c8704640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c00c6500 0x7f30c015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c8704640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00cbf50 0x7f30c00c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c8704640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0155800 0x7f30c015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c8704640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00cdb30 con 0x7f30c00c6500 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c8704640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30c00cd850 con 0x7f30c0155800 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c8704640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f30c00cde10 con 0x7f30c00cbf50 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c64a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c00c6500 0x7f30c015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c64a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c00c6500 0x7f30c015a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40632/0 (socket says 172.21.15.27:40632) 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.534+0000 7f30c64a0640 1 -- 172.21.15.27:0/837646653 learned_addr learned my addr 172.21.15.27:0/837646653 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30c64a0640 1 -- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00cbf50 msgr2=0x7f30c00c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30c64a0640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00cbf50 0x7f30c00c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30c64a0640 1 -- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0155800 msgr2=0x7f30c015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30c64a0640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0155800 0x7f30c015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30c64a0640 1 -- 172.21.15.27:0/837646653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c0152710 con 0x7f30c00c6500 2024-04-20T10:55:43.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30c64a0640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c00c6500 0x7f30c015a060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036040 tx=0x7f30b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30beffd640 1 -- 172.21.15.27:0/837646653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8043020 con 0x7f30c00c6500 2024-04-20T10:55:43.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30beffd640 1 -- 172.21.15.27:0/837646653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b803e450 con 0x7f30c00c6500 2024-04-20T10:55:43.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c01501f0 con 0x7f30c00c6500 2024-04-20T10:55:43.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30beffd640 1 -- 172.21.15.27:0/837646653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8009510 con 0x7f30c00c6500 2024-04-20T10:55:43.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.535+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c0150790 con 0x7f30c00c6500 2024-04-20T10:55:43.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.536+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c00688e0 con 0x7f30c00c6500 2024-04-20T10:55:43.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.536+0000 7f30beffd640 1 -- 172.21.15.27:0/837646653 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30b8046020 con 0x7f30c00c6500 2024-04-20T10:55:43.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.537+0000 7f30beffd640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f30ac06c8f0 0x7f30ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.537+0000 7f30beffd640 1 -- 172.21.15.27:0/837646653 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f30b80ae460 con 0x7f30c00c6500 2024-04-20T10:55:43.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.537+0000 7f30c5c9f640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f30ac06c8f0 0x7f30ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.538+0000 7f30c5c9f640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f30ac06c8f0 0x7f30ac06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30c01e69b0 tx=0x7f30b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.541+0000 7f30beffd640 1 -- 172.21.15.27:0/837646653 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f30b804d030 con 0x7f30c00c6500 2024-04-20T10:55:43.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.766+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f30c00693a0 con 0x7f30c00c6500 2024-04-20T10:55:43.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.786+0000 7f30beffd640 1 -- 172.21.15.27:0/837646653 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7f30c00693a0 con 0x7f30c00c6500 2024-04-20T10:55:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f30ac06c8f0 msgr2=0x7f30ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f30ac06c8f0 0x7f30ac06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f30c01e69b0 tx=0x7f30b4008040 comp rx=0 tx=0).stop 2024-04-20T10:55:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c00c6500 msgr2=0x7f30c015a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c00c6500 0x7f30c015a060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036040 tx=0x7f30b8004a40 comp rx=0 tx=0).stop 2024-04-20T10:55:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 shutdown_connections 2024-04-20T10:55:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f30ac06c8f0 0x7f30ac06edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c0155800 0x7f30c015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f30c00cbf50 0x7f30c00c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 --2- 172.21.15.27:0/837646653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c00c6500 0x7f30c015a060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 >> 172.21.15.27:0/837646653 conn(0x7f30c0067ae0 msgr2=0x7f30c0154a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:43.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 shutdown_connections 2024-04-20T10:55:43.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.788+0000 7f30c8704640 1 -- 172.21.15.27:0/837646653 wait complete. 2024-04-20T10:55:43.800 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2024-04-20T10:55:43.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-20T10:55:43.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.904+0000 7f2eddac0640 1 Processor -- start 2024-04-20T10:55:43.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.904+0000 7f2eddac0640 1 -- start start 2024-04-20T10:55:43.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.904+0000 7f2eddac0640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2ed815ee50 0x7f2ed815c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.904+0000 7f2eddac0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2ed815f230 0x7f2ed806cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.904+0000 7f2eddac0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 0x7f2ed8155410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.904+0000 7f2eddac0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2ed815f9a0 con 0x7f2ed815f230 2024-04-20T10:55:43.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.904+0000 7f2eddac0640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f2ed80cb870 con 0x7f2ed815ee50 2024-04-20T10:55:43.907 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.904+0000 7f2eddac0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed806d490 con 0x7f2ed80ccbd0 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed6ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2ed815f230 0x7f2ed806cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed6ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2ed815f230 0x7f2ed806cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40644/0 (socket says 172.21.15.27:40644) 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed6ffd640 1 -- 172.21.15.27:0/3328755377 learned_addr learned my addr 172.21.15.27:0/3328755377 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed77fe640 1 --2- 172.21.15.27:0/3328755377 >> v2:172.21.15.137:3300/0 conn(0x7f2ed815ee50 0x7f2ed815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed7fff640 1 --2- 172.21.15.27:0/3328755377 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 0x7f2ed8155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed6ffd640 1 -- 172.21.15.27:0/3328755377 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 msgr2=0x7f2ed8155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed6ffd640 1 --2- 172.21.15.27:0/3328755377 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 0x7f2ed8155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed6ffd640 1 -- 172.21.15.27:0/3328755377 >> v2:172.21.15.137:3300/0 conn(0x7f2ed815ee50 msgr2=0x7f2ed815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed6ffd640 1 --2- 172.21.15.27:0/3328755377 >> v2:172.21.15.137:3300/0 conn(0x7f2ed815ee50 0x7f2ed815c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.905+0000 7f2ed6ffd640 1 -- 172.21.15.27:0/3328755377 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80cbb30 con 0x7f2ed815f230 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2ed6ffd640 1 --2- 172.21.15.27:0/3328755377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815f230 0x7f2ed806cf50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ed808eeb0 tx=0x7f2ec802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc61537c61de399f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2ed67fc640 1 -- 172.21.15.27:0/3328755377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8034070 con 0x7f2ed815f230 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2ed67fc640 1 -- 172.21.15.27:0/3328755377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec8030860 con 0x7f2ed815f230 2024-04-20T10:55:43.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2ed67fc640 1 -- 172.21.15.27:0/3328755377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8030c30 con 0x7f2ed815f230 2024-04-20T10:55:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2eddac0640 1 -- 172.21.15.27:0/3328755377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815f230 msgr2=0x7f2ed806cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2eddac0640 1 --2- 172.21.15.27:0/3328755377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815f230 0x7f2ed806cf50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ed808eeb0 tx=0x7f2ec802b9a0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2eddac0640 1 -- 172.21.15.27:0/3328755377 shutdown_connections 2024-04-20T10:55:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2eddac0640 1 --2- 172.21.15.27:0/3328755377 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 0x7f2ed8155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2eddac0640 1 --2- 172.21.15.27:0/3328755377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815f230 0x7f2ed806cf50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2eddac0640 1 --2- 172.21.15.27:0/3328755377 >> v2:172.21.15.137:3300/0 conn(0x7f2ed815ee50 0x7f2ed815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.906+0000 7f2eddac0640 1 -- 172.21.15.27:0/3328755377 >> 172.21.15.27:0/3328755377 conn(0x7f2ed806e010 msgr2=0x7f2ed80d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.907+0000 7f2eddac0640 1 -- 172.21.15.27:0/3328755377 shutdown_connections 2024-04-20T10:55:43.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.907+0000 7f2eddac0640 1 -- 172.21.15.27:0/3328755377 wait complete. 2024-04-20T10:55:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.908+0000 7f2eddac0640 1 Processor -- start 2024-04-20T10:55:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.908+0000 7f2eddac0640 1 -- start start 2024-04-20T10:55:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.908+0000 7f2eddac0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 0x7f2ed8155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2eddac0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815ee50 0x7f2ed80ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2eddac0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed815f230 0x7f2ed80c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2eddac0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80cc3d0 con 0x7f2ed815ee50 2024-04-20T10:55:43.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2eddac0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80cc0f0 con 0x7f2ed815f230 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2eddac0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed80cc6b0 con 0x7f2ed80ccbd0 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815ee50 0x7f2ed80ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815ee50 0x7f2ed80ca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40646/0 (socket says 172.21.15.27:40646) 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed6ffd640 1 -- 172.21.15.27:0/1064772216 learned_addr learned my addr 172.21.15.27:0/1064772216 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed77fe640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 0x7f2ed8155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed7fff640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed815f230 0x7f2ed80c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed6ffd640 1 -- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 msgr2=0x7f2ed8155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed6ffd640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 0x7f2ed8155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed6ffd640 1 -- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed815f230 msgr2=0x7f2ed80c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed6ffd640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed815f230 0x7f2ed80c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed6ffd640 1 -- 172.21.15.27:0/1064772216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed8156260 con 0x7f2ed815ee50 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ed6ffd640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815ee50 0x7f2ed80ca800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ec802b9e0 tx=0x7f2ec8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ebffff640 1 -- 172.21.15.27:0/1064772216 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8048070 con 0x7f2ed815ee50 2024-04-20T10:55:43.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ebffff640 1 -- 172.21.15.27:0/1064772216 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec8030660 con 0x7f2ed815ee50 2024-04-20T10:55:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed80c8110 con 0x7f2ed815ee50 2024-04-20T10:55:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2ebffff640 1 -- 172.21.15.27:0/1064772216 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8034040 con 0x7f2ed815ee50 2024-04-20T10:55:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.909+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed80c86d0 con 0x7f2ed815ee50 2024-04-20T10:55:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.910+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed80688e0 con 0x7f2ed815ee50 2024-04-20T10:55:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.911+0000 7f2ebffff640 1 -- 172.21.15.27:0/1064772216 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ec804b070 con 0x7f2ed815ee50 2024-04-20T10:55:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.911+0000 7f2ebffff640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2ec406c8f0 0x7f2ec406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.911+0000 7f2ebffff640 1 -- 172.21.15.27:0/1064772216 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ec80ada20 con 0x7f2ed815ee50 2024-04-20T10:55:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.911+0000 7f2ed77fe640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2ec406c8f0 0x7f2ec406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:43.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.912+0000 7f2ed77fe640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2ec406c8f0 0x7f2ec406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009aa0 tx=0x7f2ecc009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:43.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:43.915+0000 7f2ebffff640 1 -- 172.21.15.27:0/1064772216 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ec8030830 con 0x7f2ed815ee50 2024-04-20T10:55:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.141+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f2ed80d1060 con 0x7f2ed815ee50 2024-04-20T10:55:44.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.794+0000 7f2ebffff640 1 -- 172.21.15.27:0/1064772216 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7f2ec807b800 con 0x7f2ed815ee50 2024-04-20T10:55:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.796+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2ec406c8f0 msgr2=0x7f2ec406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.796+0000 7f2eddac0640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2ec406c8f0 0x7f2ec406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009aa0 tx=0x7f2ecc009190 comp rx=0 tx=0).stop 2024-04-20T10:55:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.796+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815ee50 msgr2=0x7f2ed80ca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.796+0000 7f2eddac0640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815ee50 0x7f2ed80ca800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ec802b9e0 tx=0x7f2ec8004730 comp rx=0 tx=0).stop 2024-04-20T10:55:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.797+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 shutdown_connections 2024-04-20T10:55:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.797+0000 7f2eddac0640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2ec406c8f0 0x7f2ec406edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.797+0000 7f2eddac0640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed815f230 0x7f2ed80c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.797+0000 7f2eddac0640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ed815ee50 0x7f2ed80ca800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:44.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.797+0000 7f2eddac0640 1 --2- 172.21.15.27:0/1064772216 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2ed80ccbd0 0x7f2ed8155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:44.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.797+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 >> 172.21.15.27:0/1064772216 conn(0x7f2ed806e010 msgr2=0x7f2ed80cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:44.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.797+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 shutdown_connections 2024-04-20T10:55:44.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.797+0000 7f2eddac0640 1 -- 172.21.15.27:0/1064772216 wait complete. 2024-04-20T10:55:44.810 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T10:55:44.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.919+0000 7fbe18c06640 1 Processor -- start 2024-04-20T10:55:44.920 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.919+0000 7fbe18c06640 1 -- start start 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.920+0000 7fbe18c06640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cd9d0 0x7fbe1406e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.920+0000 7fbe18c06640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe140ca2a0 con 0x7fbe140cc150 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.920+0000 7fbe18c06640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe140cde20 con 0x7fbe140c97b0 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.920+0000 7fbe18c06640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbe140cb430 con 0x7fbe140cd9d0 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.920+0000 7fbe11d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbe140cc150 0x7fbe140c9b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53400/0 (socket says 172.21.15.27:53400) 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.920+0000 7fbe11d9b640 1 -- 172.21.15.27:0/2810638439 learned_addr learned my addr 172.21.15.27:0/2810638439 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.921+0000 7fbe12d9d640 1 --2- 172.21.15.27:0/2810638439 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cd9d0 0x7fbe1406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.921+0000 7fbe1159a640 1 -- 172.21.15.27:0/2810638439 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1270170108 0 0) 0x7fbe140ca2a0 con 0x7fbe140cc150 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.921+0000 7fbe1159a640 1 -- 172.21.15.27:0/2810638439 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe00003520 con 0x7fbe140cc150 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.921+0000 7fbe1159a640 1 -- 172.21.15.27:0/2810638439 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268492499 0 0) 0x7fbe140cde20 con 0x7fbe140c97b0 2024-04-20T10:55:44.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.921+0000 7fbe1159a640 1 -- 172.21.15.27:0/2810638439 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe140ca2a0 con 0x7fbe140c97b0 2024-04-20T10:55:44.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.921+0000 7fbe12d9d640 1 -- 172.21.15.27:0/2810638439 >> v1:172.21.15.137:6789/0 conn(0x7fbe140c97b0 legacy=0x7fbe1415c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:44.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.921+0000 7fbe12d9d640 1 -- 172.21.15.27:0/2810638439 >> v1:172.21.15.27:6789/0 conn(0x7fbe140cc150 legacy=0x7fbe140c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:44.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.921+0000 7fbe12d9d640 1 -- 172.21.15.27:0/2810638439 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe14073a00 con 0x7fbe140cd9d0 2024-04-20T10:55:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.922+0000 7fbe12d9d640 1 --2- 172.21.15.27:0/2810638439 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cd9d0 0x7fbe1406e130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0097d0 tx=0x7fbe0c02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6574ca84179def8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.922+0000 7fbe1159a640 1 -- 172.21.15.27:0/2810638439 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c034070 con 0x7fbe140cd9d0 2024-04-20T10:55:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.922+0000 7fbe1159a640 1 -- 172.21.15.27:0/2810638439 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0c030880 con 0x7fbe140cd9d0 2024-04-20T10:55:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.922+0000 7fbe1159a640 1 -- 172.21.15.27:0/2810638439 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c030c30 con 0x7fbe140cd9d0 2024-04-20T10:55:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.922+0000 7fbe18c06640 1 -- 172.21.15.27:0/2810638439 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cd9d0 msgr2=0x7fbe1406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.922+0000 7fbe18c06640 1 --2- 172.21.15.27:0/2810638439 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cd9d0 0x7fbe1406e130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0097d0 tx=0x7fbe0c02b9c0 comp rx=0 tx=0).stop 2024-04-20T10:55:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.923+0000 7fbe18c06640 1 -- 172.21.15.27:0/2810638439 shutdown_connections 2024-04-20T10:55:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.923+0000 7fbe18c06640 1 --2- 172.21.15.27:0/2810638439 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cd9d0 0x7fbe1406e130 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:44.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.923+0000 7fbe18c06640 1 -- 172.21.15.27:0/2810638439 >> 172.21.15.27:0/2810638439 conn(0x7fbe1406f760 msgr2=0x7fbe14070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:44.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.923+0000 7fbe18c06640 1 -- 172.21.15.27:0/2810638439 shutdown_connections 2024-04-20T10:55:44.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.923+0000 7fbe18c06640 1 -- 172.21.15.27:0/2810638439 wait complete. 2024-04-20T10:55:44.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.924+0000 7fbe18c06640 1 Processor -- start 2024-04-20T10:55:44.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.924+0000 7fbe18c06640 1 -- start start 2024-04-20T10:55:44.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe18c06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe140c97b0 0x7fbe14155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe18c06640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cc150 0x7fbe141f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe18c06640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe140cd9d0 0x7fbe14156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe18c06640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe140cb670 con 0x7fbe140cd9d0 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe18c06640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe140737b0 con 0x7fbe140c97b0 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe18c06640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbe140cb950 con 0x7fbe140cc150 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe12d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe140cd9d0 0x7fbe14156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe12d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe140cd9d0 0x7fbe14156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48392/0 (socket says 172.21.15.27:48392) 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe12d9d640 1 -- 172.21.15.27:0/3543438409 learned_addr learned my addr 172.21.15.27:0/3543438409 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe1259c640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe140c97b0 0x7fbe14155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe11d9b640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cc150 0x7fbe141f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe12d9d640 1 -- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cc150 msgr2=0x7fbe141f15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe12d9d640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cc150 0x7fbe141f15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe12d9d640 1 -- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe140c97b0 msgr2=0x7fbe14155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe12d9d640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe140c97b0 0x7fbe14155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe12d9d640 1 -- 172.21.15.27:0/3543438409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1415fa30 con 0x7fbe140cd9d0 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe1259c640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe140c97b0 0x7fbe14155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe12d9d640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe140cd9d0 0x7fbe14156a30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02ba00 tx=0x7fbe0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.925+0000 7fbe0affd640 1 -- 172.21.15.27:0/3543438409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c034040 con 0x7fbe140cd9d0 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.926+0000 7fbe0affd640 1 -- 172.21.15.27:0/3543438409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0c03ada0 con 0x7fbe140cd9d0 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.926+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1415eb40 con 0x7fbe140cd9d0 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.926+0000 7fbe0affd640 1 -- 172.21.15.27:0/3543438409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c006e10 con 0x7fbe140cd9d0 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.926+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe141f0df0 con 0x7fbe140cd9d0 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.927+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe1404fcf0 con 0x7fbe140cd9d0 2024-04-20T10:55:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.927+0000 7fbe0affd640 1 -- 172.21.15.27:0/3543438409 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe0c04b060 con 0x7fbe140cd9d0 2024-04-20T10:55:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.927+0000 7fbe0affd640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbdf406c8f0 0x7fbdf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.927+0000 7fbe0affd640 1 -- 172.21.15.27:0/3543438409 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe0c0adcd0 con 0x7fbe140cd9d0 2024-04-20T10:55:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.928+0000 7fbe1259c640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbdf406c8f0 0x7fbdf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.928+0000 7fbe1259c640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbdf406c8f0 0x7fbdf406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbe04001e10 tx=0x7fbe04009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:44.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:44.932+0000 7fbe0affd640 1 -- 172.21.15.27:0/3543438409 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbe0c03f070 con 0x7fbe140cd9d0 2024-04-20T10:55:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.155+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbe140ccc70 con 0x7fbe140cd9d0 2024-04-20T10:55:45.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.801+0000 7fbe0affd640 1 -- 172.21.15.27:0/3543438409 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fbe0c07c100 con 0x7fbe140cd9d0 2024-04-20T10:55:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbdf406c8f0 msgr2=0x7fbdf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbdf406c8f0 0x7fbdf406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbe04001e10 tx=0x7fbe04009040 comp rx=0 tx=0).stop 2024-04-20T10:55:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe140cd9d0 msgr2=0x7fbe14156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe140cd9d0 0x7fbe14156a30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02ba00 tx=0x7fbe0c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 shutdown_connections 2024-04-20T10:55:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbdf406c8f0 0x7fbdf406edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe140cd9d0 0x7fbe14156a30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe140cc150 0x7fbe141f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 --2- 172.21.15.27:0/3543438409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe140c97b0 0x7fbe14155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 >> 172.21.15.27:0/3543438409 conn(0x7fbe1406f760 msgr2=0x7fbe140c9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 shutdown_connections 2024-04-20T10:55:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.803+0000 7fbe18c06640 1 -- 172.21.15.27:0/3543438409 wait complete. 2024-04-20T10:55:45.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-20T10:55:45.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.925+0000 7f754643e640 1 Processor -- start 2024-04-20T10:55:45.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.925+0000 7f754643e640 1 -- start start 2024-04-20T10:55:45.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f754643e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f75400c62e0 0x7f75400c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:45.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f754643e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f754015ec40 0x7f75400cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:45.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f754643e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f754015f210 con 0x7f7540155a20 2024-04-20T10:55:45.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f754643e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7540155e00 con 0x7f754015ec40 2024-04-20T10:55:45.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f754643e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f754015ba30 con 0x7f75400c62e0 2024-04-20T10:55:45.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f75449db640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f754015ec40 0x7f75400cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:45.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f75449db640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f754015ec40 0x7f75400cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48400/0 (socket says 172.21.15.27:48400) 2024-04-20T10:55:45.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f75449db640 1 -- 172.21.15.27:0/4027337583 learned_addr learned my addr 172.21.15.27:0/4027337583 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:45.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f753f7fe640 1 --2- 172.21.15.27:0/4027337583 >> v2:172.21.15.137:3300/0 conn(0x7f75400c62e0 0x7f75400c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:45.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f75449db640 1 -- 172.21.15.27:0/4027337583 >> v2:172.21.15.137:3300/0 conn(0x7f75400c62e0 msgr2=0x7f75400c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:45.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f75449db640 1 --2- 172.21.15.27:0/4027337583 >> v2:172.21.15.137:3300/0 conn(0x7f75400c62e0 0x7f75400c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f75449db640 1 -- 172.21.15.27:0/4027337583 >> v1:172.21.15.137:6789/0 conn(0x7f7540155a20 legacy=0x7f754015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:45.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f75449db640 1 -- 172.21.15.27:0/4027337583 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754015f4f0 con 0x7f754015ec40 2024-04-20T10:55:45.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.926+0000 7f753f7fe640 1 --2- 172.21.15.27:0/4027337583 >> v2:172.21.15.137:3300/0 conn(0x7f75400c62e0 0x7f75400c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:45.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.927+0000 7f75449db640 1 --2- 172.21.15.27:0/4027337583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754015ec40 0x7f75400cbe00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f754008edf0 tx=0x7f753802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d067102cfb2ebf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:45.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.927+0000 7f753effd640 1 -- 172.21.15.27:0/4027337583 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753802cb20 con 0x7f754015ec40 2024-04-20T10:55:45.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.927+0000 7f753effd640 1 -- 172.21.15.27:0/4027337583 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f753802ccc0 con 0x7f754015ec40 2024-04-20T10:55:45.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.927+0000 7f753effd640 1 -- 172.21.15.27:0/4027337583 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538034610 con 0x7f754015ec40 2024-04-20T10:55:45.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.928+0000 7f754643e640 1 -- 172.21.15.27:0/4027337583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754015ec40 msgr2=0x7f75400cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:45.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.928+0000 7f754643e640 1 --2- 172.21.15.27:0/4027337583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754015ec40 0x7f75400cbe00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f754008edf0 tx=0x7f753802bf80 comp rx=0 tx=0).stop 2024-04-20T10:55:45.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.928+0000 7f754643e640 1 -- 172.21.15.27:0/4027337583 shutdown_connections 2024-04-20T10:55:45.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.928+0000 7f754643e640 1 --2- 172.21.15.27:0/4027337583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754015ec40 0x7f75400cbe00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.928+0000 7f754643e640 1 --2- 172.21.15.27:0/4027337583 >> v2:172.21.15.137:3300/0 conn(0x7f75400c62e0 0x7f75400c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.928+0000 7f754643e640 1 -- 172.21.15.27:0/4027337583 >> 172.21.15.27:0/4027337583 conn(0x7f75400678a0 msgr2=0x7f7540067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:45.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.928+0000 7f754643e640 1 -- 172.21.15.27:0/4027337583 shutdown_connections 2024-04-20T10:55:45.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.928+0000 7f754643e640 1 -- 172.21.15.27:0/4027337583 wait complete. 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.929+0000 7f754643e640 1 Processor -- start 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.929+0000 7f754643e640 1 -- start start 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f754643e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75400c62e0 0x7f75401e7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f754643e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7540155a20 0x7f75400ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f754643e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f754015ec40 0x7f75401e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f754643e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f754015c5d0 con 0x7f75400c62e0 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f754643e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f754015fab0 con 0x7f7540155a20 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f754643e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f75400c67c0 con 0x7f754015ec40 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f753ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75400c62e0 0x7f75401e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f753ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75400c62e0 0x7f75401e7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48412/0 (socket says 172.21.15.27:48412) 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f753ffff640 1 -- 172.21.15.27:0/3203491773 learned_addr learned my addr 172.21.15.27:0/3203491773 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f753f7fe640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7540155a20 0x7f75400ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f75449db640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f754015ec40 0x7f75401e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f753ffff640 1 -- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f754015ec40 msgr2=0x7f75401e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f753ffff640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f754015ec40 0x7f75401e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f753ffff640 1 -- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7540155a20 msgr2=0x7f75400ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f753ffff640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7540155a20 0x7f75400ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.930+0000 7f753ffff640 1 -- 172.21.15.27:0/3203491773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754015e3e0 con 0x7f75400c62e0 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.931+0000 7f753ffff640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75400c62e0 0x7f75401e7320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f753400c430 tx=0x7f753400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.931+0000 7f753cff9640 1 -- 172.21.15.27:0/3203491773 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534011790 con 0x7f75400c62e0 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.931+0000 7f753cff9640 1 -- 172.21.15.27:0/3203491773 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7534017070 con 0x7f75400c62e0 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.931+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754015e680 con 0x7f75400c62e0 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.931+0000 7f753cff9640 1 -- 172.21.15.27:0/3203491773 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534011bb0 con 0x7f75400c62e0 2024-04-20T10:55:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.931+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754015e080 con 0x7f75400c62e0 2024-04-20T10:55:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.932+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754004fcf0 con 0x7f75400c62e0 2024-04-20T10:55:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.932+0000 7f753cff9640 1 -- 172.21.15.27:0/3203491773 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f753401c460 con 0x7f75400c62e0 2024-04-20T10:55:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.933+0000 7f753cff9640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f752c06c8f0 0x7f752c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.933+0000 7f753cff9640 1 -- 172.21.15.27:0/3203491773 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f753408b8a0 con 0x7f75400c62e0 2024-04-20T10:55:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.933+0000 7f753f7fe640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f752c06c8f0 0x7f752c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.933+0000 7f753f7fe640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f752c06c8f0 0x7f752c06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f754015ac20 tx=0x7f7530008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:45.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:45.937+0000 7f753cff9640 1 -- 172.21.15.27:0/3203491773 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7534059680 con 0x7f75400c62e0 2024-04-20T10:55:46.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.162+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f75400b3850 con 0x7f75400c62e0 2024-04-20T10:55:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.807+0000 7f753cff9640 1 -- 172.21.15.27:0/3203491773 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7f753405d980 con 0x7f75400c62e0 2024-04-20T10:55:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f752c06c8f0 msgr2=0x7f752c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f752c06c8f0 0x7f752c06edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f754015ac20 tx=0x7f7530008040 comp rx=0 tx=0).stop 2024-04-20T10:55:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75400c62e0 msgr2=0x7f75401e7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75400c62e0 0x7f75401e7320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f753400c430 tx=0x7f753400c8f0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 shutdown_connections 2024-04-20T10:55:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f752c06c8f0 0x7f752c06edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f754015ec40 0x7f75401e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7540155a20 0x7f75400ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 --2- 172.21.15.27:0/3203491773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75400c62e0 0x7f75401e7320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.809+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 >> 172.21.15.27:0/3203491773 conn(0x7f75400678a0 msgr2=0x7f754015bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.810+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 shutdown_connections 2024-04-20T10:55:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.810+0000 7f754643e640 1 -- 172.21.15.27:0/3203491773 wait complete. 2024-04-20T10:55:46.823 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T10:55:46.823 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:55:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.928+0000 7f266584a640 1 Processor -- start 2024-04-20T10:55:46.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.928+0000 7f266584a640 1 -- start start 2024-04-20T10:55:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.928+0000 7f266584a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.928+0000 7f266584a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f26600d1260 0x7f26600d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f266584a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266015c4d0 con 0x7f26600cbf50 2024-04-20T10:55:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f266584a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f26600ce610 con 0x7f26600d1260 2024-04-20T10:55:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f266584a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f26600ce180 con 0x7f2660155800 2024-04-20T10:55:46.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265effd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265f7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f26600d1260 0x7f26600d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265f7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f26600d1260 0x7f26600d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47368/0 (socket says 172.21.15.27:47368) 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265effd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266015bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60560/0 (socket says 172.21.15.27:60560) 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265f7fe640 1 -- 172.21.15.27:0/328412548 learned_addr learned my addr 172.21.15.27:0/328412548 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265dffb640 1 -- 172.21.15.27:0/328412548 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239085828 0 0) 0x7f2654001110 con 0x7f26600cbf50 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265dffb640 1 -- 172.21.15.27:0/328412548 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2648003520 con 0x7f26600cbf50 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265effd640 1 -- 172.21.15.27:0/328412548 >> v2:172.21.15.137:3300/0 conn(0x7f26600d1260 msgr2=0x7f26600d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265effd640 1 --2- 172.21.15.27:0/328412548 >> v2:172.21.15.137:3300/0 conn(0x7f26600d1260 0x7f26600d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265effd640 1 -- 172.21.15.27:0/328412548 >> v1:172.21.15.137:6789/0 conn(0x7f26600cbf50 legacy=0x7f26600c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265effd640 1 -- 172.21.15.27:0/328412548 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26600cd290 con 0x7f2660155800 2024-04-20T10:55:46.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.929+0000 7f265f7fe640 1 --2- 172.21.15.27:0/328412548 >> v2:172.21.15.137:3300/0 conn(0x7f26600d1260 0x7f26600d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.930+0000 7f265effd640 1 --2- 172.21.15.27:0/328412548 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266015bf90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f26500097a0 tx=0x7f265002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0d50851c0d97258 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.930+0000 7f265dffb640 1 -- 172.21.15.27:0/328412548 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26500315a0 con 0x7f2660155800 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.930+0000 7f265dffb640 1 -- 172.21.15.27:0/328412548 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2650037070 con 0x7f2660155800 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.930+0000 7f265dffb640 1 -- 172.21.15.27:0/328412548 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650031960 con 0x7f2660155800 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.931+0000 7f266584a640 1 -- 172.21.15.27:0/328412548 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 msgr2=0x7f266015bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.931+0000 7f266584a640 1 --2- 172.21.15.27:0/328412548 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266015bf90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f26500097a0 tx=0x7f265002b840 comp rx=0 tx=0).stop 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.931+0000 7f266584a640 1 -- 172.21.15.27:0/328412548 shutdown_connections 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.931+0000 7f266584a640 1 --2- 172.21.15.27:0/328412548 >> v2:172.21.15.137:3300/0 conn(0x7f26600d1260 0x7f26600d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.931+0000 7f266584a640 1 --2- 172.21.15.27:0/328412548 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266015bf90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.931+0000 7f266584a640 1 -- 172.21.15.27:0/328412548 >> 172.21.15.27:0/328412548 conn(0x7f2660067ae0 msgr2=0x7f2660067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.931+0000 7f266584a640 1 -- 172.21.15.27:0/328412548 shutdown_connections 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.931+0000 7f266584a640 1 -- 172.21.15.27:0/328412548 wait complete. 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.932+0000 7f266584a640 1 Processor -- start 2024-04-20T10:55:46.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.932+0000 7f266584a640 1 -- start start 2024-04-20T10:55:46.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f266584a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26600cbf50 0x7f26601541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f266584a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26600d1260 0x7f266014c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f266584a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266014ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f266584a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f26600cdb30 con 0x7f26600d1260 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f266584a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26600cd850 con 0x7f26600cbf50 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f266584a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f26600cde10 con 0x7f2660155800 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26600d1260 0x7f266014c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26600d1260 0x7f266014c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48420/0 (socket says 172.21.15.27:48420) 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265e7fc640 1 -- 172.21.15.27:0/2886183482 learned_addr learned my addr 172.21.15.27:0/2886183482 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265effd640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26600cbf50 0x7f26601541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265f7fe640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266014ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265e7fc640 1 -- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 msgr2=0x7f266014ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265e7fc640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266014ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265e7fc640 1 -- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26600cbf50 msgr2=0x7f26601541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:46.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265e7fc640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26600cbf50 0x7f26601541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265e7fc640 1 -- 172.21.15.27:0/2886183482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266014ec30 con 0x7f26600d1260 2024-04-20T10:55:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.933+0000 7f265e7fc640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26600d1260 0x7f266014c790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f265400c700 tx=0x7f265400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.934+0000 7f2647fff640 1 -- 172.21.15.27:0/2886183482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26540052d0 con 0x7f26600d1260 2024-04-20T10:55:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.934+0000 7f2647fff640 1 -- 172.21.15.27:0/2886183482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2654005470 con 0x7f26600d1260 2024-04-20T10:55:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.934+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266014f770 con 0x7f26600d1260 2024-04-20T10:55:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.934+0000 7f2647fff640 1 -- 172.21.15.27:0/2886183482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265400a860 con 0x7f26600d1260 2024-04-20T10:55:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.934+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266014f1b0 con 0x7f26600d1260 2024-04-20T10:55:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.935+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26600688e0 con 0x7f26600d1260 2024-04-20T10:55:46.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.935+0000 7f2647fff640 1 -- 172.21.15.27:0/2886183482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f265400aa00 con 0x7f26600d1260 2024-04-20T10:55:46.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.936+0000 7f2647fff640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f264c06c920 0x7f264c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:46.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.936+0000 7f2647fff640 1 -- 172.21.15.27:0/2886183482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f265408b780 con 0x7f26600d1260 2024-04-20T10:55:46.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.936+0000 7f265effd640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f264c06c920 0x7f264c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:46.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.937+0000 7f265effd640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f264c06c920 0x7f264c06ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2650036000 tx=0x7f265003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:46.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:46.940+0000 7f2647fff640 1 -- 172.21.15.27:0/2886183482 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f266015c4d0 con 0x7f26600d1260 2024-04-20T10:55:47.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.168+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26600d08f0 con 0x7f26600d1260 2024-04-20T10:55:47.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.169+0000 7f2647fff640 1 -- 172.21.15.27:0/2886183482 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6843 (secure 0 0 0) 0x7f26600d08f0 con 0x7f26600d1260 2024-04-20T10:55:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:47.170 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:55:46.164496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-20T10:55:47.170 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f264c06c920 msgr2=0x7f264c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f264c06c920 0x7f264c06ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2650036000 tx=0x7f265003d040 comp rx=0 tx=0).stop 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26600d1260 msgr2=0x7f266014c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26600d1260 0x7f266014c790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f265400c700 tx=0x7f265400cbc0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 shutdown_connections 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f264c06c920 0x7f264c06ede0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2660155800 0x7f266014ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26600d1260 0x7f266014c790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 --2- 172.21.15.27:0/2886183482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26600cbf50 0x7f26601541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 >> 172.21.15.27:0/2886183482 conn(0x7f2660067ae0 msgr2=0x7f26600c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 shutdown_connections 2024-04-20T10:55:47.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.171+0000 7f266584a640 1 -- 172.21.15.27:0/2886183482 wait complete. 2024-04-20T10:55:47.183 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:47.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.282+0000 7fd0f9d5f640 1 Processor -- start 2024-04-20T10:55:47.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.282+0000 7fd0f9d5f640 1 -- start start 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f9d5f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd0f4155910 0x7fd0f40c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f9d5f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd0f415eaf0 0x7fd0f406d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f9d5f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f40c2910 0x7fd0f40c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f9d5f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f4155cf0 con 0x7fd0f4155910 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f9d5f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f40c7a70 con 0x7fd0f415eaf0 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f9d5f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f415f9b0 con 0x7fd0f40c2910 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd0f4155910 0x7fd0f40c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd0f4155910 0x7fd0f40c1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48434/0 (socket says 172.21.15.27:48434) 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f37fe640 1 -- 172.21.15.27:0/1311714847 learned_addr learned my addr 172.21.15.27:0/1311714847 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f2ffd640 1 --2- 172.21.15.27:0/1311714847 >> v2:172.21.15.137:3300/0 conn(0x7fd0f415eaf0 0x7fd0f406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f3fff640 1 --2- 172.21.15.27:0/1311714847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f40c2910 0x7fd0f40c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f37fe640 1 -- 172.21.15.27:0/1311714847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f40c2910 msgr2=0x7fd0f40c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f37fe640 1 --2- 172.21.15.27:0/1311714847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f40c2910 0x7fd0f40c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f37fe640 1 -- 172.21.15.27:0/1311714847 >> v2:172.21.15.137:3300/0 conn(0x7fd0f415eaf0 msgr2=0x7fd0f406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f37fe640 1 --2- 172.21.15.27:0/1311714847 >> v2:172.21.15.137:3300/0 conn(0x7fd0f415eaf0 0x7fd0f406d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f37fe640 1 -- 172.21.15.27:0/1311714847 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f415f110 con 0x7fd0f4155910 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.283+0000 7fd0f3fff640 1 --2- 172.21.15.27:0/1311714847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f40c2910 0x7fd0f40c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.284+0000 7fd0f37fe640 1 --2- 172.21.15.27:0/1311714847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 0x7fd0f40c1f00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd0f406f950 tx=0x7fd0e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc07674aa4031156 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.284+0000 7fd0f27fc640 1 -- 172.21.15.27:0/1311714847 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4035020 con 0x7fd0f4155910 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.284+0000 7fd0f27fc640 1 -- 172.21.15.27:0/1311714847 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e4031640 con 0x7fd0f4155910 2024-04-20T10:55:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.284+0000 7fd0f27fc640 1 -- 172.21.15.27:0/1311714847 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e402d040 con 0x7fd0f4155910 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.285+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/1311714847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 msgr2=0x7fd0f40c1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.285+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/1311714847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 0x7fd0f40c1f00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd0f406f950 tx=0x7fd0e402b8f0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.285+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/1311714847 shutdown_connections 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.285+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/1311714847 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f40c2910 0x7fd0f40c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.285+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/1311714847 >> v2:172.21.15.137:3300/0 conn(0x7fd0f415eaf0 0x7fd0f406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.285+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/1311714847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 0x7fd0f40c1f00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.285+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/1311714847 >> 172.21.15.27:0/1311714847 conn(0x7fd0f40ccee0 msgr2=0x7fd0f40cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.285+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/1311714847 shutdown_connections 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.285+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/1311714847 wait complete. 2024-04-20T10:55:47.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.286+0000 7fd0f9d5f640 1 Processor -- start 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.286+0000 7fd0f9d5f640 1 -- start start 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f9d5f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0f40c2910 0x7fd0f4149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f9d5f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 0x7fd0f41f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f9d5f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f415eaf0 0x7fd0f41f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f9d5f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f406ca00 con 0x7fd0f4155910 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f9d5f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f415f6d0 con 0x7fd0f40c2910 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f9d5f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f406cce0 con 0x7fd0f415eaf0 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 0x7fd0f41f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 0x7fd0f41f2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48438/0 (socket says 172.21.15.27:48438) 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f2ffd640 1 -- 172.21.15.27:0/2280223441 learned_addr learned my addr 172.21.15.27:0/2280223441 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f37fe640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0f40c2910 0x7fd0f4149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f3fff640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f415eaf0 0x7fd0f41f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f2ffd640 1 -- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f415eaf0 msgr2=0x7fd0f41f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f2ffd640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f415eaf0 0x7fd0f41f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f2ffd640 1 -- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0f40c2910 msgr2=0x7fd0f4149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f2ffd640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0f40c2910 0x7fd0f4149730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f2ffd640 1 -- 172.21.15.27:0/2280223441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f40afc00 con 0x7fd0f4155910 2024-04-20T10:55:47.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.287+0000 7fd0f2ffd640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 0x7fd0f41f2110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800e870 tx=0x7fd0e800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:47.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.288+0000 7fd0dbfff640 1 -- 172.21.15.27:0/2280223441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8017070 con 0x7fd0f4155910 2024-04-20T10:55:47.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.288+0000 7fd0dbfff640 1 -- 172.21.15.27:0/2280223441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e800c940 con 0x7fd0f4155910 2024-04-20T10:55:47.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.288+0000 7fd0dbfff640 1 -- 172.21.15.27:0/2280223441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e800cd30 con 0x7fd0f4155910 2024-04-20T10:55:47.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.288+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f40afee0 con 0x7fd0f4155910 2024-04-20T10:55:47.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.288+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f40b04a0 con 0x7fd0f4155910 2024-04-20T10:55:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.289+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f404fcf0 con 0x7fd0f4155910 2024-04-20T10:55:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.289+0000 7fd0dbfff640 1 -- 172.21.15.27:0/2280223441 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0e801a070 con 0x7fd0f4155910 2024-04-20T10:55:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.290+0000 7fd0dbfff640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd0e006c8f0 0x7fd0e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.290+0000 7fd0dbfff640 1 -- 172.21.15.27:0/2280223441 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd0e808bc60 con 0x7fd0f4155910 2024-04-20T10:55:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.290+0000 7fd0f37fe640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd0e006c8f0 0x7fd0e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.290+0000 7fd0f37fe640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd0e006c8f0 0x7fd0e006edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4036000 tx=0x7fd0e403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:47.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.294+0000 7fd0dbfff640 1 -- 172.21.15.27:0/2280223441 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd0e8059ad0 con 0x7fd0f4155910 2024-04-20T10:55:47.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.479+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0f41f1a50 con 0x7fd0f4155910 2024-04-20T10:55:47.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.480+0000 7fd0dbfff640 1 -- 172.21.15.27:0/2280223441 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16569 (secure 0 0 0) 0x7fd0e805ddd0 con 0x7fd0f4155910 2024-04-20T10:55:47.481 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:47.481 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:43.787038+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd0e006c8f0 msgr2=0x7fd0e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd0e006c8f0 0x7fd0e006edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4036000 tx=0x7fd0e403c040 comp rx=0 tx=0).stop 2024-04-20T10:55:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 msgr2=0x7fd0f41f2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 0x7fd0f41f2110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800e870 tx=0x7fd0e800ed30 comp rx=0 tx=0).stop 2024-04-20T10:55:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 shutdown_connections 2024-04-20T10:55:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd0e006c8f0 0x7fd0e006edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd0f415eaf0 0x7fd0f41f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0f4155910 0x7fd0f41f2110 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 --2- 172.21.15.27:0/2280223441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0f40c2910 0x7fd0f4149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 >> 172.21.15.27:0/2280223441 conn(0x7fd0f40ccee0 msgr2=0x7fd0f40cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 shutdown_connections 2024-04-20T10:55:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.482+0000 7fd0f9d5f640 1 -- 172.21.15.27:0/2280223441 wait complete. 2024-04-20T10:55:47.494 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T10:55:47.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:55:47.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.594+0000 7fb68f231640 1 Processor -- start 2024-04-20T10:55:47.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.594+0000 7fb68f231640 1 -- start start 2024-04-20T10:55:47.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb68f231640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb6881537f0 0x7fb6880c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb68f231640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb688159260 0x7fb6880c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb68f231640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6880c7cd0 con 0x7fb688153410 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb68f231640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb68806cad0 con 0x7fb6881537f0 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb68f231640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fb6880c7fb0 con 0x7fb688159260 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb687fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb6881537f0 0x7fb6880c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb687fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb6881537f0 0x7fb6880c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48452/0 (socket says 172.21.15.27:48452) 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb687fff640 1 -- 172.21.15.27:0/339073574 learned_addr learned my addr 172.21.15.27:0/339073574 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb68d7ce640 1 --2- 172.21.15.27:0/339073574 >> v2:172.21.15.137:3300/0 conn(0x7fb688159260 0x7fb6880c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb687fff640 1 -- 172.21.15.27:0/339073574 >> v2:172.21.15.137:3300/0 conn(0x7fb688159260 msgr2=0x7fb6880c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb687fff640 1 --2- 172.21.15.27:0/339073574 >> v2:172.21.15.137:3300/0 conn(0x7fb688159260 0x7fb6880c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb687fff640 1 -- 172.21.15.27:0/339073574 >> v1:172.21.15.137:6789/0 conn(0x7fb688153410 legacy=0x7fb68815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.595+0000 7fb687fff640 1 -- 172.21.15.27:0/339073574 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6880c8290 con 0x7fb6881537f0 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.596+0000 7fb687fff640 1 --2- 172.21.15.27:0/339073574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6881537f0 0x7fb6880c6730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb68806e640 tx=0x7fb67402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea8044136c0dba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.596+0000 7fb6877fe640 1 -- 172.21.15.27:0/339073574 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67402f850 con 0x7fb6881537f0 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.596+0000 7fb6877fe640 1 -- 172.21.15.27:0/339073574 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67402f9f0 con 0x7fb6881537f0 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.596+0000 7fb6877fe640 1 -- 172.21.15.27:0/339073574 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67402fdc0 con 0x7fb6881537f0 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.596+0000 7fb68f231640 1 -- 172.21.15.27:0/339073574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6881537f0 msgr2=0x7fb6880c6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.596+0000 7fb68f231640 1 --2- 172.21.15.27:0/339073574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6881537f0 0x7fb6880c6730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb68806e640 tx=0x7fb67402ba30 comp rx=0 tx=0).stop 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.597+0000 7fb68f231640 1 -- 172.21.15.27:0/339073574 shutdown_connections 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.597+0000 7fb68f231640 1 --2- 172.21.15.27:0/339073574 >> v2:172.21.15.137:3300/0 conn(0x7fb688159260 0x7fb6880c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.597+0000 7fb68f231640 1 --2- 172.21.15.27:0/339073574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6881537f0 0x7fb6880c6730 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.597+0000 7fb68f231640 1 -- 172.21.15.27:0/339073574 >> 172.21.15.27:0/339073574 conn(0x7fb6880ccbd0 msgr2=0x7fb6880ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.597+0000 7fb68f231640 1 -- 172.21.15.27:0/339073574 shutdown_connections 2024-04-20T10:55:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.597+0000 7fb68f231640 1 -- 172.21.15.27:0/339073574 wait complete. 2024-04-20T10:55:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.598+0000 7fb68f231640 1 Processor -- start 2024-04-20T10:55:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.598+0000 7fb68f231640 1 -- start start 2024-04-20T10:55:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.598+0000 7fb68f231640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb688153410 0x7fb688071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.598+0000 7fb68f231640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6881537f0 0x7fb68815b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.598+0000 7fb68f231640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb688159260 0x7fb6881f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68f231640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb6880c75d0 con 0x7fb688153410 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68f231640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb68815cc50 con 0x7fb6881537f0 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68f231640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fb6880c7890 con 0x7fb688159260 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68cfcd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb688153410 0x7fb688071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68cfcd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb688153410 0x7fb688071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48460/0 (socket says 172.21.15.27:48460) 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68cfcd640 1 -- 172.21.15.27:0/2903537128 learned_addr learned my addr 172.21.15.27:0/2903537128 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb687fff640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6881537f0 0x7fb68815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68d7ce640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb688159260 0x7fb6881f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68cfcd640 1 -- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb688159260 msgr2=0x7fb6881f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68cfcd640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb688159260 0x7fb6881f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68cfcd640 1 -- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6881537f0 msgr2=0x7fb68815b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68cfcd640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6881537f0 0x7fb68815b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68cfcd640 1 -- 172.21.15.27:0/2903537128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb688071bf0 con 0x7fb688153410 2024-04-20T10:55:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68d7ce640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb688159260 0x7fb6881f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68cfcd640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb688153410 0x7fb688071360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00c3f0 tx=0x7fb67c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb6857fa640 1 -- 172.21.15.27:0/2903537128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c0052d0 con 0x7fb688153410 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb6857fa640 1 -- 172.21.15.27:0/2903537128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67c005490 con 0x7fb688153410 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb6857fa640 1 -- 172.21.15.27:0/2903537128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c007260 con 0x7fb688153410 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6881f5170 con 0x7fb688153410 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.599+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6881f5730 con 0x7fb688153410 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.600+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6880707b0 con 0x7fb688153410 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.601+0000 7fb6857fa640 1 -- 172.21.15.27:0/2903537128 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb67c007400 con 0x7fb688153410 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.601+0000 7fb6857fa640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb66c06c8f0 0x7fb66c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.601+0000 7fb6857fa640 1 -- 172.21.15.27:0/2903537128 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb67c08afd0 con 0x7fb688153410 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.601+0000 7fb687fff640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb66c06c8f0 0x7fb66c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.602+0000 7fb687fff640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb66c06c8f0 0x7fb66c06edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb674036000 tx=0x7fb674008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:47.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.606+0000 7fb6857fa640 1 -- 172.21.15.27:0/2903537128 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb6880707b0 con 0x7fb688153410 2024-04-20T10:55:47.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.791+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb688145fc0 con 0x7fb66c06c8f0 2024-04-20T10:55:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.798+0000 7fb6857fa640 1 -- 172.21.15.27:0/2903537128 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251848 (secure 0 0 0) 0x7fb688145fc0 con 0x7fb66c06c8f0 2024-04-20T10:55:47.803 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:47.806 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T10:55:47.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb66c06c8f0 msgr2=0x7fb66c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb66c06c8f0 0x7fb66c06edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb674036000 tx=0x7fb674008030 comp rx=0 tx=0).stop 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb688153410 msgr2=0x7fb688071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb688153410 0x7fb688071360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00c3f0 tx=0x7fb67c00c8b0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 shutdown_connections 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb66c06c8f0 0x7fb66c06edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb688159260 0x7fb6881f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6881537f0 0x7fb68815b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 --2- 172.21.15.27:0/2903537128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb688153410 0x7fb688071360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 >> 172.21.15.27:0/2903537128 conn(0x7fb6880ccbd0 msgr2=0x7fb688147d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 shutdown_connections 2024-04-20T10:55:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.801+0000 7fb68f231640 1 -- 172.21.15.27:0/2903537128 wait complete. 2024-04-20T10:55:47.814 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-20T10:55:47.703381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218876,"kb_used_data":2700,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755844,"statfs":{"total":773094113280,"available":772869984256,"internally_reserved":0,"allocated":2764800,"data_stored":1716708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001375"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287393+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T10:55:44.287393+0000","last_peered":"2024-04-20T10:55:44.287393+0000","last_clean":"2024-04-20T10:55:44.287393+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T10:55:44.287393+0000","last_undegraded":"2024-04-20T10:55:44.287393+0000","last_fullsized":"2024-04-20T10:55:44.287393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797629+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T10:55:43.797629+0000","last_peered":"2024-04-20T10:55:43.797629+0000","last_clean":"2024-04-20T10:55:43.797629+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T10:55:43.797629+0000","last_undegraded":"2024-04-20T10:55:43.797629+0000","last_fullsized":"2024-04-20T10:55:43.797629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285486+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T10:55:44.285486+0000","last_peered":"2024-04-20T10:55:44.285486+0000","last_clean":"2024-04-20T10:55:44.285486+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T10:55:44.285486+0000","last_undegraded":"2024-04-20T10:55:44.285486+0000","last_fullsized":"2024-04-20T10:55:44.285486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286383+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T10:55:44.286383+0000","last_peered":"2024-04-20T10:55:44.286383+0000","last_clean":"2024-04-20T10:55:44.286383+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T10:55:44.286383+0000","last_undegraded":"2024-04-20T10:55:44.286383+0000","last_fullsized":"2024-04-20T10:55:44.286383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286289+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T10:55:44.286289+0000","last_peered":"2024-04-20T10:55:44.286289+0000","last_clean":"2024-04-20T10:55:44.286289+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T10:55:44.286289+0000","last_undegraded":"2024-04-20T10:55:44.286289+0000","last_fullsized":"2024-04-20T10:55:44.286289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287329+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T10:55:44.287329+0000","last_peered":"2024-04-20T10:55:44.287329+0000","last_clean":"2024-04-20T10:55:44.287329+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T10:55:44.287329+0000","last_undegraded":"2024-04-20T10:55:44.287329+0000","last_fullsized":"2024-04-20T10:55:44.287329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287439+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T10:55:44.287439+0000","last_peered":"2024-04-20T10:55:44.287439+0000","last_clean":"2024-04-20T10:55:44.287439+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T10:55:44.287439+0000","last_undegraded":"2024-04-20T10:55:44.287439+0000","last_fullsized":"2024-04-20T10:55:44.287439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792859+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T10:55:43.792859+0000","last_peered":"2024-04-20T10:55:43.792859+0000","last_clean":"2024-04-20T10:55:43.792859+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T10:55:43.792859+0000","last_undegraded":"2024-04-20T10:55:43.792859+0000","last_fullsized":"2024-04-20T10:55:43.792859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286907+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T10:55:44.286907+0000","last_peered":"2024-04-20T10:55:44.286907+0000","last_clean":"2024-04-20T10:55:44.286907+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T10:55:44.286907+0000","last_undegraded":"2024-04-20T10:55:44.286907+0000","last_fullsized":"2024-04-20T10:55:44.286907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286541+0000","last_change":"2024-04-20T10:55:42.802716+0000","last_active":"2024-04-20T10:55:44.286541+0000","last_peered":"2024-04-20T10:55:44.286541+0000","last_clean":"2024-04-20T10:55:44.286541+0000","last_became_active":"2024-04-20T10:55:42.802418+0000","last_became_peered":"2024-04-20T10:55:42.802418+0000","last_unstale":"2024-04-20T10:55:44.286541+0000","last_undegraded":"2024-04-20T10:55:44.286541+0000","last_fullsized":"2024-04-20T10:55:44.286541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:12:22.186243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286228+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T10:55:44.286228+0000","last_peered":"2024-04-20T10:55:44.286228+0000","last_clean":"2024-04-20T10:55:44.286228+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T10:55:44.286228+0000","last_undegraded":"2024-04-20T10:55:44.286228+0000","last_fullsized":"2024-04-20T10:55:44.286228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801002+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T10:55:43.801002+0000","last_peered":"2024-04-20T10:55:43.801002+0000","last_clean":"2024-04-20T10:55:43.801002+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T10:55:43.801002+0000","last_undegraded":"2024-04-20T10:55:43.801002+0000","last_fullsized":"2024-04-20T10:55:43.801002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793050+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T10:55:43.793050+0000","last_peered":"2024-04-20T10:55:43.793050+0000","last_clean":"2024-04-20T10:55:43.793050+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T10:55:43.793050+0000","last_undegraded":"2024-04-20T10:55:43.793050+0000","last_fullsized":"2024-04-20T10:55:43.793050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797332+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T10:55:43.797332+0000","last_peered":"2024-04-20T10:55:43.797332+0000","last_clean":"2024-04-20T10:55:43.797332+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T10:55:43.797332+0000","last_undegraded":"2024-04-20T10:55:43.797332+0000","last_fullsized":"2024-04-20T10:55:43.797332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285851+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T10:55:44.285851+0000","last_peered":"2024-04-20T10:55:44.285851+0000","last_clean":"2024-04-20T10:55:44.285851+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T10:55:44.285851+0000","last_undegraded":"2024-04-20T10:55:44.285851+0000","last_fullsized":"2024-04-20T10:55:44.285851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286491+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T10:55:44.286491+0000","last_peered":"2024-04-20T10:55:44.286491+0000","last_clean":"2024-04-20T10:55:44.286491+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T10:55:44.286491+0000","last_undegraded":"2024-04-20T10:55:44.286491+0000","last_fullsized":"2024-04-20T10:55:44.286491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285217+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T10:55:44.285217+0000","last_peered":"2024-04-20T10:55:44.285217+0000","last_clean":"2024-04-20T10:55:44.285217+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T10:55:44.285217+0000","last_undegraded":"2024-04-20T10:55:44.285217+0000","last_fullsized":"2024-04-20T10:55:44.285217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287175+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T10:55:44.287175+0000","last_peered":"2024-04-20T10:55:44.287175+0000","last_clean":"2024-04-20T10:55:44.287175+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T10:55:44.287175+0000","last_undegraded":"2024-04-20T10:55:44.287175+0000","last_fullsized":"2024-04-20T10:55:44.287175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286994+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T10:55:44.286994+0000","last_peered":"2024-04-20T10:55:44.286994+0000","last_clean":"2024-04-20T10:55:44.286994+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T10:55:44.286994+0000","last_undegraded":"2024-04-20T10:55:44.286994+0000","last_fullsized":"2024-04-20T10:55:44.286994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285751+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T10:55:44.285751+0000","last_peered":"2024-04-20T10:55:44.285751+0000","last_clean":"2024-04-20T10:55:44.285751+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T10:55:44.285751+0000","last_undegraded":"2024-04-20T10:55:44.285751+0000","last_fullsized":"2024-04-20T10:55:44.285751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287484+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T10:55:44.287484+0000","last_peered":"2024-04-20T10:55:44.287484+0000","last_clean":"2024-04-20T10:55:44.287484+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T10:55:44.287484+0000","last_undegraded":"2024-04-20T10:55:44.287484+0000","last_fullsized":"2024-04-20T10:55:44.287484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792921+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T10:55:43.792921+0000","last_peered":"2024-04-20T10:55:43.792921+0000","last_clean":"2024-04-20T10:55:43.792921+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T10:55:43.792921+0000","last_undegraded":"2024-04-20T10:55:43.792921+0000","last_fullsized":"2024-04-20T10:55:43.792921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793499+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T10:55:43.793499+0000","last_peered":"2024-04-20T10:55:43.793499+0000","last_clean":"2024-04-20T10:55:43.793499+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T10:55:43.793499+0000","last_undegraded":"2024-04-20T10:55:43.793499+0000","last_fullsized":"2024-04-20T10:55:43.793499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286683+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T10:55:44.286683+0000","last_peered":"2024-04-20T10:55:44.286683+0000","last_clean":"2024-04-20T10:55:44.286683+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T10:55:44.286683+0000","last_undegraded":"2024-04-20T10:55:44.286683+0000","last_fullsized":"2024-04-20T10:55:44.286683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797439+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T10:55:43.797439+0000","last_peered":"2024-04-20T10:55:43.797439+0000","last_clean":"2024-04-20T10:55:43.797439+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T10:55:43.797439+0000","last_undegraded":"2024-04-20T10:55:43.797439+0000","last_fullsized":"2024-04-20T10:55:43.797439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793232+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T10:55:43.793232+0000","last_peered":"2024-04-20T10:55:43.793232+0000","last_clean":"2024-04-20T10:55:43.793232+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T10:55:43.793232+0000","last_undegraded":"2024-04-20T10:55:43.793232+0000","last_fullsized":"2024-04-20T10:55:43.793232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285757+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T10:55:44.285757+0000","last_peered":"2024-04-20T10:55:44.285757+0000","last_clean":"2024-04-20T10:55:44.285757+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T10:55:44.285757+0000","last_undegraded":"2024-04-20T10:55:44.285757+0000","last_fullsized":"2024-04-20T10:55:44.285757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287493+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T10:55:44.287493+0000","last_peered":"2024-04-20T10:55:44.287493+0000","last_clean":"2024-04-20T10:55:44.287493+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T10:55:44.287493+0000","last_undegraded":"2024-04-20T10:55:44.287493+0000","last_fullsized":"2024-04-20T10:55:44.287493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285309+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T10:55:44.285309+0000","last_peered":"2024-04-20T10:55:44.285309+0000","last_clean":"2024-04-20T10:55:44.285309+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T10:55:44.285309+0000","last_undegraded":"2024-04-20T10:55:44.285309+0000","last_fullsized":"2024-04-20T10:55:44.285309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285746+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T10:55:44.285746+0000","last_peered":"2024-04-20T10:55:44.285746+0000","last_clean":"2024-04-20T10:55:44.285746+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T10:55:44.285746+0000","last_undegraded":"2024-04-20T10:55:44.285746+0000","last_fullsized":"2024-04-20T10:55:44.285746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800661+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T10:55:43.800661+0000","last_peered":"2024-04-20T10:55:43.800661+0000","last_clean":"2024-04-20T10:55:43.800661+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T10:55:43.800661+0000","last_undegraded":"2024-04-20T10:55:43.800661+0000","last_fullsized":"2024-04-20T10:55:43.800661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793161+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T10:55:43.793161+0000","last_peered":"2024-04-20T10:55:43.793161+0000","last_clean":"2024-04-20T10:55:43.793161+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T10:55:43.793161+0000","last_undegraded":"2024-04-20T10:55:43.793161+0000","last_fullsized":"2024-04-20T10:55:43.793161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285840+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T10:55:44.285840+0000","last_peered":"2024-04-20T10:55:44.285840+0000","last_clean":"2024-04-20T10:55:44.285840+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T10:55:44.285840+0000","last_undegraded":"2024-04-20T10:55:44.285840+0000","last_fullsized":"2024-04-20T10:55:44.285840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286466+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T10:55:44.286466+0000","last_peered":"2024-04-20T10:55:44.286466+0000","last_clean":"2024-04-20T10:55:44.286466+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T10:55:44.286466+0000","last_undegraded":"2024-04-20T10:55:44.286466+0000","last_fullsized":"2024-04-20T10:55:44.286466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793178+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T10:55:43.793178+0000","last_peered":"2024-04-20T10:55:43.793178+0000","last_clean":"2024-04-20T10:55:43.793178+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T10:55:43.793178+0000","last_undegraded":"2024-04-20T10:55:43.793178+0000","last_fullsized":"2024-04-20T10:55:43.793178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792877+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T10:55:43.792877+0000","last_peered":"2024-04-20T10:55:43.792877+0000","last_clean":"2024-04-20T10:55:43.792877+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T10:55:43.792877+0000","last_undegraded":"2024-04-20T10:55:43.792877+0000","last_fullsized":"2024-04-20T10:55:43.792877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286112+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T10:55:44.286112+0000","last_peered":"2024-04-20T10:55:44.286112+0000","last_clean":"2024-04-20T10:55:44.286112+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T10:55:44.286112+0000","last_undegraded":"2024-04-20T10:55:44.286112+0000","last_fullsized":"2024-04-20T10:55:44.286112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797544+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T10:55:43.797544+0000","last_peered":"2024-04-20T10:55:43.797544+0000","last_clean":"2024-04-20T10:55:43.797544+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T10:55:43.797544+0000","last_undegraded":"2024-04-20T10:55:43.797544+0000","last_fullsized":"2024-04-20T10:55:43.797544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286640+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T10:55:44.286640+0000","last_peered":"2024-04-20T10:55:44.286640+0000","last_clean":"2024-04-20T10:55:44.286640+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T10:55:44.286640+0000","last_undegraded":"2024-04-20T10:55:44.286640+0000","last_fullsized":"2024-04-20T10:55:44.286640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287062+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T10:55:44.287062+0000","last_peered":"2024-04-20T10:55:44.287062+0000","last_clean":"2024-04-20T10:55:44.287062+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T10:55:44.287062+0000","last_undegraded":"2024-04-20T10:55:44.287062+0000","last_fullsized":"2024-04-20T10:55:44.287062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287172+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T10:55:44.287172+0000","last_peered":"2024-04-20T10:55:44.287172+0000","last_clean":"2024-04-20T10:55:44.287172+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T10:55:44.287172+0000","last_undegraded":"2024-04-20T10:55:44.287172+0000","last_fullsized":"2024-04-20T10:55:44.287172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285669+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T10:55:44.285669+0000","last_peered":"2024-04-20T10:55:44.285669+0000","last_clean":"2024-04-20T10:55:44.285669+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T10:55:44.285669+0000","last_undegraded":"2024-04-20T10:55:44.285669+0000","last_fullsized":"2024-04-20T10:55:44.285669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797533+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T10:55:43.797533+0000","last_peered":"2024-04-20T10:55:43.797533+0000","last_clean":"2024-04-20T10:55:43.797533+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T10:55:43.797533+0000","last_undegraded":"2024-04-20T10:55:43.797533+0000","last_fullsized":"2024-04-20T10:55:43.797533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801052+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T10:55:43.801052+0000","last_peered":"2024-04-20T10:55:43.801052+0000","last_clean":"2024-04-20T10:55:43.801052+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T10:55:43.801052+0000","last_undegraded":"2024-04-20T10:55:43.801052+0000","last_fullsized":"2024-04-20T10:55:43.801052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287101+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T10:55:44.287101+0000","last_peered":"2024-04-20T10:55:44.287101+0000","last_clean":"2024-04-20T10:55:44.287101+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T10:55:44.287101+0000","last_undegraded":"2024-04-20T10:55:44.287101+0000","last_fullsized":"2024-04-20T10:55:44.287101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287336+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T10:55:44.287336+0000","last_peered":"2024-04-20T10:55:44.287336+0000","last_clean":"2024-04-20T10:55:44.287336+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T10:55:44.287336+0000","last_undegraded":"2024-04-20T10:55:44.287336+0000","last_fullsized":"2024-04-20T10:55:44.287336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285817+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T10:55:44.285817+0000","last_peered":"2024-04-20T10:55:44.285817+0000","last_clean":"2024-04-20T10:55:44.285817+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T10:55:44.285817+0000","last_undegraded":"2024-04-20T10:55:44.285817+0000","last_fullsized":"2024-04-20T10:55:44.285817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285622+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T10:55:44.285622+0000","last_peered":"2024-04-20T10:55:44.285622+0000","last_clean":"2024-04-20T10:55:44.285622+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T10:55:44.285622+0000","last_undegraded":"2024-04-20T10:55:44.285622+0000","last_fullsized":"2024-04-20T10:55:44.285622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287576+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T10:55:44.287576+0000","last_peered":"2024-04-20T10:55:44.287576+0000","last_clean":"2024-04-20T10:55:44.287576+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T10:55:44.287576+0000","last_undegraded":"2024-04-20T10:55:44.287576+0000","last_fullsized":"2024-04-20T10:55:44.287576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286631+0000","last_change":"2024-04-20T10:55:42.805169+0000","last_active":"2024-04-20T10:55:44.286631+0000","last_peered":"2024-04-20T10:55:44.286631+0000","last_clean":"2024-04-20T10:55:44.286631+0000","last_became_active":"2024-04-20T10:55:42.804573+0000","last_became_peered":"2024-04-20T10:55:42.804573+0000","last_unstale":"2024-04-20T10:55:44.286631+0000","last_undegraded":"2024-04-20T10:55:44.286631+0000","last_fullsized":"2024-04-20T10:55:44.286631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:16.103447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287223+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T10:55:44.287223+0000","last_peered":"2024-04-20T10:55:44.287223+0000","last_clean":"2024-04-20T10:55:44.287223+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T10:55:44.287223+0000","last_undegraded":"2024-04-20T10:55:44.287223+0000","last_fullsized":"2024-04-20T10:55:44.287223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285762+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T10:55:44.285762+0000","last_peered":"2024-04-20T10:55:44.285762+0000","last_clean":"2024-04-20T10:55:44.285762+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T10:55:44.285762+0000","last_undegraded":"2024-04-20T10:55:44.285762+0000","last_fullsized":"2024-04-20T10:55:44.285762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285071+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T10:55:44.285071+0000","last_peered":"2024-04-20T10:55:44.285071+0000","last_clean":"2024-04-20T10:55:44.285071+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T10:55:44.285071+0000","last_undegraded":"2024-04-20T10:55:44.285071+0000","last_fullsized":"2024-04-20T10:55:44.285071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287518+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T10:55:44.287518+0000","last_peered":"2024-04-20T10:55:44.287518+0000","last_clean":"2024-04-20T10:55:44.287518+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T10:55:44.287518+0000","last_undegraded":"2024-04-20T10:55:44.287518+0000","last_fullsized":"2024-04-20T10:55:44.287518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800769+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T10:55:43.800769+0000","last_peered":"2024-04-20T10:55:43.800769+0000","last_clean":"2024-04-20T10:55:43.800769+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T10:55:43.800769+0000","last_undegraded":"2024-04-20T10:55:43.800769+0000","last_fullsized":"2024-04-20T10:55:43.800769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286460+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T10:55:44.286460+0000","last_peered":"2024-04-20T10:55:44.286460+0000","last_clean":"2024-04-20T10:55:44.286460+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T10:55:44.286460+0000","last_undegraded":"2024-04-20T10:55:44.286460+0000","last_fullsized":"2024-04-20T10:55:44.286460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285191+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T10:55:44.285191+0000","last_peered":"2024-04-20T10:55:44.285191+0000","last_clean":"2024-04-20T10:55:44.285191+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T10:55:44.285191+0000","last_undegraded":"2024-04-20T10:55:44.285191+0000","last_fullsized":"2024-04-20T10:55:44.285191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287067+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T10:55:44.287067+0000","last_peered":"2024-04-20T10:55:44.287067+0000","last_clean":"2024-04-20T10:55:44.287067+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T10:55:44.287067+0000","last_undegraded":"2024-04-20T10:55:44.287067+0000","last_fullsized":"2024-04-20T10:55:44.287067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797568+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T10:55:43.797568+0000","last_peered":"2024-04-20T10:55:43.797568+0000","last_clean":"2024-04-20T10:55:43.797568+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T10:55:43.797568+0000","last_undegraded":"2024-04-20T10:55:43.797568+0000","last_fullsized":"2024-04-20T10:55:43.797568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286721+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T10:55:44.286721+0000","last_peered":"2024-04-20T10:55:44.286721+0000","last_clean":"2024-04-20T10:55:44.286721+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T10:55:44.286721+0000","last_undegraded":"2024-04-20T10:55:44.286721+0000","last_fullsized":"2024-04-20T10:55:44.286721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285213+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T10:55:44.285213+0000","last_peered":"2024-04-20T10:55:44.285213+0000","last_clean":"2024-04-20T10:55:44.285213+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T10:55:44.285213+0000","last_undegraded":"2024-04-20T10:55:44.285213+0000","last_fullsized":"2024-04-20T10:55:44.285213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286934+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T10:55:44.286934+0000","last_peered":"2024-04-20T10:55:44.286934+0000","last_clean":"2024-04-20T10:55:44.286934+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T10:55:44.286934+0000","last_undegraded":"2024-04-20T10:55:44.286934+0000","last_fullsized":"2024-04-20T10:55:44.286934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286525+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T10:55:44.286525+0000","last_peered":"2024-04-20T10:55:44.286525+0000","last_clean":"2024-04-20T10:55:44.286525+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T10:55:44.286525+0000","last_undegraded":"2024-04-20T10:55:44.286525+0000","last_fullsized":"2024-04-20T10:55:44.286525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793252+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T10:55:43.793252+0000","last_peered":"2024-04-20T10:55:43.793252+0000","last_clean":"2024-04-20T10:55:43.793252+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T10:55:43.793252+0000","last_undegraded":"2024-04-20T10:55:43.793252+0000","last_fullsized":"2024-04-20T10:55:43.793252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287230+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T10:55:44.287230+0000","last_peered":"2024-04-20T10:55:44.287230+0000","last_clean":"2024-04-20T10:55:44.287230+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T10:55:44.287230+0000","last_undegraded":"2024-04-20T10:55:44.287230+0000","last_fullsized":"2024-04-20T10:55:44.287230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285651+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T10:55:44.285651+0000","last_peered":"2024-04-20T10:55:44.285651+0000","last_clean":"2024-04-20T10:55:44.285651+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T10:55:44.285651+0000","last_undegraded":"2024-04-20T10:55:44.285651+0000","last_fullsized":"2024-04-20T10:55:44.285651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285287+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T10:55:44.285287+0000","last_peered":"2024-04-20T10:55:44.285287+0000","last_clean":"2024-04-20T10:55:44.285287+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T10:55:44.285287+0000","last_undegraded":"2024-04-20T10:55:44.285287+0000","last_fullsized":"2024-04-20T10:55:44.285287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797319+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T10:55:43.797319+0000","last_peered":"2024-04-20T10:55:43.797319+0000","last_clean":"2024-04-20T10:55:43.797319+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T10:55:43.797319+0000","last_undegraded":"2024-04-20T10:55:43.797319+0000","last_fullsized":"2024-04-20T10:55:43.797319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287600+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T10:55:44.287600+0000","last_peered":"2024-04-20T10:55:44.287600+0000","last_clean":"2024-04-20T10:55:44.287600+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T10:55:44.287600+0000","last_undegraded":"2024-04-20T10:55:44.287600+0000","last_fullsized":"2024-04-20T10:55:44.287600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800914+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T10:55:43.800914+0000","last_peered":"2024-04-20T10:55:43.800914+0000","last_clean":"2024-04-20T10:55:43.800914+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T10:55:43.800914+0000","last_undegraded":"2024-04-20T10:55:43.800914+0000","last_fullsized":"2024-04-20T10:55:43.800914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797497+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T10:55:43.797497+0000","last_peered":"2024-04-20T10:55:43.797497+0000","last_clean":"2024-04-20T10:55:43.797497+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T10:55:43.797497+0000","last_undegraded":"2024-04-20T10:55:43.797497+0000","last_fullsized":"2024-04-20T10:55:43.797497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287087+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T10:55:44.287087+0000","last_peered":"2024-04-20T10:55:44.287087+0000","last_clean":"2024-04-20T10:55:44.287087+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T10:55:44.287087+0000","last_undegraded":"2024-04-20T10:55:44.287087+0000","last_fullsized":"2024-04-20T10:55:44.287087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287263+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T10:55:44.287263+0000","last_peered":"2024-04-20T10:55:44.287263+0000","last_clean":"2024-04-20T10:55:44.287263+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T10:55:44.287263+0000","last_undegraded":"2024-04-20T10:55:44.287263+0000","last_fullsized":"2024-04-20T10:55:44.287263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797411+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T10:55:43.797411+0000","last_peered":"2024-04-20T10:55:43.797411+0000","last_clean":"2024-04-20T10:55:43.797411+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T10:55:43.797411+0000","last_undegraded":"2024-04-20T10:55:43.797411+0000","last_fullsized":"2024-04-20T10:55:43.797411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797512+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T10:55:43.797512+0000","last_peered":"2024-04-20T10:55:43.797512+0000","last_clean":"2024-04-20T10:55:43.797512+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T10:55:43.797512+0000","last_undegraded":"2024-04-20T10:55:43.797512+0000","last_fullsized":"2024-04-20T10:55:43.797512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793562+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T10:55:43.793562+0000","last_peered":"2024-04-20T10:55:43.793562+0000","last_clean":"2024-04-20T10:55:43.793562+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T10:55:43.793562+0000","last_undegraded":"2024-04-20T10:55:43.793562+0000","last_fullsized":"2024-04-20T10:55:43.793562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286739+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T10:55:44.286739+0000","last_peered":"2024-04-20T10:55:44.286739+0000","last_clean":"2024-04-20T10:55:44.286739+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T10:55:44.286739+0000","last_undegraded":"2024-04-20T10:55:44.286739+0000","last_fullsized":"2024-04-20T10:55:44.286739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285397+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T10:55:44.285397+0000","last_peered":"2024-04-20T10:55:44.285397+0000","last_clean":"2024-04-20T10:55:44.285397+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T10:55:44.285397+0000","last_undegraded":"2024-04-20T10:55:44.285397+0000","last_fullsized":"2024-04-20T10:55:44.285397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285923+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T10:55:44.285923+0000","last_peered":"2024-04-20T10:55:44.285923+0000","last_clean":"2024-04-20T10:55:44.285923+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T10:55:44.285923+0000","last_undegraded":"2024-04-20T10:55:44.285923+0000","last_fullsized":"2024-04-20T10:55:44.285923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800994+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T10:55:43.800994+0000","last_peered":"2024-04-20T10:55:43.800994+0000","last_clean":"2024-04-20T10:55:43.800994+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T10:55:43.800994+0000","last_undegraded":"2024-04-20T10:55:43.800994+0000","last_fullsized":"2024-04-20T10:55:43.800994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797429+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T10:55:43.797429+0000","last_peered":"2024-04-20T10:55:43.797429+0000","last_clean":"2024-04-20T10:55:43.797429+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T10:55:43.797429+0000","last_undegraded":"2024-04-20T10:55:43.797429+0000","last_fullsized":"2024-04-20T10:55:43.797429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793024+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T10:55:43.793024+0000","last_peered":"2024-04-20T10:55:43.793024+0000","last_clean":"2024-04-20T10:55:43.793024+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T10:55:43.793024+0000","last_undegraded":"2024-04-20T10:55:43.793024+0000","last_fullsized":"2024-04-20T10:55:43.793024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797381+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T10:55:43.797381+0000","last_peered":"2024-04-20T10:55:43.797381+0000","last_clean":"2024-04-20T10:55:43.797381+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T10:55:43.797381+0000","last_undegraded":"2024-04-20T10:55:43.797381+0000","last_fullsized":"2024-04-20T10:55:43.797381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793105+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T10:55:43.793105+0000","last_peered":"2024-04-20T10:55:43.793105+0000","last_clean":"2024-04-20T10:55:43.793105+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T10:55:43.793105+0000","last_undegraded":"2024-04-20T10:55:43.793105+0000","last_fullsized":"2024-04-20T10:55:43.793105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286780+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T10:55:44.286780+0000","last_peered":"2024-04-20T10:55:44.286780+0000","last_clean":"2024-04-20T10:55:44.286780+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T10:55:44.286780+0000","last_undegraded":"2024-04-20T10:55:44.286780+0000","last_fullsized":"2024-04-20T10:55:44.286780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286369+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T10:55:44.286369+0000","last_peered":"2024-04-20T10:55:44.286369+0000","last_clean":"2024-04-20T10:55:44.286369+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T10:55:44.286369+0000","last_undegraded":"2024-04-20T10:55:44.286369+0000","last_fullsized":"2024-04-20T10:55:44.286369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286936+0000","last_change":"2024-04-20T10:55:42.792905+0000","last_active":"2024-04-20T10:55:44.286936+0000","last_peered":"2024-04-20T10:55:44.286936+0000","last_clean":"2024-04-20T10:55:44.286936+0000","last_became_active":"2024-04-20T10:55:42.792587+0000","last_became_peered":"2024-04-20T10:55:42.792587+0000","last_unstale":"2024-04-20T10:55:44.286936+0000","last_undegraded":"2024-04-20T10:55:44.286936+0000","last_fullsized":"2024-04-20T10:55:44.286936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:02.987448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286090+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T10:55:44.286090+0000","last_peered":"2024-04-20T10:55:44.286090+0000","last_clean":"2024-04-20T10:55:44.286090+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T10:55:44.286090+0000","last_undegraded":"2024-04-20T10:55:44.286090+0000","last_fullsized":"2024-04-20T10:55:44.286090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286560+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T10:55:44.286560+0000","last_peered":"2024-04-20T10:55:44.286560+0000","last_clean":"2024-04-20T10:55:44.286560+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T10:55:44.286560+0000","last_undegraded":"2024-04-20T10:55:44.286560+0000","last_fullsized":"2024-04-20T10:55:44.286560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287033+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T10:55:44.287033+0000","last_peered":"2024-04-20T10:55:44.287033+0000","last_clean":"2024-04-20T10:55:44.287033+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T10:55:44.287033+0000","last_undegraded":"2024-04-20T10:55:44.287033+0000","last_fullsized":"2024-04-20T10:55:44.287033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793368+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T10:55:43.793368+0000","last_peered":"2024-04-20T10:55:43.793368+0000","last_clean":"2024-04-20T10:55:43.793368+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T10:55:43.793368+0000","last_undegraded":"2024-04-20T10:55:43.793368+0000","last_fullsized":"2024-04-20T10:55:43.793368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793370+0000","last_change":"2024-04-20T10:55:42.802227+0000","last_active":"2024-04-20T10:55:43.793370+0000","last_peered":"2024-04-20T10:55:43.793370+0000","last_clean":"2024-04-20T10:55:43.793370+0000","last_became_active":"2024-04-20T10:55:42.801944+0000","last_became_peered":"2024-04-20T10:55:42.801944+0000","last_unstale":"2024-04-20T10:55:43.793370+0000","last_undegraded":"2024-04-20T10:55:43.793370+0000","last_fullsized":"2024-04-20T10:55:43.793370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:29.862643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285734+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T10:55:44.285734+0000","last_peered":"2024-04-20T10:55:44.285734+0000","last_clean":"2024-04-20T10:55:44.285734+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T10:55:44.285734+0000","last_undegraded":"2024-04-20T10:55:44.285734+0000","last_fullsized":"2024-04-20T10:55:44.285734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":655002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":786432,"data_stored":655002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:55:47.819 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:55:47.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.957+0000 7fc230255640 1 Processor -- start 2024-04-20T10:55:47.958 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.957+0000 7fc230255640 1 -- start start 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.958+0000 7fc230255640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc228155950 0x7fc2280cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.958+0000 7fc230255640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc2280b5c00 0x7fc2280d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.958+0000 7fc230255640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22815b9c0 con 0x7fc2280cc330 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.958+0000 7fc230255640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fc22815bca0 con 0x7fc2280b5c00 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.958+0000 7fc230255640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc22815bf80 con 0x7fc228155950 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.958+0000 7fc22d7f0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc2280cc330 0x7fc2280b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53404/0 (socket says 172.21.15.27:53404) 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.958+0000 7fc22d7f0640 1 -- 172.21.15.27:0/1376213956 learned_addr learned my addr 172.21.15.27:0/1376213956 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22e7f2640 1 --2- 172.21.15.27:0/1376213956 >> v2:172.21.15.137:3300/0 conn(0x7fc2280b5c00 0x7fc2280d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 683762743 0 0) 0x7fc22815b9c0 con 0x7fc2280cc330 2024-04-20T10:55:47.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc210003520 con 0x7fc2280cc330 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22dff1640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc228155950 0x7fc2280cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3246909149 0 0) 0x7fc210003520 con 0x7fc2280cc330 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc228155950 msgr2=0x7fc2280cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc228155950 0x7fc2280cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 >> v2:172.21.15.137:3300/0 conn(0x7fc2280b5c00 msgr2=0x7fc2280d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 --2- 172.21.15.27:0/1376213956 >> v2:172.21.15.137:3300/0 conn(0x7fc2280b5c00 0x7fc2280d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22815c260 con 0x7fc2280cc330 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7fc21c001670 con 0x7fc2280cc330 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22e7f2640 1 --2- 172.21.15.27:0/1376213956 >> v2:172.21.15.137:3300/0 conn(0x7fc2280b5c00 0x7fc2280d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 >> v1:172.21.15.27:6789/0 conn(0x7fc2280cc330 legacy=0x7fc2280b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100039b0 0x7fc210003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc228155950 0x7fc210004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc210004d40 0x7fc2100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815bca0 con 0x7fc210004d40 2024-04-20T10:55:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815bf80 con 0x7fc2100039b0 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc21000ba80 con 0x7fc228155950 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22e7f2640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc210004d40 0x7fc2100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22dff1640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100039b0 0x7fc210003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22d7f0640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc228155950 0x7fc210004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22e7f2640 1 -- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc228155950 msgr2=0x7fc210004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22e7f2640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc228155950 0x7fc210004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22e7f2640 1 -- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100039b0 msgr2=0x7fc210003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22e7f2640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2100039b0 0x7fc210003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22e7f2640 1 -- 172.21.15.27:0/1376213956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc210003520 con 0x7fc210004d40 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.959+0000 7fc22d7f0640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc228155950 0x7fc210004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.960+0000 7fc22e7f2640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc210004d40 0x7fc2100051c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc218009c90 tx=0x7fc210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7f5eb5b64993fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.960+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218039070 con 0x7fc210004d40 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.960+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21802c980 con 0x7fc210004d40 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.960+0000 7fc22cfef640 1 -- 172.21.15.27:0/1376213956 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218009860 con 0x7fc210004d40 2024-04-20T10:55:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.961+0000 7fc230255640 1 -- 172.21.15.27:0/1376213956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc210004d40 msgr2=0x7fc2100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.961+0000 7fc230255640 1 --2- 172.21.15.27:0/1376213956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc210004d40 0x7fc2100051c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc218009c90 tx=0x7fc210005830 comp rx=0 tx=0).stop 2024-04-20T10:55:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.961+0000 7fc22e7f2640 1 -- 172.21.15.27:0/1376213956 reap_dead start 2024-04-20T10:55:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.961+0000 7fc230255640 1 -- 172.21.15.27:0/1376213956 shutdown_connections 2024-04-20T10:55:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.961+0000 7fc230255640 1 -- 172.21.15.27:0/1376213956 >> 172.21.15.27:0/1376213956 conn(0x7fc2280676c0 msgr2=0x7fc228067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.961+0000 7fc230255640 1 -- 172.21.15.27:0/1376213956 shutdown_connections 2024-04-20T10:55:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.961+0000 7fc230255640 1 -- 172.21.15.27:0/1376213956 wait complete. 2024-04-20T10:55:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.962+0000 7fc230255640 1 Processor -- start 2024-04-20T10:55:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.962+0000 7fc230255640 1 -- start start 2024-04-20T10:55:47.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc230255640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc22815d1a0 0x7fc22815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc230255640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc228159540 0x7fc22815e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc230255640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc22815e840 0x7fc2281f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc230255640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815fd10 con 0x7fc22815e840 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc230255640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815fb90 con 0x7fc228159540 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc230255640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc228155d30 con 0x7fc22815d1a0 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22e7f2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc22815e840 0x7fc2281f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22e7f2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc22815e840 0x7fc2281f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48492/0 (socket says 172.21.15.27:48492) 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22e7f2640 1 -- 172.21.15.27:0/2312041162 learned_addr learned my addr 172.21.15.27:0/2312041162 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22dff1640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc22815d1a0 0x7fc22815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22d7f0640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc228159540 0x7fc22815e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22e7f2640 1 -- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc22815d1a0 msgr2=0x7fc22815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22e7f2640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc22815d1a0 0x7fc22815d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22e7f2640 1 -- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc228159540 msgr2=0x7fc22815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22e7f2640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc228159540 0x7fc22815e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:47.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22e7f2640 1 -- 172.21.15.27:0/2312041162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2281f50b0 con 0x7fc22815e840 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc22e7f2640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc22815e840 0x7fc2281f1750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc218009c90 tx=0x7fc218004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.963+0000 7fc2267fc640 1 -- 172.21.15.27:0/2312041162 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218039070 con 0x7fc22815e840 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.964+0000 7fc2267fc640 1 -- 172.21.15.27:0/2312041162 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc218043070 con 0x7fc22815e840 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.964+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2281f5370 con 0x7fc22815e840 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.964+0000 7fc2267fc640 1 -- 172.21.15.27:0/2312041162 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218009860 con 0x7fc22815e840 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.964+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2280c7900 con 0x7fc22815e840 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.965+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2280c7b40 con 0x7fc22815e840 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.965+0000 7fc2267fc640 1 -- 172.21.15.27:0/2312041162 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc218048460 con 0x7fc22815e840 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.965+0000 7fc2267fc640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc21406c920 0x7fc21406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.965+0000 7fc2267fc640 1 -- 172.21.15.27:0/2312041162 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc2180ad090 con 0x7fc22815e840 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.966+0000 7fc22dff1640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc21406c920 0x7fc21406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:47.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.966+0000 7fc22dff1640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc21406c920 0x7fc21406ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc220003aa0 tx=0x7fc220009760 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:47.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:47.970+0000 7fc2267fc640 1 -- 172.21.15.27:0/2312041162 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc2280c7b40 con 0x7fc22815e840 2024-04-20T10:55:48.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.150+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc22804fcf0 con 0x7fc21406c920 2024-04-20T10:55:48.157 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.156+0000 7fc2267fc640 1 -- 172.21.15.27:0/2312041162 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251848 (secure 0 0 0) 0x7fc22804fcf0 con 0x7fc21406c920 2024-04-20T10:55:48.159 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:48.161 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T10:55:48.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc21406c920 msgr2=0x7fc21406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc21406c920 0x7fc21406ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc220003aa0 tx=0x7fc220009760 comp rx=0 tx=0).stop 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc22815e840 msgr2=0x7fc2281f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc22815e840 0x7fc2281f1750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc218009c90 tx=0x7fc218004730 comp rx=0 tx=0).stop 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 shutdown_connections 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc21406c920 0x7fc21406ede0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc22815e840 0x7fc2281f1750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc228159540 0x7fc22815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 --2- 172.21.15.27:0/2312041162 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc22815d1a0 0x7fc22815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 >> 172.21.15.27:0/2312041162 conn(0x7fc2280676c0 msgr2=0x7fc2280d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 shutdown_connections 2024-04-20T10:55:48.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.159+0000 7fc230255640 1 -- 172.21.15.27:0/2312041162 wait complete. 2024-04-20T10:55:48.170 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-20T10:55:47.703381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218876,"kb_used_data":2700,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755844,"statfs":{"total":773094113280,"available":772869984256,"internally_reserved":0,"allocated":2764800,"data_stored":1716708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001375"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287393+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T10:55:44.287393+0000","last_peered":"2024-04-20T10:55:44.287393+0000","last_clean":"2024-04-20T10:55:44.287393+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T10:55:44.287393+0000","last_undegraded":"2024-04-20T10:55:44.287393+0000","last_fullsized":"2024-04-20T10:55:44.287393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797629+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T10:55:43.797629+0000","last_peered":"2024-04-20T10:55:43.797629+0000","last_clean":"2024-04-20T10:55:43.797629+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T10:55:43.797629+0000","last_undegraded":"2024-04-20T10:55:43.797629+0000","last_fullsized":"2024-04-20T10:55:43.797629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285486+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T10:55:44.285486+0000","last_peered":"2024-04-20T10:55:44.285486+0000","last_clean":"2024-04-20T10:55:44.285486+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T10:55:44.285486+0000","last_undegraded":"2024-04-20T10:55:44.285486+0000","last_fullsized":"2024-04-20T10:55:44.285486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286383+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T10:55:44.286383+0000","last_peered":"2024-04-20T10:55:44.286383+0000","last_clean":"2024-04-20T10:55:44.286383+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T10:55:44.286383+0000","last_undegraded":"2024-04-20T10:55:44.286383+0000","last_fullsized":"2024-04-20T10:55:44.286383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286289+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T10:55:44.286289+0000","last_peered":"2024-04-20T10:55:44.286289+0000","last_clean":"2024-04-20T10:55:44.286289+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T10:55:44.286289+0000","last_undegraded":"2024-04-20T10:55:44.286289+0000","last_fullsized":"2024-04-20T10:55:44.286289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287329+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T10:55:44.287329+0000","last_peered":"2024-04-20T10:55:44.287329+0000","last_clean":"2024-04-20T10:55:44.287329+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T10:55:44.287329+0000","last_undegraded":"2024-04-20T10:55:44.287329+0000","last_fullsized":"2024-04-20T10:55:44.287329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287439+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T10:55:44.287439+0000","last_peered":"2024-04-20T10:55:44.287439+0000","last_clean":"2024-04-20T10:55:44.287439+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T10:55:44.287439+0000","last_undegraded":"2024-04-20T10:55:44.287439+0000","last_fullsized":"2024-04-20T10:55:44.287439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792859+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T10:55:43.792859+0000","last_peered":"2024-04-20T10:55:43.792859+0000","last_clean":"2024-04-20T10:55:43.792859+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T10:55:43.792859+0000","last_undegraded":"2024-04-20T10:55:43.792859+0000","last_fullsized":"2024-04-20T10:55:43.792859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286907+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T10:55:44.286907+0000","last_peered":"2024-04-20T10:55:44.286907+0000","last_clean":"2024-04-20T10:55:44.286907+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T10:55:44.286907+0000","last_undegraded":"2024-04-20T10:55:44.286907+0000","last_fullsized":"2024-04-20T10:55:44.286907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286541+0000","last_change":"2024-04-20T10:55:42.802716+0000","last_active":"2024-04-20T10:55:44.286541+0000","last_peered":"2024-04-20T10:55:44.286541+0000","last_clean":"2024-04-20T10:55:44.286541+0000","last_became_active":"2024-04-20T10:55:42.802418+0000","last_became_peered":"2024-04-20T10:55:42.802418+0000","last_unstale":"2024-04-20T10:55:44.286541+0000","last_undegraded":"2024-04-20T10:55:44.286541+0000","last_fullsized":"2024-04-20T10:55:44.286541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:12:22.186243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286228+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T10:55:44.286228+0000","last_peered":"2024-04-20T10:55:44.286228+0000","last_clean":"2024-04-20T10:55:44.286228+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T10:55:44.286228+0000","last_undegraded":"2024-04-20T10:55:44.286228+0000","last_fullsized":"2024-04-20T10:55:44.286228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801002+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T10:55:43.801002+0000","last_peered":"2024-04-20T10:55:43.801002+0000","last_clean":"2024-04-20T10:55:43.801002+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T10:55:43.801002+0000","last_undegraded":"2024-04-20T10:55:43.801002+0000","last_fullsized":"2024-04-20T10:55:43.801002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793050+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T10:55:43.793050+0000","last_peered":"2024-04-20T10:55:43.793050+0000","last_clean":"2024-04-20T10:55:43.793050+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T10:55:43.793050+0000","last_undegraded":"2024-04-20T10:55:43.793050+0000","last_fullsized":"2024-04-20T10:55:43.793050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797332+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T10:55:43.797332+0000","last_peered":"2024-04-20T10:55:43.797332+0000","last_clean":"2024-04-20T10:55:43.797332+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T10:55:43.797332+0000","last_undegraded":"2024-04-20T10:55:43.797332+0000","last_fullsized":"2024-04-20T10:55:43.797332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285851+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T10:55:44.285851+0000","last_peered":"2024-04-20T10:55:44.285851+0000","last_clean":"2024-04-20T10:55:44.285851+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T10:55:44.285851+0000","last_undegraded":"2024-04-20T10:55:44.285851+0000","last_fullsized":"2024-04-20T10:55:44.285851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286491+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T10:55:44.286491+0000","last_peered":"2024-04-20T10:55:44.286491+0000","last_clean":"2024-04-20T10:55:44.286491+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T10:55:44.286491+0000","last_undegraded":"2024-04-20T10:55:44.286491+0000","last_fullsized":"2024-04-20T10:55:44.286491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285217+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T10:55:44.285217+0000","last_peered":"2024-04-20T10:55:44.285217+0000","last_clean":"2024-04-20T10:55:44.285217+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T10:55:44.285217+0000","last_undegraded":"2024-04-20T10:55:44.285217+0000","last_fullsized":"2024-04-20T10:55:44.285217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287175+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T10:55:44.287175+0000","last_peered":"2024-04-20T10:55:44.287175+0000","last_clean":"2024-04-20T10:55:44.287175+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T10:55:44.287175+0000","last_undegraded":"2024-04-20T10:55:44.287175+0000","last_fullsized":"2024-04-20T10:55:44.287175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286994+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T10:55:44.286994+0000","last_peered":"2024-04-20T10:55:44.286994+0000","last_clean":"2024-04-20T10:55:44.286994+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T10:55:44.286994+0000","last_undegraded":"2024-04-20T10:55:44.286994+0000","last_fullsized":"2024-04-20T10:55:44.286994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285751+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T10:55:44.285751+0000","last_peered":"2024-04-20T10:55:44.285751+0000","last_clean":"2024-04-20T10:55:44.285751+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T10:55:44.285751+0000","last_undegraded":"2024-04-20T10:55:44.285751+0000","last_fullsized":"2024-04-20T10:55:44.285751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287484+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T10:55:44.287484+0000","last_peered":"2024-04-20T10:55:44.287484+0000","last_clean":"2024-04-20T10:55:44.287484+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T10:55:44.287484+0000","last_undegraded":"2024-04-20T10:55:44.287484+0000","last_fullsized":"2024-04-20T10:55:44.287484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792921+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T10:55:43.792921+0000","last_peered":"2024-04-20T10:55:43.792921+0000","last_clean":"2024-04-20T10:55:43.792921+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T10:55:43.792921+0000","last_undegraded":"2024-04-20T10:55:43.792921+0000","last_fullsized":"2024-04-20T10:55:43.792921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793499+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T10:55:43.793499+0000","last_peered":"2024-04-20T10:55:43.793499+0000","last_clean":"2024-04-20T10:55:43.793499+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T10:55:43.793499+0000","last_undegraded":"2024-04-20T10:55:43.793499+0000","last_fullsized":"2024-04-20T10:55:43.793499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286683+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T10:55:44.286683+0000","last_peered":"2024-04-20T10:55:44.286683+0000","last_clean":"2024-04-20T10:55:44.286683+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T10:55:44.286683+0000","last_undegraded":"2024-04-20T10:55:44.286683+0000","last_fullsized":"2024-04-20T10:55:44.286683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797439+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T10:55:43.797439+0000","last_peered":"2024-04-20T10:55:43.797439+0000","last_clean":"2024-04-20T10:55:43.797439+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T10:55:43.797439+0000","last_undegraded":"2024-04-20T10:55:43.797439+0000","last_fullsized":"2024-04-20T10:55:43.797439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793232+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T10:55:43.793232+0000","last_peered":"2024-04-20T10:55:43.793232+0000","last_clean":"2024-04-20T10:55:43.793232+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T10:55:43.793232+0000","last_undegraded":"2024-04-20T10:55:43.793232+0000","last_fullsized":"2024-04-20T10:55:43.793232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285757+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T10:55:44.285757+0000","last_peered":"2024-04-20T10:55:44.285757+0000","last_clean":"2024-04-20T10:55:44.285757+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T10:55:44.285757+0000","last_undegraded":"2024-04-20T10:55:44.285757+0000","last_fullsized":"2024-04-20T10:55:44.285757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287493+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T10:55:44.287493+0000","last_peered":"2024-04-20T10:55:44.287493+0000","last_clean":"2024-04-20T10:55:44.287493+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T10:55:44.287493+0000","last_undegraded":"2024-04-20T10:55:44.287493+0000","last_fullsized":"2024-04-20T10:55:44.287493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285309+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T10:55:44.285309+0000","last_peered":"2024-04-20T10:55:44.285309+0000","last_clean":"2024-04-20T10:55:44.285309+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T10:55:44.285309+0000","last_undegraded":"2024-04-20T10:55:44.285309+0000","last_fullsized":"2024-04-20T10:55:44.285309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285746+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T10:55:44.285746+0000","last_peered":"2024-04-20T10:55:44.285746+0000","last_clean":"2024-04-20T10:55:44.285746+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T10:55:44.285746+0000","last_undegraded":"2024-04-20T10:55:44.285746+0000","last_fullsized":"2024-04-20T10:55:44.285746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800661+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T10:55:43.800661+0000","last_peered":"2024-04-20T10:55:43.800661+0000","last_clean":"2024-04-20T10:55:43.800661+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T10:55:43.800661+0000","last_undegraded":"2024-04-20T10:55:43.800661+0000","last_fullsized":"2024-04-20T10:55:43.800661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793161+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T10:55:43.793161+0000","last_peered":"2024-04-20T10:55:43.793161+0000","last_clean":"2024-04-20T10:55:43.793161+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T10:55:43.793161+0000","last_undegraded":"2024-04-20T10:55:43.793161+0000","last_fullsized":"2024-04-20T10:55:43.793161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285840+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T10:55:44.285840+0000","last_peered":"2024-04-20T10:55:44.285840+0000","last_clean":"2024-04-20T10:55:44.285840+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T10:55:44.285840+0000","last_undegraded":"2024-04-20T10:55:44.285840+0000","last_fullsized":"2024-04-20T10:55:44.285840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286466+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T10:55:44.286466+0000","last_peered":"2024-04-20T10:55:44.286466+0000","last_clean":"2024-04-20T10:55:44.286466+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T10:55:44.286466+0000","last_undegraded":"2024-04-20T10:55:44.286466+0000","last_fullsized":"2024-04-20T10:55:44.286466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793178+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T10:55:43.793178+0000","last_peered":"2024-04-20T10:55:43.793178+0000","last_clean":"2024-04-20T10:55:43.793178+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T10:55:43.793178+0000","last_undegraded":"2024-04-20T10:55:43.793178+0000","last_fullsized":"2024-04-20T10:55:43.793178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792877+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T10:55:43.792877+0000","last_peered":"2024-04-20T10:55:43.792877+0000","last_clean":"2024-04-20T10:55:43.792877+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T10:55:43.792877+0000","last_undegraded":"2024-04-20T10:55:43.792877+0000","last_fullsized":"2024-04-20T10:55:43.792877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286112+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T10:55:44.286112+0000","last_peered":"2024-04-20T10:55:44.286112+0000","last_clean":"2024-04-20T10:55:44.286112+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T10:55:44.286112+0000","last_undegraded":"2024-04-20T10:55:44.286112+0000","last_fullsized":"2024-04-20T10:55:44.286112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797544+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T10:55:43.797544+0000","last_peered":"2024-04-20T10:55:43.797544+0000","last_clean":"2024-04-20T10:55:43.797544+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T10:55:43.797544+0000","last_undegraded":"2024-04-20T10:55:43.797544+0000","last_fullsized":"2024-04-20T10:55:43.797544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286640+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T10:55:44.286640+0000","last_peered":"2024-04-20T10:55:44.286640+0000","last_clean":"2024-04-20T10:55:44.286640+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T10:55:44.286640+0000","last_undegraded":"2024-04-20T10:55:44.286640+0000","last_fullsized":"2024-04-20T10:55:44.286640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287062+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T10:55:44.287062+0000","last_peered":"2024-04-20T10:55:44.287062+0000","last_clean":"2024-04-20T10:55:44.287062+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T10:55:44.287062+0000","last_undegraded":"2024-04-20T10:55:44.287062+0000","last_fullsized":"2024-04-20T10:55:44.287062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287172+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T10:55:44.287172+0000","last_peered":"2024-04-20T10:55:44.287172+0000","last_clean":"2024-04-20T10:55:44.287172+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T10:55:44.287172+0000","last_undegraded":"2024-04-20T10:55:44.287172+0000","last_fullsized":"2024-04-20T10:55:44.287172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285669+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T10:55:44.285669+0000","last_peered":"2024-04-20T10:55:44.285669+0000","last_clean":"2024-04-20T10:55:44.285669+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T10:55:44.285669+0000","last_undegraded":"2024-04-20T10:55:44.285669+0000","last_fullsized":"2024-04-20T10:55:44.285669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797533+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T10:55:43.797533+0000","last_peered":"2024-04-20T10:55:43.797533+0000","last_clean":"2024-04-20T10:55:43.797533+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T10:55:43.797533+0000","last_undegraded":"2024-04-20T10:55:43.797533+0000","last_fullsized":"2024-04-20T10:55:43.797533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801052+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T10:55:43.801052+0000","last_peered":"2024-04-20T10:55:43.801052+0000","last_clean":"2024-04-20T10:55:43.801052+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T10:55:43.801052+0000","last_undegraded":"2024-04-20T10:55:43.801052+0000","last_fullsized":"2024-04-20T10:55:43.801052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287101+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T10:55:44.287101+0000","last_peered":"2024-04-20T10:55:44.287101+0000","last_clean":"2024-04-20T10:55:44.287101+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T10:55:44.287101+0000","last_undegraded":"2024-04-20T10:55:44.287101+0000","last_fullsized":"2024-04-20T10:55:44.287101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287336+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T10:55:44.287336+0000","last_peered":"2024-04-20T10:55:44.287336+0000","last_clean":"2024-04-20T10:55:44.287336+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T10:55:44.287336+0000","last_undegraded":"2024-04-20T10:55:44.287336+0000","last_fullsized":"2024-04-20T10:55:44.287336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285817+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T10:55:44.285817+0000","last_peered":"2024-04-20T10:55:44.285817+0000","last_clean":"2024-04-20T10:55:44.285817+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T10:55:44.285817+0000","last_undegraded":"2024-04-20T10:55:44.285817+0000","last_fullsized":"2024-04-20T10:55:44.285817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285622+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T10:55:44.285622+0000","last_peered":"2024-04-20T10:55:44.285622+0000","last_clean":"2024-04-20T10:55:44.285622+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T10:55:44.285622+0000","last_undegraded":"2024-04-20T10:55:44.285622+0000","last_fullsized":"2024-04-20T10:55:44.285622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287576+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T10:55:44.287576+0000","last_peered":"2024-04-20T10:55:44.287576+0000","last_clean":"2024-04-20T10:55:44.287576+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T10:55:44.287576+0000","last_undegraded":"2024-04-20T10:55:44.287576+0000","last_fullsized":"2024-04-20T10:55:44.287576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286631+0000","last_change":"2024-04-20T10:55:42.805169+0000","last_active":"2024-04-20T10:55:44.286631+0000","last_peered":"2024-04-20T10:55:44.286631+0000","last_clean":"2024-04-20T10:55:44.286631+0000","last_became_active":"2024-04-20T10:55:42.804573+0000","last_became_peered":"2024-04-20T10:55:42.804573+0000","last_unstale":"2024-04-20T10:55:44.286631+0000","last_undegraded":"2024-04-20T10:55:44.286631+0000","last_fullsized":"2024-04-20T10:55:44.286631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:16.103447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287223+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T10:55:44.287223+0000","last_peered":"2024-04-20T10:55:44.287223+0000","last_clean":"2024-04-20T10:55:44.287223+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T10:55:44.287223+0000","last_undegraded":"2024-04-20T10:55:44.287223+0000","last_fullsized":"2024-04-20T10:55:44.287223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285762+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T10:55:44.285762+0000","last_peered":"2024-04-20T10:55:44.285762+0000","last_clean":"2024-04-20T10:55:44.285762+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T10:55:44.285762+0000","last_undegraded":"2024-04-20T10:55:44.285762+0000","last_fullsized":"2024-04-20T10:55:44.285762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285071+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T10:55:44.285071+0000","last_peered":"2024-04-20T10:55:44.285071+0000","last_clean":"2024-04-20T10:55:44.285071+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T10:55:44.285071+0000","last_undegraded":"2024-04-20T10:55:44.285071+0000","last_fullsized":"2024-04-20T10:55:44.285071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287518+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T10:55:44.287518+0000","last_peered":"2024-04-20T10:55:44.287518+0000","last_clean":"2024-04-20T10:55:44.287518+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T10:55:44.287518+0000","last_undegraded":"2024-04-20T10:55:44.287518+0000","last_fullsized":"2024-04-20T10:55:44.287518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800769+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T10:55:43.800769+0000","last_peered":"2024-04-20T10:55:43.800769+0000","last_clean":"2024-04-20T10:55:43.800769+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T10:55:43.800769+0000","last_undegraded":"2024-04-20T10:55:43.800769+0000","last_fullsized":"2024-04-20T10:55:43.800769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286460+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T10:55:44.286460+0000","last_peered":"2024-04-20T10:55:44.286460+0000","last_clean":"2024-04-20T10:55:44.286460+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T10:55:44.286460+0000","last_undegraded":"2024-04-20T10:55:44.286460+0000","last_fullsized":"2024-04-20T10:55:44.286460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285191+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T10:55:44.285191+0000","last_peered":"2024-04-20T10:55:44.285191+0000","last_clean":"2024-04-20T10:55:44.285191+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T10:55:44.285191+0000","last_undegraded":"2024-04-20T10:55:44.285191+0000","last_fullsized":"2024-04-20T10:55:44.285191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287067+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T10:55:44.287067+0000","last_peered":"2024-04-20T10:55:44.287067+0000","last_clean":"2024-04-20T10:55:44.287067+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T10:55:44.287067+0000","last_undegraded":"2024-04-20T10:55:44.287067+0000","last_fullsized":"2024-04-20T10:55:44.287067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797568+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T10:55:43.797568+0000","last_peered":"2024-04-20T10:55:43.797568+0000","last_clean":"2024-04-20T10:55:43.797568+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T10:55:43.797568+0000","last_undegraded":"2024-04-20T10:55:43.797568+0000","last_fullsized":"2024-04-20T10:55:43.797568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286721+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T10:55:44.286721+0000","last_peered":"2024-04-20T10:55:44.286721+0000","last_clean":"2024-04-20T10:55:44.286721+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T10:55:44.286721+0000","last_undegraded":"2024-04-20T10:55:44.286721+0000","last_fullsized":"2024-04-20T10:55:44.286721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285213+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T10:55:44.285213+0000","last_peered":"2024-04-20T10:55:44.285213+0000","last_clean":"2024-04-20T10:55:44.285213+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T10:55:44.285213+0000","last_undegraded":"2024-04-20T10:55:44.285213+0000","last_fullsized":"2024-04-20T10:55:44.285213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286934+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T10:55:44.286934+0000","last_peered":"2024-04-20T10:55:44.286934+0000","last_clean":"2024-04-20T10:55:44.286934+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T10:55:44.286934+0000","last_undegraded":"2024-04-20T10:55:44.286934+0000","last_fullsized":"2024-04-20T10:55:44.286934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286525+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T10:55:44.286525+0000","last_peered":"2024-04-20T10:55:44.286525+0000","last_clean":"2024-04-20T10:55:44.286525+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T10:55:44.286525+0000","last_undegraded":"2024-04-20T10:55:44.286525+0000","last_fullsized":"2024-04-20T10:55:44.286525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793252+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T10:55:43.793252+0000","last_peered":"2024-04-20T10:55:43.793252+0000","last_clean":"2024-04-20T10:55:43.793252+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T10:55:43.793252+0000","last_undegraded":"2024-04-20T10:55:43.793252+0000","last_fullsized":"2024-04-20T10:55:43.793252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287230+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T10:55:44.287230+0000","last_peered":"2024-04-20T10:55:44.287230+0000","last_clean":"2024-04-20T10:55:44.287230+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T10:55:44.287230+0000","last_undegraded":"2024-04-20T10:55:44.287230+0000","last_fullsized":"2024-04-20T10:55:44.287230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285651+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T10:55:44.285651+0000","last_peered":"2024-04-20T10:55:44.285651+0000","last_clean":"2024-04-20T10:55:44.285651+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T10:55:44.285651+0000","last_undegraded":"2024-04-20T10:55:44.285651+0000","last_fullsized":"2024-04-20T10:55:44.285651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285287+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T10:55:44.285287+0000","last_peered":"2024-04-20T10:55:44.285287+0000","last_clean":"2024-04-20T10:55:44.285287+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T10:55:44.285287+0000","last_undegraded":"2024-04-20T10:55:44.285287+0000","last_fullsized":"2024-04-20T10:55:44.285287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797319+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T10:55:43.797319+0000","last_peered":"2024-04-20T10:55:43.797319+0000","last_clean":"2024-04-20T10:55:43.797319+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T10:55:43.797319+0000","last_undegraded":"2024-04-20T10:55:43.797319+0000","last_fullsized":"2024-04-20T10:55:43.797319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287600+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T10:55:44.287600+0000","last_peered":"2024-04-20T10:55:44.287600+0000","last_clean":"2024-04-20T10:55:44.287600+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T10:55:44.287600+0000","last_undegraded":"2024-04-20T10:55:44.287600+0000","last_fullsized":"2024-04-20T10:55:44.287600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800914+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T10:55:43.800914+0000","last_peered":"2024-04-20T10:55:43.800914+0000","last_clean":"2024-04-20T10:55:43.800914+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T10:55:43.800914+0000","last_undegraded":"2024-04-20T10:55:43.800914+0000","last_fullsized":"2024-04-20T10:55:43.800914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797497+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T10:55:43.797497+0000","last_peered":"2024-04-20T10:55:43.797497+0000","last_clean":"2024-04-20T10:55:43.797497+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T10:55:43.797497+0000","last_undegraded":"2024-04-20T10:55:43.797497+0000","last_fullsized":"2024-04-20T10:55:43.797497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287087+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T10:55:44.287087+0000","last_peered":"2024-04-20T10:55:44.287087+0000","last_clean":"2024-04-20T10:55:44.287087+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T10:55:44.287087+0000","last_undegraded":"2024-04-20T10:55:44.287087+0000","last_fullsized":"2024-04-20T10:55:44.287087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287263+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T10:55:44.287263+0000","last_peered":"2024-04-20T10:55:44.287263+0000","last_clean":"2024-04-20T10:55:44.287263+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T10:55:44.287263+0000","last_undegraded":"2024-04-20T10:55:44.287263+0000","last_fullsized":"2024-04-20T10:55:44.287263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797411+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T10:55:43.797411+0000","last_peered":"2024-04-20T10:55:43.797411+0000","last_clean":"2024-04-20T10:55:43.797411+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T10:55:43.797411+0000","last_undegraded":"2024-04-20T10:55:43.797411+0000","last_fullsized":"2024-04-20T10:55:43.797411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797512+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T10:55:43.797512+0000","last_peered":"2024-04-20T10:55:43.797512+0000","last_clean":"2024-04-20T10:55:43.797512+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T10:55:43.797512+0000","last_undegraded":"2024-04-20T10:55:43.797512+0000","last_fullsized":"2024-04-20T10:55:43.797512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793562+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T10:55:43.793562+0000","last_peered":"2024-04-20T10:55:43.793562+0000","last_clean":"2024-04-20T10:55:43.793562+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T10:55:43.793562+0000","last_undegraded":"2024-04-20T10:55:43.793562+0000","last_fullsized":"2024-04-20T10:55:43.793562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286739+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T10:55:44.286739+0000","last_peered":"2024-04-20T10:55:44.286739+0000","last_clean":"2024-04-20T10:55:44.286739+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T10:55:44.286739+0000","last_undegraded":"2024-04-20T10:55:44.286739+0000","last_fullsized":"2024-04-20T10:55:44.286739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285397+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T10:55:44.285397+0000","last_peered":"2024-04-20T10:55:44.285397+0000","last_clean":"2024-04-20T10:55:44.285397+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T10:55:44.285397+0000","last_undegraded":"2024-04-20T10:55:44.285397+0000","last_fullsized":"2024-04-20T10:55:44.285397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285923+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T10:55:44.285923+0000","last_peered":"2024-04-20T10:55:44.285923+0000","last_clean":"2024-04-20T10:55:44.285923+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T10:55:44.285923+0000","last_undegraded":"2024-04-20T10:55:44.285923+0000","last_fullsized":"2024-04-20T10:55:44.285923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800994+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T10:55:43.800994+0000","last_peered":"2024-04-20T10:55:43.800994+0000","last_clean":"2024-04-20T10:55:43.800994+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T10:55:43.800994+0000","last_undegraded":"2024-04-20T10:55:43.800994+0000","last_fullsized":"2024-04-20T10:55:43.800994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797429+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T10:55:43.797429+0000","last_peered":"2024-04-20T10:55:43.797429+0000","last_clean":"2024-04-20T10:55:43.797429+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T10:55:43.797429+0000","last_undegraded":"2024-04-20T10:55:43.797429+0000","last_fullsized":"2024-04-20T10:55:43.797429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793024+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T10:55:43.793024+0000","last_peered":"2024-04-20T10:55:43.793024+0000","last_clean":"2024-04-20T10:55:43.793024+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T10:55:43.793024+0000","last_undegraded":"2024-04-20T10:55:43.793024+0000","last_fullsized":"2024-04-20T10:55:43.793024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797381+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T10:55:43.797381+0000","last_peered":"2024-04-20T10:55:43.797381+0000","last_clean":"2024-04-20T10:55:43.797381+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T10:55:43.797381+0000","last_undegraded":"2024-04-20T10:55:43.797381+0000","last_fullsized":"2024-04-20T10:55:43.797381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793105+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T10:55:43.793105+0000","last_peered":"2024-04-20T10:55:43.793105+0000","last_clean":"2024-04-20T10:55:43.793105+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T10:55:43.793105+0000","last_undegraded":"2024-04-20T10:55:43.793105+0000","last_fullsized":"2024-04-20T10:55:43.793105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286780+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T10:55:44.286780+0000","last_peered":"2024-04-20T10:55:44.286780+0000","last_clean":"2024-04-20T10:55:44.286780+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T10:55:44.286780+0000","last_undegraded":"2024-04-20T10:55:44.286780+0000","last_fullsized":"2024-04-20T10:55:44.286780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286369+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T10:55:44.286369+0000","last_peered":"2024-04-20T10:55:44.286369+0000","last_clean":"2024-04-20T10:55:44.286369+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T10:55:44.286369+0000","last_undegraded":"2024-04-20T10:55:44.286369+0000","last_fullsized":"2024-04-20T10:55:44.286369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286936+0000","last_change":"2024-04-20T10:55:42.792905+0000","last_active":"2024-04-20T10:55:44.286936+0000","last_peered":"2024-04-20T10:55:44.286936+0000","last_clean":"2024-04-20T10:55:44.286936+0000","last_became_active":"2024-04-20T10:55:42.792587+0000","last_became_peered":"2024-04-20T10:55:42.792587+0000","last_unstale":"2024-04-20T10:55:44.286936+0000","last_undegraded":"2024-04-20T10:55:44.286936+0000","last_fullsized":"2024-04-20T10:55:44.286936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:02.987448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286090+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T10:55:44.286090+0000","last_peered":"2024-04-20T10:55:44.286090+0000","last_clean":"2024-04-20T10:55:44.286090+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T10:55:44.286090+0000","last_undegraded":"2024-04-20T10:55:44.286090+0000","last_fullsized":"2024-04-20T10:55:44.286090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286560+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T10:55:44.286560+0000","last_peered":"2024-04-20T10:55:44.286560+0000","last_clean":"2024-04-20T10:55:44.286560+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T10:55:44.286560+0000","last_undegraded":"2024-04-20T10:55:44.286560+0000","last_fullsized":"2024-04-20T10:55:44.286560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287033+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T10:55:44.287033+0000","last_peered":"2024-04-20T10:55:44.287033+0000","last_clean":"2024-04-20T10:55:44.287033+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T10:55:44.287033+0000","last_undegraded":"2024-04-20T10:55:44.287033+0000","last_fullsized":"2024-04-20T10:55:44.287033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793368+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T10:55:43.793368+0000","last_peered":"2024-04-20T10:55:43.793368+0000","last_clean":"2024-04-20T10:55:43.793368+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T10:55:43.793368+0000","last_undegraded":"2024-04-20T10:55:43.793368+0000","last_fullsized":"2024-04-20T10:55:43.793368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793370+0000","last_change":"2024-04-20T10:55:42.802227+0000","last_active":"2024-04-20T10:55:43.793370+0000","last_peered":"2024-04-20T10:55:43.793370+0000","last_clean":"2024-04-20T10:55:43.793370+0000","last_became_active":"2024-04-20T10:55:42.801944+0000","last_became_peered":"2024-04-20T10:55:42.801944+0000","last_unstale":"2024-04-20T10:55:43.793370+0000","last_undegraded":"2024-04-20T10:55:43.793370+0000","last_fullsized":"2024-04-20T10:55:43.793370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:29.862643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285734+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T10:55:44.285734+0000","last_peered":"2024-04-20T10:55:44.285734+0000","last_clean":"2024-04-20T10:55:44.285734+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T10:55:44.285734+0000","last_undegraded":"2024-04-20T10:55:44.285734+0000","last_fullsized":"2024-04-20T10:55:44.285734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":655002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":786432,"data_stored":655002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:55:48.174 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T10:55:48.174 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T10:55:48.174 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-20T10:55:48.175 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-20T10:55:48.175 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-20T10:55:48.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.310+0000 7fc03ec70640 1 Processor -- start 2024-04-20T10:55:48.311 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.310+0000 7fc03ec70640 1 -- start start 2024-04-20T10:55:48.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ec70640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc038155800 0x7fc03815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ec70640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc0380cbf50 0x7fc0380c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ec70640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03815c4d0 con 0x7fc0380c6500 2024-04-20T10:55:48.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ec70640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc0380ce610 con 0x7fc038155800 2024-04-20T10:55:48.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ec70640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fc0380ce180 con 0x7fc0380cbf50 2024-04-20T10:55:48.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ca0c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc038155800 0x7fc03815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ca0c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc038155800 0x7fc03815bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48500/0 (socket says 172.21.15.27:48500) 2024-04-20T10:55:48.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ca0c640 1 -- 172.21.15.27:0/1694352294 learned_addr learned my addr 172.21.15.27:0/1694352294 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:48.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ca0c640 1 -- 172.21.15.27:0/1694352294 >> v2:172.21.15.137:3300/0 conn(0x7fc0380cbf50 msgr2=0x7fc0380c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:48.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ca0c640 1 --2- 172.21.15.27:0/1694352294 >> v2:172.21.15.137:3300/0 conn(0x7fc0380cbf50 0x7fc0380c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ca0c640 1 -- 172.21.15.27:0/1694352294 >> v1:172.21.15.137:6789/0 conn(0x7fc0380c6500 legacy=0x7fc0380d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:48.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.311+0000 7fc03ca0c640 1 -- 172.21.15.27:0/1694352294 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0380cd290 con 0x7fc038155800 2024-04-20T10:55:48.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.312+0000 7fc03ca0c640 1 --2- 172.21.15.27:0/1694352294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc038155800 0x7fc03815bf90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc03806ec00 tx=0x7fc03002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ca815b0ebd992 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:48.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.312+0000 7fc0377fe640 1 -- 172.21.15.27:0/1694352294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03002c9f0 con 0x7fc038155800 2024-04-20T10:55:48.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.312+0000 7fc0377fe640 1 -- 172.21.15.27:0/1694352294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03002cb90 con 0x7fc038155800 2024-04-20T10:55:48.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.312+0000 7fc0377fe640 1 -- 172.21.15.27:0/1694352294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0300343f0 con 0x7fc038155800 2024-04-20T10:55:48.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.313+0000 7fc03ec70640 1 -- 172.21.15.27:0/1694352294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc038155800 msgr2=0x7fc03815bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.313+0000 7fc03ec70640 1 --2- 172.21.15.27:0/1694352294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc038155800 0x7fc03815bf90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc03806ec00 tx=0x7fc03002be50 comp rx=0 tx=0).stop 2024-04-20T10:55:48.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.313+0000 7fc03ec70640 1 -- 172.21.15.27:0/1694352294 shutdown_connections 2024-04-20T10:55:48.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.313+0000 7fc03ec70640 1 --2- 172.21.15.27:0/1694352294 >> v2:172.21.15.137:3300/0 conn(0x7fc0380cbf50 0x7fc0380c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.313+0000 7fc03ec70640 1 --2- 172.21.15.27:0/1694352294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc038155800 0x7fc03815bf90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.313+0000 7fc03ec70640 1 -- 172.21.15.27:0/1694352294 >> 172.21.15.27:0/1694352294 conn(0x7fc038067ae0 msgr2=0x7fc038067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:48.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.313+0000 7fc03ec70640 1 -- 172.21.15.27:0/1694352294 shutdown_connections 2024-04-20T10:55:48.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.313+0000 7fc03ec70640 1 -- 172.21.15.27:0/1694352294 wait complete. 2024-04-20T10:55:48.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.314+0000 7fc03ec70640 1 Processor -- start 2024-04-20T10:55:48.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.314+0000 7fc03ec70640 1 -- start start 2024-04-20T10:55:48.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc03ec70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0380c6500 0x7fc03814b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc03ec70640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0380cbf50 0x7fc0381f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc03ec70640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc038155800 0x7fc038150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc03ec70640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380cdb30 con 0x7fc0380cbf50 2024-04-20T10:55:48.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc03ec70640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0380cd850 con 0x7fc0380c6500 2024-04-20T10:55:48.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc03ec70640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc0380cde10 con 0x7fc038155800 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc037fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0380cbf50 0x7fc0381f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc037fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0380cbf50 0x7fc0381f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48504/0 (socket says 172.21.15.27:48504) 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc037fff640 1 -- 172.21.15.27:0/1472494823 learned_addr learned my addr 172.21.15.27:0/1472494823 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc03d20d640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc038155800 0x7fc038150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc03ca0c640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0380c6500 0x7fc03814b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc037fff640 1 -- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc038155800 msgr2=0x7fc038150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc037fff640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc038155800 0x7fc038150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc037fff640 1 -- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0380c6500 msgr2=0x7fc03814b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc037fff640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0380c6500 0x7fc03814b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc037fff640 1 -- 172.21.15.27:0/1472494823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0381f55d0 con 0x7fc0380cbf50 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.315+0000 7fc037fff640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0380cbf50 0x7fc0381f1200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00b4d0 tx=0x7fc02c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.316+0000 7fc0357fa640 1 -- 172.21.15.27:0/1472494823 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c014070 con 0x7fc0380cbf50 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.316+0000 7fc0357fa640 1 -- 172.21.15.27:0/1472494823 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c010870 con 0x7fc0380cbf50 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.316+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc038159d30 con 0x7fc0380cbf50 2024-04-20T10:55:48.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.316+0000 7fc0357fa640 1 -- 172.21.15.27:0/1472494823 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c010c60 con 0x7fc0380cbf50 2024-04-20T10:55:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.316+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0381f4af0 con 0x7fc0380cbf50 2024-04-20T10:55:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.317+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0380688e0 con 0x7fc0380cbf50 2024-04-20T10:55:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.317+0000 7fc0357fa640 1 -- 172.21.15.27:0/1472494823 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc02c017070 con 0x7fc0380cbf50 2024-04-20T10:55:48.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.318+0000 7fc0357fa640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc02406c8f0 0x7fc02406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.318+0000 7fc0357fa640 1 -- 172.21.15.27:0/1472494823 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc02c08b910 con 0x7fc0380cbf50 2024-04-20T10:55:48.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.318+0000 7fc03ca0c640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc02406c8f0 0x7fc02406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.318+0000 7fc03ca0c640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc02406c8f0 0x7fc02406edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc030035000 tx=0x7fc030030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:48.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.322+0000 7fc0357fa640 1 -- 172.21.15.27:0/1472494823 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc02c059d40 con 0x7fc0380cbf50 2024-04-20T10:55:48.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.555+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc0381523d0 con 0x7fc0380cbf50 2024-04-20T10:55:48.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.558+0000 7fc0357fa640 1 -- 172.21.15.27:0/1472494823 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7fc02c059640 con 0x7fc0380cbf50 2024-04-20T10:55:48.561 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:48.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc02406c8f0 msgr2=0x7fc02406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc02406c8f0 0x7fc02406edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc030035000 tx=0x7fc030030000 comp rx=0 tx=0).stop 2024-04-20T10:55:48.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0380cbf50 msgr2=0x7fc0381f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0380cbf50 0x7fc0381f1200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00b4d0 tx=0x7fc02c00b990 comp rx=0 tx=0).stop 2024-04-20T10:55:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 shutdown_connections 2024-04-20T10:55:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc02406c8f0 0x7fc02406edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc038155800 0x7fc038150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0380cbf50 0x7fc0381f1200 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 --2- 172.21.15.27:0/1472494823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0380c6500 0x7fc03814b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 >> 172.21.15.27:0/1472494823 conn(0x7fc038067ae0 msgr2=0x7fc0381545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 shutdown_connections 2024-04-20T10:55:48.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.561+0000 7fc03ec70640 1 -- 172.21.15.27:0/1472494823 wait complete. 2024-04-20T10:55:48.572 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2036904965},{"type":"v1","addr":"172.21.15.137:6833","nonce":2036904965}]},"active_addr":"172.21.15.137:6833/2036904965","active_change":"2024-04-20T10:55:21.688727+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":601545398}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":3177544704}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":2955492386}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":1287433524}]}]} 2024-04-20T10:55:48.575 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-20T10:55:48.575 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-20T10:55:48.576 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:48.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.675+0000 7f122d088640 1 Processor -- start 2024-04-20T10:55:48.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.675+0000 7f122d088640 1 -- start start 2024-04-20T10:55:48.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.675+0000 7f122d088640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 0x7f12280ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.675+0000 7f122d088640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f122815b790 con 0x7f12280cdea0 2024-04-20T10:55:48.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.675+0000 7f122d088640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f122815ba70 con 0x7f12280cd4f0 2024-04-20T10:55:48.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.675+0000 7f122d088640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f12280cbe60 con 0x7f12280cc060 2024-04-20T10:55:48.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f1226d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f12280cdea0 0x7f12280ccde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53408/0 (socket says 172.21.15.27:53408) 2024-04-20T10:55:48.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f1226d9d640 1 -- 172.21.15.27:0/936579526 learned_addr learned my addr 172.21.15.27:0/936579526 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:48.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f122759e640 1 --2- 172.21.15.27:0/936579526 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 0x7f12280ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f122659c640 1 -- 172.21.15.27:0/936579526 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2076534651 0 0) 0x7f122815b790 con 0x7f12280cdea0 2024-04-20T10:55:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f122659c640 1 -- 172.21.15.27:0/936579526 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1214003520 con 0x7f12280cdea0 2024-04-20T10:55:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f122659c640 1 -- 172.21.15.27:0/936579526 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093078205 0 0) 0x7f122815ba70 con 0x7f12280cd4f0 2024-04-20T10:55:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f122659c640 1 -- 172.21.15.27:0/936579526 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f122815b790 con 0x7f12280cd4f0 2024-04-20T10:55:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f122759e640 1 -- 172.21.15.27:0/936579526 >> v1:172.21.15.137:6789/0 conn(0x7f12280cd4f0 legacy=0x7f1228163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f122759e640 1 -- 172.21.15.27:0/936579526 >> v1:172.21.15.27:6789/0 conn(0x7f12280cdea0 legacy=0x7f12280ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.676+0000 7f122759e640 1 -- 172.21.15.27:0/936579526 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122815bd50 con 0x7f12280cc060 2024-04-20T10:55:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.677+0000 7f122759e640 1 --2- 172.21.15.27:0/936579526 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 0x7f12280ce280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1220009dc0 tx=0x7f122002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0be3549ec440e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.677+0000 7f122659c640 1 -- 172.21.15.27:0/936579526 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220039070 con 0x7f12280cc060 2024-04-20T10:55:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.677+0000 7f122659c640 1 -- 172.21.15.27:0/936579526 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122002ccd0 con 0x7f12280cc060 2024-04-20T10:55:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.677+0000 7f122659c640 1 -- 172.21.15.27:0/936579526 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220034640 con 0x7f12280cc060 2024-04-20T10:55:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.678+0000 7f122d088640 1 -- 172.21.15.27:0/936579526 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 msgr2=0x7f12280ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.678+0000 7f122d088640 1 --2- 172.21.15.27:0/936579526 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 0x7f12280ce280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1220009dc0 tx=0x7f122002bf20 comp rx=0 tx=0).stop 2024-04-20T10:55:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.678+0000 7f122d088640 1 -- 172.21.15.27:0/936579526 shutdown_connections 2024-04-20T10:55:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.678+0000 7f122d088640 1 --2- 172.21.15.27:0/936579526 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 0x7f12280ce280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.678+0000 7f122d088640 1 -- 172.21.15.27:0/936579526 >> 172.21.15.27:0/936579526 conn(0x7f122806dee0 msgr2=0x7f122806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.678+0000 7f122d088640 1 -- 172.21.15.27:0/936579526 shutdown_connections 2024-04-20T10:55:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.678+0000 7f122d088640 1 -- 172.21.15.27:0/936579526 wait complete. 2024-04-20T10:55:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.679+0000 7f122d088640 1 Processor -- start 2024-04-20T10:55:48.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.679+0000 7f122d088640 1 -- start start 2024-04-20T10:55:48.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f122d088640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 0x7f12280cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f122d088640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12280cd4f0 0x7f1228155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f122d088640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12280cdea0 0x7f1228145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f122d088640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f122815c310 con 0x7f12280cd4f0 2024-04-20T10:55:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f122d088640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12280cc950 con 0x7f12280cdea0 2024-04-20T10:55:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f122d088640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f1228155b20 con 0x7f12280cc060 2024-04-20T10:55:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12280cd4f0 0x7f1228155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12280cd4f0 0x7f1228155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48512/0 (socket says 172.21.15.27:48512) 2024-04-20T10:55:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121ffff640 1 -- 172.21.15.27:0/2561610754 learned_addr learned my addr 172.21.15.27:0/2561610754 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f1226d9d640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 0x7f12280cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f122759e640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12280cdea0 0x7f1228145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121ffff640 1 -- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 msgr2=0x7f12280cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121ffff640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 0x7f12280cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121ffff640 1 -- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12280cdea0 msgr2=0x7f1228145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121ffff640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12280cdea0 0x7f1228145390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121ffff640 1 -- 172.21.15.27:0/2561610754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12281585c0 con 0x7f12280cd4f0 2024-04-20T10:55:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121ffff640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12280cd4f0 0x7f1228155dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f121000c400 tx=0x7f121000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121f7fe640 1 -- 172.21.15.27:0/2561610754 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210017070 con 0x7f12280cd4f0 2024-04-20T10:55:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121f7fe640 1 -- 172.21.15.27:0/2561610754 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12100052d0 con 0x7f12280cd4f0 2024-04-20T10:55:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12281588a0 con 0x7f12280cd4f0 2024-04-20T10:55:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f121f7fe640 1 -- 172.21.15.27:0/2561610754 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1210003810 con 0x7f12280cd4f0 2024-04-20T10:55:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.680+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12280ca500 con 0x7f12280cd4f0 2024-04-20T10:55:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.681+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12280ca010 con 0x7f12280cd4f0 2024-04-20T10:55:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.682+0000 7f121f7fe640 1 -- 172.21.15.27:0/2561610754 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f121001a070 con 0x7f12280cd4f0 2024-04-20T10:55:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.682+0000 7f121f7fe640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f120806c8f0 0x7f120806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.682+0000 7f121f7fe640 1 -- 172.21.15.27:0/2561610754 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f121008aef0 con 0x7f12280cd4f0 2024-04-20T10:55:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.682+0000 7f1226d9d640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f120806c8f0 0x7f120806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.683+0000 7f1226d9d640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f120806c8f0 0x7f120806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12180011e0 tx=0x7f1218007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.687+0000 7f121f7fe640 1 -- 172.21.15.27:0/2561610754 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1210058cd0 con 0x7f12280cd4f0 2024-04-20T10:55:48.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.872+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f122804fcf0 con 0x7f12280cd4f0 2024-04-20T10:55:48.873 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.873+0000 7f121f7fe640 1 -- 172.21.15.27:0/2561610754 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16569 (secure 0 0 0) 0x7f121005cfd0 con 0x7f12280cd4f0 2024-04-20T10:55:48.874 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:48.874 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:43.787038+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f120806c8f0 msgr2=0x7f120806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f120806c8f0 0x7f120806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12180011e0 tx=0x7f1218007d30 comp rx=0 tx=0).stop 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12280cd4f0 msgr2=0x7f1228155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12280cd4f0 0x7f1228155dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f121000c400 tx=0x7f121000c8c0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 shutdown_connections 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f120806c8f0 0x7f120806edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12280cdea0 0x7f1228145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12280cd4f0 0x7f1228155dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 --2- 172.21.15.27:0/2561610754 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f12280cc060 0x7f12280cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 >> 172.21.15.27:0/2561610754 conn(0x7f122806dee0 msgr2=0x7f12280ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:48.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 shutdown_connections 2024-04-20T10:55:48.880 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.875+0000 7f122d088640 1 -- 172.21.15.27:0/2561610754 wait complete. 2024-04-20T10:55:48.887 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-20T10:55:48.887 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:55:48.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.987+0000 7fbb0fcc8640 1 Processor -- start 2024-04-20T10:55:48.988 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.987+0000 7fbb0fcc8640 1 -- start start 2024-04-20T10:55:48.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0fcc8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb0815ee50 0x7fbb0815c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0fcc8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb080ccbd0 0x7fbb08155410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0fcc8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb0815f9a0 con 0x7fbb0815f230 2024-04-20T10:55:48.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0fcc8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbb080cb870 con 0x7fbb0815ee50 2024-04-20T10:55:48.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0fcc8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbb080cbb30 con 0x7fbb080ccbd0 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0da64640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb0815ee50 0x7fbb0815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0da64640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb0815ee50 0x7fbb0815c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48516/0 (socket says 172.21.15.27:48516) 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0d263640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbb0815f230 0x7fbb0806cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53414/0 (socket says 172.21.15.27:53414) 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0da64640 1 -- 172.21.15.27:0/79389052 learned_addr learned my addr 172.21.15.27:0/79389052 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0e265640 1 --2- 172.21.15.27:0/79389052 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb080ccbd0 0x7fbb08155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0da64640 1 -- 172.21.15.27:0/79389052 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb080ccbd0 msgr2=0x7fbb08155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0da64640 1 --2- 172.21.15.27:0/79389052 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb080ccbd0 0x7fbb08155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0da64640 1 -- 172.21.15.27:0/79389052 >> v1:172.21.15.27:6789/0 conn(0x7fbb0815f230 legacy=0x7fbb0806cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0da64640 1 -- 172.21.15.27:0/79389052 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb080cbe10 con 0x7fbb0815ee50 2024-04-20T10:55:48.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.988+0000 7fbb0ca62640 1 -- 172.21.15.27:0/79389052 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3195523656 0 0) 0x7fbaf8001110 con 0x7fbb0815f230 2024-04-20T10:55:48.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.989+0000 7fbb0da64640 1 --2- 172.21.15.27:0/79389052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0815ee50 0x7fbb0815c180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009870 tx=0x7fbb0402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbeee12e105f34d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.989+0000 7fbb0ca62640 1 -- 172.21.15.27:0/79389052 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0402f840 con 0x7fbb0815ee50 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.989+0000 7fbb0ca62640 1 -- 172.21.15.27:0/79389052 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0402f9e0 con 0x7fbb0815ee50 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.989+0000 7fbb0ca62640 1 -- 172.21.15.27:0/79389052 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0402fdb0 con 0x7fbb0815ee50 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.990+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/79389052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0815ee50 msgr2=0x7fbb0815c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.990+0000 7fbb0fcc8640 1 --2- 172.21.15.27:0/79389052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0815ee50 0x7fbb0815c180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009870 tx=0x7fbb0402ba20 comp rx=0 tx=0).stop 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.990+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/79389052 shutdown_connections 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.990+0000 7fbb0fcc8640 1 --2- 172.21.15.27:0/79389052 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb080ccbd0 0x7fbb08155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.990+0000 7fbb0fcc8640 1 --2- 172.21.15.27:0/79389052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0815ee50 0x7fbb0815c180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.990+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/79389052 >> 172.21.15.27:0/79389052 conn(0x7fbb0806e010 msgr2=0x7fbb080d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.990+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/79389052 shutdown_connections 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.990+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/79389052 wait complete. 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.991+0000 7fbb0fcc8640 1 Processor -- start 2024-04-20T10:55:48.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.991+0000 7fbb0fcc8640 1 -- start start 2024-04-20T10:55:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0fcc8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080ccbd0 0x7fbb0815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0fcc8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb0815ee50 0x7fbb0814e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0fcc8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb0815f230 0x7fbb081f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0fcc8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbb080cc6b0 con 0x7fbb080ccbd0 2024-04-20T10:55:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0fcc8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb080cc3d0 con 0x7fbb0815f230 2024-04-20T10:55:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0fcc8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0815cae0 con 0x7fbb0815ee50 2024-04-20T10:55:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0da64640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080ccbd0 0x7fbb0815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0da64640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080ccbd0 0x7fbb0815b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48526/0 (socket says 172.21.15.27:48526) 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0da64640 1 -- 172.21.15.27:0/1463695470 learned_addr learned my addr 172.21.15.27:0/1463695470 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0d263640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb0815ee50 0x7fbb0814e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0e265640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb0815f230 0x7fbb081f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0da64640 1 -- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb0815ee50 msgr2=0x7fbb0814e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0da64640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb0815ee50 0x7fbb0814e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0da64640 1 -- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb0815f230 msgr2=0x7fbb081f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0da64640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb0815f230 0x7fbb081f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0da64640 1 -- 172.21.15.27:0/1463695470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0814fda0 con 0x7fbb080ccbd0 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0da64640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080ccbd0 0x7fbb0815b920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000c00 tx=0x7fbb04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbafe7fc640 1 -- 172.21.15.27:0/1463695470 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04037c90 con 0x7fbb080ccbd0 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbafe7fc640 1 -- 172.21.15.27:0/1463695470 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb04046070 con 0x7fbb080ccbd0 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0814d050 con 0x7fbb080ccbd0 2024-04-20T10:55:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.992+0000 7fbafe7fc640 1 -- 172.21.15.27:0/1463695470 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04041640 con 0x7fbb080ccbd0 2024-04-20T10:55:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.993+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0814df80 con 0x7fbb080ccbd0 2024-04-20T10:55:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.994+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb080688e0 con 0x7fbb080ccbd0 2024-04-20T10:55:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.994+0000 7fbafe7fc640 1 -- 172.21.15.27:0/1463695470 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb04040070 con 0x7fbb080ccbd0 2024-04-20T10:55:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.994+0000 7fbafe7fc640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbaec06c8f0 0x7fbaec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.994+0000 7fbafe7fc640 1 -- 172.21.15.27:0/1463695470 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbb040adfd0 con 0x7fbb080ccbd0 2024-04-20T10:55:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.994+0000 7fbb0d263640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbaec06c8f0 0x7fbaec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.995+0000 7fbb0d263640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbaec06c8f0 0x7fbaec06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbb080b5a30 tx=0x7fbaf8007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:48.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:48.998+0000 7fbafe7fc640 1 -- 172.21.15.27:0/1463695470 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbb04033030 con 0x7fbb080ccbd0 2024-04-20T10:55:49.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.183+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb0804fcf0 con 0x7fbb080ccbd0 2024-04-20T10:55:49.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.185+0000 7fbafe7fc640 1 -- 172.21.15.27:0/1463695470 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16569 (secure 0 0 0) 0x7fbb0407be40 con 0x7fbb080ccbd0 2024-04-20T10:55:49.185 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:49.186 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:43.787038+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:55:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.186+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbaec06c8f0 msgr2=0x7fbaec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.186+0000 7fbb0fcc8640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbaec06c8f0 0x7fbaec06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbb080b5a30 tx=0x7fbaf8007b50 comp rx=0 tx=0).stop 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.186+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080ccbd0 msgr2=0x7fbb0815b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.186+0000 7fbb0fcc8640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080ccbd0 0x7fbb0815b920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000c00 tx=0x7fbb04004730 comp rx=0 tx=0).stop 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.186+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 shutdown_connections 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.186+0000 7fbb0fcc8640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbaec06c8f0 0x7fbaec06edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.186+0000 7fbb0fcc8640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb0815f230 0x7fbb081f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.186+0000 7fbb0fcc8640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbb0815ee50 0x7fbb0814e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.186+0000 7fbb0fcc8640 1 --2- 172.21.15.27:0/1463695470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080ccbd0 0x7fbb0815b920 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.187+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 >> 172.21.15.27:0/1463695470 conn(0x7fbb0806e010 msgr2=0x7fbb080ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.187+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 shutdown_connections 2024-04-20T10:55:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.187+0000 7fbb0fcc8640 1 -- 172.21.15.27:0/1463695470 wait complete. 2024-04-20T10:55:49.198 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-20T10:55:49.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-20T10:55:49.199 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-20T10:55:49.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-20T10:55:49.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-20T10:55:49.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-20T10:55:49.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-20T10:55:49.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-20T10:55:49.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.374+0000 7f8aa0481640 1 Processor -- start 2024-04-20T10:55:49.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.375+0000 7f8aa0481640 1 -- start start 2024-04-20T10:55:49.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.375+0000 7f536ebe1640 1 Processor -- start 2024-04-20T10:55:49.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.375+0000 7f71b5890640 1 Processor -- start 2024-04-20T10:55:49.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.376+0000 7f71b5890640 1 -- start start 2024-04-20T10:55:49.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.376+0000 7f8aa0481640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 0x7f8a900066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.375+0000 7f44a759e640 1 Processor -- start 2024-04-20T10:55:49.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.376+0000 7f44a759e640 1 -- start start 2024-04-20T10:55:49.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7ff12b3e1640 1 Processor -- start 2024-04-20T10:55:49.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.376+0000 7f536ebe1640 1 -- start start 2024-04-20T10:55:49.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.376+0000 7f8aa0481640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8a900e6690 0x7f8a900e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.376+0000 7f8aa0481640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a900dd840 con 0x7f8a900dd460 2024-04-20T10:55:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.376+0000 7f8aa0481640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9004ba90 con 0x7f8a900e6690 2024-04-20T10:55:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.376+0000 7f8aa0481640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8a900e7600 con 0x7f8a900062b0 2024-04-20T10:55:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.376+0000 7f8a9da1c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 0x7f8a900066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f8a9da1c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 0x7f8a900066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60690/0 (socket says 172.21.15.27:60690) 2024-04-20T10:55:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f8a9da1c640 1 -- 172.21.15.27:0/3373475687 learned_addr learned my addr 172.21.15.27:0/3373475687 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f8a9ea1e640 1 --2- 172.21.15.27:0/3373475687 >> v2:172.21.15.137:3300/0 conn(0x7f8a900e6690 0x7f8a900e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f71b5890640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f71a80dd6c0 0x7f71a8007d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f71b5890640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f71a8046300 0x7f71a804bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f71b5890640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a8006ff0 con 0x7f71a8047bf0 2024-04-20T10:55:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f71b5890640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f71a80ddaa0 con 0x7f71a80dd6c0 2024-04-20T10:55:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f71b5890640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f71a80078e0 con 0x7f71a8046300 2024-04-20T10:55:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7feebe88c640 1 Processor -- start 2024-04-20T10:55:49.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7feebe88c640 1 -- start start 2024-04-20T10:55:49.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f44a759e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f44a000e2a0 0x7f44a00d5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f44a759e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a004b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f44a759e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a0045ed0 con 0x7f44a00db830 2024-04-20T10:55:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f44a759e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f44a0046f30 con 0x7f44a000e2a0 2024-04-20T10:55:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f44a759e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f44a0046170 con 0x7f44a004b090 2024-04-20T10:55:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a6d9d640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f44a00db830 0x7f44a0005e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:46606/0 (socket says 172.21.15.27:46606) 2024-04-20T10:55:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a6d9d640 1 -- 172.21.15.27:0/1279854179 learned_addr learned my addr 172.21.15.27:0/1279854179 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a5d9b640 1 --2- 172.21.15.27:0/1279854179 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a004b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a659c640 1 --2- 172.21.15.27:0/1279854179 >> v2:172.21.15.137:3300/0 conn(0x7f44a000e2a0 0x7f44a00d5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f536ebe1640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f53680732f0 0x7f536815f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f536ebe1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 0x7f53680ca490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f536ebe1640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5368163080 con 0x7f53680c4850 2024-04-20T10:55:49.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f536ebe1640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f5368163360 con 0x7f53680732f0 2024-04-20T10:55:49.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f536ebe1640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5368163640 con 0x7f53680736d0 2024-04-20T10:55:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f71b508f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f71a8047bf0 0x7f71a80470b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53418/0 (socket says 172.21.15.27:53418) 2024-04-20T10:55:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f71b508f640 1 -- 172.21.15.27:0/2747830800 learned_addr learned my addr 172.21.15.27:0/2747830800 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f71af7fe640 1 -- 172.21.15.27:0/2747830800 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 606270530 0 0) 0x7f71a8006ff0 con 0x7f71a8047bf0 2024-04-20T10:55:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f71af7fe640 1 -- 172.21.15.27:0/2747830800 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71a0003520 con 0x7f71a8047bf0 2024-04-20T10:55:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f71b488e640 1 --2- 172.21.15.27:0/2747830800 >> v2:172.21.15.27:3300/0 conn(0x7f71a80dd6c0 0x7f71a8007d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f71affff640 1 --2- 172.21.15.27:0/2747830800 >> v2:172.21.15.137:3300/0 conn(0x7f71a8046300 0x7f71a804bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f536dbdf640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f53680732f0 0x7f536815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a559a640 1 -- 172.21.15.27:0/1279854179 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 176402935 0 0) 0x7f44a0045ed0 con 0x7f44a00db830 2024-04-20T10:55:49.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a559a640 1 -- 172.21.15.27:0/1279854179 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f448c003520 con 0x7f44a00db830 2024-04-20T10:55:49.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7ff12b3e1640 1 -- start start 2024-04-20T10:55:49.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7ff12b3e1640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff11c04ba60 0x7ff11c04be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7ff12b3e1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11c046b70 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7ff12b3e1640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11c0dd7a0 con 0x7ff11c0e3090 2024-04-20T10:55:49.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7ff12b3e1640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7ff11c045450 con 0x7ff11c04ba60 2024-04-20T10:55:49.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7feebe88c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7feeb000e290 0x7feeb004b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7feebe88c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feeb004b080 0x7feeb00db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7feebe88c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeb00dbd60 con 0x7feeb00d4f80 2024-04-20T10:55:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7feebe88c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7feeb0046f20 con 0x7feeb004b080 2024-04-20T10:55:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7feebe88c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7feeb0046160 con 0x7feeb000e290 2024-04-20T10:55:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a5d9b640 1 -- 172.21.15.27:0/1279854179 >> v2:172.21.15.137:3300/0 conn(0x7f44a000e2a0 msgr2=0x7f44a00d5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a5d9b640 1 --2- 172.21.15.27:0/1279854179 >> v2:172.21.15.137:3300/0 conn(0x7f44a000e2a0 0x7f44a00d5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a5d9b640 1 -- 172.21.15.27:0/1279854179 >> v1:172.21.15.137:6789/0 conn(0x7f44a00db830 legacy=0x7f44a0005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f536dbdf640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f53680732f0 0x7f536815f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47536/0 (socket says 172.21.15.27:47536) 2024-04-20T10:55:49.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f536dbdf640 1 -- 172.21.15.27:0/4139374238 learned_addr learned my addr 172.21.15.27:0/4139374238 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f536d3de640 1 --2- 172.21.15.27:0/4139374238 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 0x7f53680ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f8a9da1c640 1 -- 172.21.15.27:0/3373475687 >> v2:172.21.15.137:3300/0 conn(0x7f8a900e6690 msgr2=0x7f8a900e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f8a9da1c640 1 --2- 172.21.15.27:0/3373475687 >> v2:172.21.15.137:3300/0 conn(0x7f8a900e6690 0x7f8a900e6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f8a9da1c640 1 -- 172.21.15.27:0/3373475687 >> v1:172.21.15.137:6789/0 conn(0x7f8a900dd460 legacy=0x7f8a9004b380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:55:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7f8a9da1c640 1 -- 172.21.15.27:0/3373475687 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a90007190 con 0x7f8a900062b0 2024-04-20T10:55:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f8a9da1c640 1 --2- 172.21.15.27:0/3373475687 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 0x7f8a900066d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a9802db60 tx=0x7f8a980a8290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db228193804b6067 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f536cbdd640 1 -- 172.21.15.27:0/4139374238 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992982727 0 0) 0x7f5368163080 con 0x7f53680c4850 2024-04-20T10:55:49.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7fe599fd8640 1 Processor -- start 2024-04-20T10:55:49.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.377+0000 7fe599fd8640 1 -- start start 2024-04-20T10:55:49.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7fe599fd8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 0x7fe58c04b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f536cbdd640 1 -- 172.21.15.27:0/4139374238 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5354003520 con 0x7f53680c4850 2024-04-20T10:55:49.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f662aee9640 1 Processor -- start 2024-04-20T10:55:49.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7feebce29640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7feeb00d4f80 0x7feeb0005e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:46632/0 (socket says 172.21.15.27:46632) 2024-04-20T10:55:49.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7feebce29640 1 -- 172.21.15.27:0/2884199683 learned_addr learned my addr 172.21.15.27:0/2884199683 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:49.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7feeb7fff640 1 --2- 172.21.15.27:0/2884199683 >> v2:172.21.15.137:3300/0 conn(0x7feeb000e290 0x7feeb004b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe598fd6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 0x7fe58c04b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe598fd6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 0x7fe58c04b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60718/0 (socket says 172.21.15.27:60718) 2024-04-20T10:55:49.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe598fd6640 1 -- 172.21.15.27:0/2449194392 learned_addr learned my addr 172.21.15.27:0/2449194392 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe599fd8640 1 --2- 172.21.15.27:0/2449194392 >> v2:172.21.15.137:3300/0 conn(0x7fe58c0e6760 0x7fe58c0e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7feeb6ffd640 1 -- 172.21.15.27:0/2884199683 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3808847063 0 0) 0x7feeac001110 con 0x7feeb00d4f80 2024-04-20T10:55:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7feeb6ffd640 1 -- 172.21.15.27:0/2884199683 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feea8003520 con 0x7feeb00d4f80 2024-04-20T10:55:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7feeb7fff640 1 -- 172.21.15.27:0/2884199683 >> v2:172.21.15.27:3300/0 conn(0x7feeb004b080 msgr2=0x7feeb00db820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f536dbdf640 1 -- 172.21.15.27:0/4139374238 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 msgr2=0x7f53680ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f536dbdf640 1 --2- 172.21.15.27:0/4139374238 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 0x7f53680ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f536dbdf640 1 -- 172.21.15.27:0/4139374238 >> v1:172.21.15.137:6789/0 conn(0x7f53680c4850 legacy=0x7f5368159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f536dbdf640 1 -- 172.21.15.27:0/4139374238 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5368163920 con 0x7f53680732f0 2024-04-20T10:55:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe5997d7640 1 --2- 172.21.15.27:0/2449194392 >> v2:172.21.15.137:3300/0 conn(0x7fe58c0e6760 0x7fe58c0e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe599fd8640 1 -- 172.21.15.27:0/2449194392 --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58c04b830 con 0x7fe58c0452c0 2024-04-20T10:55:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe599fd8640 1 -- 172.21.15.27:0/2449194392 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fe58c007640 con 0x7fe58c0e6760 2024-04-20T10:55:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe599fd8640 1 -- 172.21.15.27:0/2449194392 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c006a30 con 0x7fe58c0dd440 2024-04-20T10:55:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f662aee9640 1 -- start start 2024-04-20T10:55:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7feeb7fff640 1 --2- 172.21.15.27:0/2884199683 >> v2:172.21.15.27:3300/0 conn(0x7feeb004b080 0x7feeb00db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7feeb7fff640 1 -- 172.21.15.27:0/2884199683 >> v1:172.21.15.137:6789/0 conn(0x7feeb00d4f80 legacy=0x7feeb0005e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.378+0000 7f44a5d9b640 1 -- 172.21.15.27:0/1279854179 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a0046450 con 0x7f44a004b090 2024-04-20T10:55:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f71b488e640 1 -- 172.21.15.27:0/2747830800 >> v2:172.21.15.137:3300/0 conn(0x7f71a8046300 msgr2=0x7f71a804bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7f71b488e640 1 --2- 172.21.15.27:0/2747830800 >> v2:172.21.15.137:3300/0 conn(0x7f71a8046300 0x7f71a804bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7feeb7fff640 1 -- 172.21.15.27:0/2884199683 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb0046440 con 0x7feeb000e290 2024-04-20T10:55:49.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe598fd6640 1 -- 172.21.15.27:0/2449194392 >> v2:172.21.15.137:3300/0 conn(0x7fe58c0e6760 msgr2=0x7fe58c0e6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe598fd6640 1 --2- 172.21.15.27:0/2449194392 >> v2:172.21.15.137:3300/0 conn(0x7fe58c0e6760 0x7fe58c0e6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7f8a9d21b640 1 -- 172.21.15.27:0/3373475687 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a980b5070 con 0x7f8a900062b0 2024-04-20T10:55:49.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7f8a9d21b640 1 -- 172.21.15.27:0/3373475687 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a980b0640 con 0x7f8a900062b0 2024-04-20T10:55:49.396 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7f8a9d21b640 1 -- 172.21.15.27:0/3373475687 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a980b0ab0 con 0x7f8a900062b0 2024-04-20T10:55:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe598fd6640 1 -- 172.21.15.27:0/2449194392 >> v1:172.21.15.27:6789/0 conn(0x7fe58c0452c0 legacy=0x7fe58c0456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7fe5937fe640 1 -- 172.21.15.27:0/2449194392 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 115522774 0 0) 0x7fe58c04b830 con 0x7fe58c0452c0 2024-04-20T10:55:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7fe598fd6640 1 -- 172.21.15.27:0/2449194392 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c006d10 con 0x7fe58c0dd440 2024-04-20T10:55:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7f536d3de640 1 --2- 172.21.15.27:0/4139374238 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 0x7f53680ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:49.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7f71b488e640 1 -- 172.21.15.27:0/2747830800 >> v1:172.21.15.27:6789/0 conn(0x7f71a8047bf0 legacy=0x7f71a80470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7ff12a3df640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff11c0dd3c0 0x7ff11c046460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53432/0 (socket says 172.21.15.27:53432) 2024-04-20T10:55:49.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7f71af7fe640 1 -- 172.21.15.27:0/2747830800 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1783143083 0 0) 0x7f71a0003520 con 0x7f71a8047bf0 2024-04-20T10:55:49.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7f44a5d9b640 1 --2- 172.21.15.27:0/1279854179 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a004b620 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f44980097a0 tx=0x7f449802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f0e68ba99ebc2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7f536dbdf640 1 --2- 172.21.15.27:0/4139374238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53680732f0 0x7f536815f670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5364009870 tx=0x7f536402ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6494825911cc319 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f536cbdd640 1 -- 172.21.15.27:0/4139374238 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536402f870 con 0x7f53680732f0 2024-04-20T10:55:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f536cbdd640 1 -- 172.21.15.27:0/4139374238 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536402fa10 con 0x7f53680732f0 2024-04-20T10:55:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f536cbdd640 1 -- 172.21.15.27:0/4139374238 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536402fe20 con 0x7f53680732f0 2024-04-20T10:55:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7ff12a3df640 1 -- 172.21.15.27:0/2459568819 learned_addr learned my addr 172.21.15.27:0/2459568819 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.379+0000 7ff129bde640 1 --2- 172.21.15.27:0/2459568819 >> v2:172.21.15.137:3300/0 conn(0x7ff11c04ba60 0x7ff11c04be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7ff129bde640 1 -- 172.21.15.27:0/2459568819 >> v1:172.21.15.137:6789/0 conn(0x7ff11c0e3090 legacy=0x7ff11c0e34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7ff129bde640 1 -- 172.21.15.27:0/2459568819 >> v1:172.21.15.27:6789/0 conn(0x7ff11c0dd3c0 legacy=0x7ff11c046460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.380+0000 7ff129bde640 1 -- 172.21.15.27:0/2459568819 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11c0e3bd0 con 0x7ff11c04ba60 2024-04-20T10:55:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f662aee9640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f661c0d51f0 0x7f661c04b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f662aee9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c0db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f44a559a640 1 -- 172.21.15.27:0/1279854179 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498034070 con 0x7f44a004b090 2024-04-20T10:55:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f44a559a640 1 -- 172.21.15.27:0/1279854179 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4498030800 con 0x7f44a004b090 2024-04-20T10:55:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f44a559a640 1 -- 172.21.15.27:0/1279854179 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498034070 con 0x7f44a004b090 2024-04-20T10:55:49.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f66296e6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c0db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f8aa0481640 1 -- 172.21.15.27:0/3373475687 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 msgr2=0x7f8a900066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f8aa0481640 1 --2- 172.21.15.27:0/3373475687 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 0x7f8a900066d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a9802db60 tx=0x7f8a980a8290 comp rx=0 tx=0).stop 2024-04-20T10:55:49.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f66296e6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c0db820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60724/0 (socket says 172.21.15.27:60724) 2024-04-20T10:55:49.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f66296e6640 1 -- 172.21.15.27:0/2075170832 learned_addr learned my addr 172.21.15.27:0/2075170832 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7fe598fd6640 1 --2- 172.21.15.27:0/2449194392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 0x7fe58c04b2f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe588009870 tx=0x7fe58802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453a442653b87297 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f662a6e8640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f661c005e80 0x7f661c045320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53452/0 (socket says 172.21.15.27:53452) 2024-04-20T10:55:49.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7feeb7fff640 1 --2- 172.21.15.27:0/2884199683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feeb000e290 0x7feeb004b610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000b60 tx=0x7feeb80a8310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a0515607f6cf5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f662aee9640 1 -- 172.21.15.27:0/2075170832 --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f661c0dbd60 con 0x7f661c005e80 2024-04-20T10:55:49.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f6629ee7640 1 --2- 172.21.15.27:0/2075170832 >> v2:172.21.15.137:3300/0 conn(0x7f661c0d51f0 0x7f661c04b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f71b488e640 1 -- 172.21.15.27:0/2747830800 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a80e3eb0 con 0x7f71a80dd6c0 2024-04-20T10:55:49.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7ff129bde640 1 --2- 172.21.15.27:0/2459568819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff11c04ba60 0x7ff11c04be80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff124076850 tx=0x7ff1240a80d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232996757204042f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f8aa0481640 1 -- 172.21.15.27:0/3373475687 shutdown_connections 2024-04-20T10:55:49.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f8aa0481640 1 --2- 172.21.15.27:0/3373475687 >> v2:172.21.15.137:3300/0 conn(0x7f8a900e6690 0x7f8a900e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f8aa0481640 1 --2- 172.21.15.27:0/3373475687 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 0x7f8a900066d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f8aa0481640 1 -- 172.21.15.27:0/3373475687 >> 172.21.15.27:0/3373475687 conn(0x7f8a90050a00 msgr2=0x7f8a90050e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7fe5937fe640 1 -- 172.21.15.27:0/2449194392 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58802f830 con 0x7fe58c0dd440 2024-04-20T10:55:49.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7feeb6ffd640 1 -- 172.21.15.27:0/2884199683 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80b5070 con 0x7feeb000e290 2024-04-20T10:55:49.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7feeb6ffd640 1 -- 172.21.15.27:0/2884199683 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb80b0640 con 0x7feeb000e290 2024-04-20T10:55:49.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f536ebe1640 1 -- 172.21.15.27:0/4139374238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53680732f0 msgr2=0x7f536815f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f536ebe1640 1 --2- 172.21.15.27:0/4139374238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53680732f0 0x7f536815f670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5364009870 tx=0x7f536402ba50 comp rx=0 tx=0).stop 2024-04-20T10:55:49.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7f6628ee5640 1 -- 172.21.15.27:0/2075170832 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939846586 0 0) 0x7f6620001110 con 0x7f661c005e80 2024-04-20T10:55:49.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f662aee9640 1 -- 172.21.15.27:0/2075170832 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f661c045ec0 con 0x7f661c0d51f0 2024-04-20T10:55:49.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f662aee9640 1 -- 172.21.15.27:0/2075170832 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f661c046f20 con 0x7f661c04b080 2024-04-20T10:55:49.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f536ebe1640 1 -- 172.21.15.27:0/4139374238 shutdown_connections 2024-04-20T10:55:49.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f536ebe1640 1 --2- 172.21.15.27:0/4139374238 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 0x7f53680ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f536ebe1640 1 --2- 172.21.15.27:0/4139374238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53680732f0 0x7f536815f670 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f536ebe1640 1 -- 172.21.15.27:0/4139374238 >> 172.21.15.27:0/4139374238 conn(0x7f536806f990 msgr2=0x7f536806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7fe5937fe640 1 -- 172.21.15.27:0/2449194392 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58802f9d0 con 0x7fe58c0dd440 2024-04-20T10:55:49.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.381+0000 7fe5937fe640 1 -- 172.21.15.27:0/2449194392 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58802fda0 con 0x7fe58c0dd440 2024-04-20T10:55:49.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7feeb6ffd640 1 -- 172.21.15.27:0/2884199683 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80b0a30 con 0x7feeb000e290 2024-04-20T10:55:49.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f8aa0481640 1 -- 172.21.15.27:0/3373475687 shutdown_connections 2024-04-20T10:55:49.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f71b488e640 1 --2- 172.21.15.27:0/2747830800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71a80dd6c0 0x7f71a8007d70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71a800dfa0 tx=0x7f71a402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e047574f576e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f6628ee5640 1 -- 172.21.15.27:0/2075170832 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6610003520 con 0x7f661c005e80 2024-04-20T10:55:49.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7feebe88c640 1 -- 172.21.15.27:0/2884199683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feeb000e290 msgr2=0x7feeb004b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7feebe88c640 1 --2- 172.21.15.27:0/2884199683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feeb000e290 0x7feeb004b610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000b60 tx=0x7feeb80a8310 comp rx=0 tx=0).stop 2024-04-20T10:55:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7feebe88c640 1 -- 172.21.15.27:0/2884199683 shutdown_connections 2024-04-20T10:55:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7feebe88c640 1 --2- 172.21.15.27:0/2884199683 >> v2:172.21.15.27:3300/0 conn(0x7feeb004b080 0x7feeb00db820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7feebe88c640 1 --2- 172.21.15.27:0/2884199683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feeb000e290 0x7feeb004b610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7feebe88c640 1 -- 172.21.15.27:0/2884199683 >> 172.21.15.27:0/2884199683 conn(0x7feeb0050170 msgr2=0x7feeb0050580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f8aa0481640 1 -- 172.21.15.27:0/3373475687 wait complete. 2024-04-20T10:55:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f44a759e640 1 -- 172.21.15.27:0/1279854179 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 msgr2=0x7f44a004b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f44a759e640 1 --2- 172.21.15.27:0/1279854179 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a004b620 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f44980097a0 tx=0x7f449802b940 comp rx=0 tx=0).stop 2024-04-20T10:55:49.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f44a759e640 1 -- 172.21.15.27:0/1279854179 shutdown_connections 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f44a759e640 1 --2- 172.21.15.27:0/1279854179 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a004b620 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f44a759e640 1 --2- 172.21.15.27:0/1279854179 >> v2:172.21.15.137:3300/0 conn(0x7f44a000e2a0 0x7f44a00d5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f44a759e640 1 -- 172.21.15.27:0/1279854179 >> 172.21.15.27:0/1279854179 conn(0x7f44a0050180 msgr2=0x7f44a0050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7f44a759e640 1 -- 172.21.15.27:0/1279854179 shutdown_connections 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7f44a759e640 1 -- 172.21.15.27:0/1279854179 wait complete. 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7fe599fd8640 1 -- 172.21.15.27:0/2449194392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 msgr2=0x7fe58c04b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7fe599fd8640 1 --2- 172.21.15.27:0/2449194392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 0x7fe58c04b2f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe588009870 tx=0x7fe58802ba10 comp rx=0 tx=0).stop 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7fe599fd8640 1 -- 172.21.15.27:0/2449194392 shutdown_connections 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7fe599fd8640 1 --2- 172.21.15.27:0/2449194392 >> v2:172.21.15.137:3300/0 conn(0x7fe58c0e6760 0x7fe58c0e6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7fe599fd8640 1 --2- 172.21.15.27:0/2449194392 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 0x7fe58c04b2f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7fe599fd8640 1 -- 172.21.15.27:0/2449194392 >> 172.21.15.27:0/2449194392 conn(0x7fe58c050170 msgr2=0x7fe58c050580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7ff1293dd640 1 -- 172.21.15.27:0/2459568819 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b5070 con 0x7ff11c04ba60 2024-04-20T10:55:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7ff1293dd640 1 -- 172.21.15.27:0/2459568819 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1240b0440 con 0x7ff11c04ba60 2024-04-20T10:55:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7ff1293dd640 1 -- 172.21.15.27:0/2459568819 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b08f0 con 0x7ff11c04ba60 2024-04-20T10:55:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f536ebe1640 1 -- 172.21.15.27:0/4139374238 shutdown_connections 2024-04-20T10:55:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f536ebe1640 1 -- 172.21.15.27:0/4139374238 wait complete. 2024-04-20T10:55:49.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f66296e6640 1 -- 172.21.15.27:0/2075170832 >> v2:172.21.15.137:3300/0 conn(0x7f661c0d51f0 msgr2=0x7f661c04b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f66296e6640 1 --2- 172.21.15.27:0/2075170832 >> v2:172.21.15.137:3300/0 conn(0x7f661c0d51f0 0x7f661c04b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f66296e6640 1 -- 172.21.15.27:0/2075170832 >> v1:172.21.15.27:6789/0 conn(0x7f661c005e80 legacy=0x7f661c045320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f66296e6640 1 -- 172.21.15.27:0/2075170832 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c046160 con 0x7f661c04b080 2024-04-20T10:55:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f71af7fe640 1 -- 172.21.15.27:0/2747830800 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4037070 con 0x7f71a80dd6c0 2024-04-20T10:55:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.382+0000 7f71af7fe640 1 -- 172.21.15.27:0/2747830800 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a4030510 con 0x7f71a80dd6c0 2024-04-20T10:55:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7f71af7fe640 1 -- 172.21.15.27:0/2747830800 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4037070 con 0x7f71a80dd6c0 2024-04-20T10:55:49.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7ff12b3e1640 1 -- 172.21.15.27:0/2459568819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff11c04ba60 msgr2=0x7ff11c04be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7ff12b3e1640 1 --2- 172.21.15.27:0/2459568819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff11c04ba60 0x7ff11c04be80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff124076850 tx=0x7ff1240a80d0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7f66296e6640 1 --2- 172.21.15.27:0/2075170832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c0db820 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f662402db60 tx=0x7f66240aa6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7fe541a94914d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7f71b5890640 1 -- 172.21.15.27:0/2747830800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71a80dd6c0 msgr2=0x7f71a8007d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.383+0000 7f71b5890640 1 --2- 172.21.15.27:0/2747830800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71a80dd6c0 0x7f71a8007d70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71a800dfa0 tx=0x7f71a402b790 comp rx=0 tx=0).stop 2024-04-20T10:55:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f71b5890640 1 -- 172.21.15.27:0/2747830800 shutdown_connections 2024-04-20T10:55:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f71b5890640 1 --2- 172.21.15.27:0/2747830800 >> v2:172.21.15.137:3300/0 conn(0x7f71a8046300 0x7f71a804bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f71b5890640 1 --2- 172.21.15.27:0/2747830800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71a80dd6c0 0x7f71a8007d70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f71b5890640 1 -- 172.21.15.27:0/2747830800 >> 172.21.15.27:0/2747830800 conn(0x7f71a804dd30 msgr2=0x7f71a804e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f6628ee5640 1 -- 172.21.15.27:0/2075170832 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624072040 con 0x7f661c04b080 2024-04-20T10:55:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f6628ee5640 1 -- 172.21.15.27:0/2075170832 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66240af7d0 con 0x7f661c04b080 2024-04-20T10:55:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f71b5890640 1 -- 172.21.15.27:0/2747830800 shutdown_connections 2024-04-20T10:55:49.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7feebe88c640 1 -- 172.21.15.27:0/2884199683 shutdown_connections 2024-04-20T10:55:49.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f6628ee5640 1 -- 172.21.15.27:0/2075170832 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624072040 con 0x7f661c04b080 2024-04-20T10:55:49.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7feebe88c640 1 -- 172.21.15.27:0/2884199683 wait complete. 2024-04-20T10:55:49.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f8aa0481640 1 Processor -- start 2024-04-20T10:55:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f536ebe1640 1 Processor -- start 2024-04-20T10:55:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7fe599fd8640 1 -- 172.21.15.27:0/2449194392 shutdown_connections 2024-04-20T10:55:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f8aa0481640 1 -- start start 2024-04-20T10:55:49.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7fe599fd8640 1 -- 172.21.15.27:0/2449194392 wait complete. 2024-04-20T10:55:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f662aee9640 1 -- 172.21.15.27:0/2075170832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 msgr2=0x7f661c0db820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7ff12b3e1640 1 -- 172.21.15.27:0/2459568819 shutdown_connections 2024-04-20T10:55:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7ff12b3e1640 1 --2- 172.21.15.27:0/2459568819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff11c04ba60 0x7ff11c04be80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7ff12b3e1640 1 -- 172.21.15.27:0/2459568819 >> 172.21.15.27:0/2459568819 conn(0x7ff11c050180 msgr2=0x7ff11c050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f662aee9640 1 --2- 172.21.15.27:0/2075170832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c0db820 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f662402db60 tx=0x7f66240aa6d0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f71b5890640 1 -- 172.21.15.27:0/2747830800 wait complete. 2024-04-20T10:55:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f44a759e640 1 Processor -- start 2024-04-20T10:55:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f662aee9640 1 -- 172.21.15.27:0/2075170832 shutdown_connections 2024-04-20T10:55:49.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f662aee9640 1 --2- 172.21.15.27:0/2075170832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c0db820 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f662aee9640 1 --2- 172.21.15.27:0/2075170832 >> v2:172.21.15.137:3300/0 conn(0x7f661c0d51f0 0x7f661c04b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f662aee9640 1 -- 172.21.15.27:0/2075170832 >> 172.21.15.27:0/2075170832 conn(0x7f661c050170 msgr2=0x7f661c050580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f8aa0481640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 0x7f8a90185a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f8aa0481640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a900dd460 0x7f8a9004a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f8aa0481640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a900e6690 0x7f8a900de040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f8aa0481640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90007750 con 0x7f8a900dd460 2024-04-20T10:55:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f8aa0481640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90007da0 con 0x7f8a900e6690 2024-04-20T10:55:49.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f8aa0481640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8a90007a30 con 0x7f8a900062b0 2024-04-20T10:55:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.384+0000 7f536ebe1640 1 -- start start 2024-04-20T10:55:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f536ebe1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53680732f0 0x7f53681586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7feebe88c640 1 Processor -- start 2024-04-20T10:55:49.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f536ebe1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 0x7f53681f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f662aee9640 1 -- 172.21.15.27:0/2075170832 shutdown_connections 2024-04-20T10:55:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f71b5890640 1 Processor -- start 2024-04-20T10:55:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f536dbdf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53680732f0 0x7f53681586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f536dbdf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53680732f0 0x7f53681586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48564/0 (socket says 172.21.15.27:48564) 2024-04-20T10:55:49.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.385+0000 7f662aee9640 1 -- 172.21.15.27:0/2075170832 wait complete. 2024-04-20T10:55:49.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f71b5890640 1 -- start start 2024-04-20T10:55:49.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7feebe88c640 1 -- start start 2024-04-20T10:55:49.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f8a9da1c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a900dd460 0x7f8a9004a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f536dbdf640 1 -- 172.21.15.27:0/1709528732 learned_addr learned my addr 172.21.15.27:0/1709528732 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f536ebe1640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53680c4850 0x7f53681503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7ff12b3e1640 1 -- 172.21.15.27:0/2459568819 shutdown_connections 2024-04-20T10:55:49.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5368162de0 con 0x7f53680732f0 2024-04-20T10:55:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f8a9da1c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a900dd460 0x7f8a9004a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48562/0 (socket says 172.21.15.27:48562) 2024-04-20T10:55:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f8a9da1c640 1 -- 172.21.15.27:0/1893082912 learned_addr learned my addr 172.21.15.27:0/1893082912 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f8a9ea1e640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a900e6690 0x7f8a900de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7fe599fd8640 1 Processor -- start 2024-04-20T10:55:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f536d3de640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 0x7f53681f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f536e3e0640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53680c4850 0x7f53681503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5368162c60 con 0x7f53680c4850 2024-04-20T10:55:49.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f53680ca9d0 con 0x7f53680736d0 2024-04-20T10:55:49.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f8a9e21d640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 0x7f8a90185a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7fe599fd8640 1 -- start start 2024-04-20T10:55:49.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f8a9ea1e640 1 -- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 msgr2=0x7f8a90185a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f8a9ea1e640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a900062b0 0x7f8a90185a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f8a9ea1e640 1 -- 172.21.15.27:0/1893082912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a900dd460 msgr2=0x7f8a9004a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f536d3de640 1 -- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53680c4850 msgr2=0x7f53681503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f536d3de640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53680c4850 0x7f53681503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f536d3de640 1 -- 172.21.15.27:0/1709528732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53680732f0 msgr2=0x7f53681586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f536d3de640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53680732f0 0x7f53681586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f536d3de640 1 -- 172.21.15.27:0/1709528732 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53681f9120 con 0x7f53680736d0 2024-04-20T10:55:49.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f536dbdf640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53680732f0 0x7f53681586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feebe88c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7feeb000e290 0x7feeb00bc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feebe88c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feeb004b080 0x7feeb0035c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feebe88c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00d4f80 0x7feeb004f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feebe88c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feeb0006590 con 0x7feeb00d4f80 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feebe88c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feeb0046a00 con 0x7feeb004b080 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feebe88c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7feeb00d53d0 con 0x7feeb000e290 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feeb7fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7feeb000e290 0x7feeb00bc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feeb7fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7feeb000e290 0x7feeb00bc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60754/0 (socket says 172.21.15.27:60754) 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feeb7fff640 1 -- 172.21.15.27:0/2607895053 learned_addr learned my addr 172.21.15.27:0/2607895053 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feebce29640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00d4f80 0x7feeb004f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f71b5890640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8046300 0x7f71a8048630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f71b5890640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f71a8047bf0 0x7f71a80e4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feeb77fe640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feeb004b080 0x7feeb0035c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7feebce29640 1 -- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7feeb000e290 msgr2=0x7feeb00bc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7feebce29640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7feeb000e290 0x7feeb00bc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7feebce29640 1 -- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feeb004b080 msgr2=0x7feeb0035c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7feebce29640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feeb004b080 0x7feeb0035c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7feebce29640 1 -- 172.21.15.27:0/2607895053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb00d9930 con 0x7feeb00d4f80 2024-04-20T10:55:49.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f71b488e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8046300 0x7f71a8048630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f71b488e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8046300 0x7f71a8048630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47612/0 (socket says 172.21.15.27:47612) 2024-04-20T10:55:49.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7ff12b3e1640 1 -- 172.21.15.27:0/2459568819 wait complete. 2024-04-20T10:55:49.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7ff12b3e1640 1 Processor -- start 2024-04-20T10:55:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f71b488e640 1 -- 172.21.15.27:0/892027529 learned_addr learned my addr 172.21.15.27:0/892027529 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7feeb7fff640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7feeb000e290 0x7feeb00bc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f71b5890640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71a80dd6c0 0x7f71a80e4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7feeb77fe640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feeb004b080 0x7feeb0035c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:49.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe599fd8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe58c0452c0 0x7fe58c0c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7ff12b3e1640 1 -- start start 2024-04-20T10:55:49.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.386+0000 7f44a759e640 1 -- start start 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f44a759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44a000e2a0 0x7f44a00c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f44a759e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a00cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f44a759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a00db830 0x7f44a00cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f44a759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f44a00065a0 con 0x7f44a000e2a0 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f44a759e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0046a10 con 0x7f44a00db830 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f44a759e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f44a00dbcc0 con 0x7f44a004b090 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f44a5d9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a00cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f44a5d9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a00cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60768/0 (socket says 172.21.15.27:60768) 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.387+0000 7f44a5d9b640 1 -- 172.21.15.27:0/4191619435 learned_addr learned my addr 172.21.15.27:0/4191619435 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f71b508f640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71a80dd6c0 0x7f71a80e4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe599fd8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 0x7fe58c0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f8a9ea1e640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a900dd460 0x7f8a9004a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f8a9ea1e640 1 -- 172.21.15.27:0/1893082912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a900de780 con 0x7f8a900e6690 2024-04-20T10:55:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe598fd6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe58c0452c0 0x7fe58c0c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe598fd6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe58c0452c0 0x7fe58c0c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47614/0 (socket says 172.21.15.27:47614) 2024-04-20T10:55:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f662aee9640 1 Processor -- start 2024-04-20T10:55:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe598fd6640 1 -- 172.21.15.27:0/2425314965 learned_addr learned my addr 172.21.15.27:0/2425314965 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f662aee9640 1 -- start start 2024-04-20T10:55:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f44a5d9b640 1 -- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a00db830 msgr2=0x7f44a00cdd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:55:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f44a5d9b640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44a00db830 0x7f44a00cdd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f44a5d9b640 1 -- 172.21.15.27:0/4191619435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44a000e2a0 msgr2=0x7f44a00c7f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:55:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f44a5d9b640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44a000e2a0 0x7f44a00c7f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7feebce29640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00d4f80 0x7feeb004f8e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c3c0 tx=0x7feeac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f44a5d9b640 1 -- 172.21.15.27:0/4191619435 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a00473c0 con 0x7f44a004b090 2024-04-20T10:55:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f536d3de640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 0x7f53681f2f80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f535800e7e0 tx=0x7f535800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe599fd8640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe58c0e6760 0x7fe58c0d3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe599fd8640 1 -- 172.21.15.27:0/2425314965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0dd820 con 0x7fe58c0e6760 2024-04-20T10:55:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe593fff640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 0x7fe58c0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe599fd8640 1 -- 172.21.15.27:0/2425314965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0072d0 con 0x7fe58c0452c0 2024-04-20T10:55:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe599fd8640 1 -- 172.21.15.27:0/2425314965 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c04c6f0 con 0x7fe58c0dd440 2024-04-20T10:55:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f8a9da1c640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a900dd460 0x7f8a9004a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f535e7fc640 1 -- 172.21.15.27:0/1709528732 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535800f040 con 0x7f53680736d0 2024-04-20T10:55:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7feeb4ff9640 1 -- 172.21.15.27:0/2607895053 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac017070 con 0x7feeb00d4f80 2024-04-20T10:55:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7fe5997d7640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe58c0e6760 0x7fe58c0d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f44a5d9b640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a00cd850 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f44980098c0 tx=0x7f4498004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7feeb4ff9640 1 -- 172.21.15.27:0/2607895053 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeac005ce0 con 0x7feeb00d4f80 2024-04-20T10:55:49.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f8a9ea1e640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a900e6690 0x7f8a900de040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e7f0 tx=0x7f8a9400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7fe598fd6640 1 -- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 msgr2=0x7fe58c0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7fe598fd6640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe58c0dd440 0x7fe58c0d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7fe598fd6640 1 -- 172.21.15.27:0/2425314965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe58c0e6760 msgr2=0x7fe58c0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7fe598fd6640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe58c0e6760 0x7fe58c0d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7fe598fd6640 1 -- 172.21.15.27:0/2425314965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c0c4ab0 con 0x7fe58c0452c0 2024-04-20T10:55:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f449effd640 1 -- 172.21.15.27:0/4191619435 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498048070 con 0x7f44a004b090 2024-04-20T10:55:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f535e7fc640 1 -- 172.21.15.27:0/1709528732 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5358004030 con 0x7f53680736d0 2024-04-20T10:55:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5368157be0 con 0x7f53680736d0 2024-04-20T10:55:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f535e7fc640 1 -- 172.21.15.27:0/1709528732 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535800f040 con 0x7f53680736d0 2024-04-20T10:55:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f449effd640 1 -- 172.21.15.27:0/4191619435 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4498030e30 con 0x7f44a004b090 2024-04-20T10:55:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f8a8effd640 1 -- 172.21.15.27:0/1893082912 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94017070 con 0x7f8a900e6690 2024-04-20T10:55:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f8a8effd640 1 -- 172.21.15.27:0/1893082912 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9400c730 con 0x7f8a900e6690 2024-04-20T10:55:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f44a759e640 1 -- 172.21.15.27:0/4191619435 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a00cbf20 con 0x7f44a004b090 2024-04-20T10:55:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f8a8effd640 1 -- 172.21.15.27:0/1893082912 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400cb20 con 0x7f8a900e6690 2024-04-20T10:55:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a900dea40 con 0x7f8a900e6690 2024-04-20T10:55:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff12b3e1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff11c04ba60 0x7ff11c185790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f449effd640 1 -- 172.21.15.27:0/4191619435 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498034070 con 0x7f44a004b090 2024-04-20T10:55:49.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff12b3e1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff11c0dd3c0 0x7ff11c04e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f53680c51b0 con 0x7f53680736d0 2024-04-20T10:55:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7fe598fd6640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe58c0452c0 0x7fe58c0c4570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe58802c020 tx=0x7fe5880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff12b3e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff11c0e3090 0x7ff11c1823b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff12b3e1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c0456a0 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff12b3e1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff11c045fd0 con 0x7ff11c0e3090 2024-04-20T10:55:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f662aee9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f661c005e80 0x7f661c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f662aee9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c1707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a90185220 con 0x7f8a900e6690 2024-04-20T10:55:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff12b3e1640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff11c045980 con 0x7ff11c04ba60 2024-04-20T10:55:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff129bde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff11c0dd3c0 0x7ff11c04e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff12a3df640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff11c04ba60 0x7ff11c185790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f662aee9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f661c0d51f0 0x7f661c170d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f662aee9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f661c046a00 con 0x7f661c005e80 2024-04-20T10:55:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f662aee9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f661c046720 con 0x7f661c0d51f0 2024-04-20T10:55:49.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f662aee9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f661c045a30 con 0x7f661c04b080 2024-04-20T10:55:49.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff129bde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff11c0dd3c0 0x7ff11c04e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48620/0 (socket says 172.21.15.27:48620) 2024-04-20T10:55:49.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff129bde640 1 -- 172.21.15.27:0/779274262 learned_addr learned my addr 172.21.15.27:0/779274262 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7feebe88c640 1 -- 172.21.15.27:0/2607895053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb00361d0 con 0x7feeb00d4f80 2024-04-20T10:55:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7ff12a3df640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff11c04ba60 0x7ff11c185790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60788/0 (socket says 172.21.15.27:60788) 2024-04-20T10:55:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7ff12abe0640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff11c0e3090 0x7ff11c1823b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7ff129bde640 1 -- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff11c04ba60 msgr2=0x7ff11c185790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7ff129bde640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff11c04ba60 0x7ff11c185790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7ff129bde640 1 -- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff11c0e3090 msgr2=0x7ff11c1823b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7ff129bde640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff11c0e3090 0x7ff11c1823b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7fe5917fa640 1 -- 172.21.15.27:0/2425314965 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588047070 con 0x7fe58c0452c0 2024-04-20T10:55:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7fe5917fa640 1 -- 172.21.15.27:0/2425314965 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe588042460 con 0x7fe58c0452c0 2024-04-20T10:55:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7fe5917fa640 1 -- 172.21.15.27:0/2425314965 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588042850 con 0x7fe58c0452c0 2024-04-20T10:55:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7ff129bde640 1 -- 172.21.15.27:0/779274262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11c0476a0 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53681f9400 con 0x7f53680736d0 2024-04-20T10:55:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.391+0000 7ff129bde640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff11c0dd3c0 0x7ff11c04e210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff12406adb0 tx=0x7ff12406a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7fe599fd8640 1 -- 172.21.15.27:0/2425314965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c035070 con 0x7fe58c0452c0 2024-04-20T10:55:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7fe599fd8640 1 -- 172.21.15.27:0/2425314965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c035630 con 0x7fe58c0452c0 2024-04-20T10:55:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.391+0000 7f535e7fc640 1 -- 172.21.15.27:0/1709528732 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53580041e0 con 0x7f53680736d0 2024-04-20T10:55:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.391+0000 7f449effd640 1 -- 172.21.15.27:0/4191619435 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f449804b070 con 0x7f44a004b090 2024-04-20T10:55:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.391+0000 7f44a759e640 1 -- 172.21.15.27:0/4191619435 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a00481a0 con 0x7f44a004b090 2024-04-20T10:55:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7f535e7fc640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f534c06cb20 0x7f534c06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7fe599fd8640 1 -- 172.21.15.27:0/2425314965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe574001030 con 0x7fe58c0452c0 2024-04-20T10:55:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7f535e7fc640 1 -- 172.21.15.27:0/1709528732 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f535808b7e0 con 0x7f53680736d0 2024-04-20T10:55:49.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7f535e7fc640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f534c074cc0 0x7f534c0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7f449effd640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f449006c970 0x7f449006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7f535e7fc640 1 -- 172.21.15.27:0/1709528732 --> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f534c077790 con 0x7f534c074cc0 2024-04-20T10:55:49.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7f8a8effd640 1 -- 172.21.15.27:0/1893082912 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a9401a070 con 0x7f8a900e6690 2024-04-20T10:55:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.391+0000 7ff11affd640 1 -- 172.21.15.27:0/779274262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b5050 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.391+0000 7ff11affd640 1 -- 172.21.15.27:0/779274262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1240b6040 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f536e3e0640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f534c074cc0 0x7f534c0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.391+0000 7ff11affd640 1 -- 172.21.15.27:0/779274262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240afc40 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11c047980 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11c181e90 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff11c005390 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f71a80e4190 con 0x7f71a80dd6c0 2024-04-20T10:55:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8007570 con 0x7f71a8046300 2024-04-20T10:55:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.389+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f71a80477c0 con 0x7f71a8047bf0 2024-04-20T10:55:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7fe5997d7640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe58c0e6760 0x7fe58c0d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.388+0000 7f71affff640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f71a8047bf0 0x7f71a80e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f8a8effd640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8a7c06c9f0 0x7f8a7c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7fe5917fa640 1 -- 172.21.15.27:0/2425314965 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe58804a070 con 0x7fe58c0452c0 2024-04-20T10:55:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f71affff640 1 -- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8046300 msgr2=0x7f71a8048630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f8a8effd640 1 -- 172.21.15.27:0/1893082912 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a9408b500 con 0x7f8a900e6690 2024-04-20T10:55:49.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f71affff640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8046300 0x7f71a8048630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f71affff640 1 -- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71a80dd6c0 msgr2=0x7f71a80e4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f71affff640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71a80dd6c0 0x7f71a80e4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f536dbdf640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f534c06cb20 0x7f534c06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f71affff640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a804b070 con 0x7f71a8047bf0 2024-04-20T10:55:49.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f8a9e21d640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8a7c06c9f0 0x7f8a7c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f71b488e640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8046300 0x7f71a8048630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:49.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7feeb4ff9640 1 -- 172.21.15.27:0/2607895053 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0072f0 con 0x7feeb00d4f80 2024-04-20T10:55:49.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.390+0000 7feebe88c640 1 -- 172.21.15.27:0/2607895053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb0034860 con 0x7feeb00d4f80 2024-04-20T10:55:49.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.391+0000 7feebe88c640 1 -- 172.21.15.27:0/2607895053 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feeb0005380 con 0x7feeb00d4f80 2024-04-20T10:55:49.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7feeb4ff9640 1 -- 172.21.15.27:0/2607895053 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feeac01a070 con 0x7feeb00d4f80 2024-04-20T10:55:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7feeb4ff9640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee9c06c9c0 0x7fee9c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.392+0000 7feeb4ff9640 1 -- 172.21.15.27:0/2607895053 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feeac08adc0 con 0x7feeb00d4f80 2024-04-20T10:55:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7feeb4ff9640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7fee9c074bc0 0x7fee9c076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7feeb4ff9640 1 -- 172.21.15.27:0/2607895053 --> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feeac009e50 con 0x7fee9c074bc0 2024-04-20T10:55:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7feeb4ff9640 1 -- 172.21.15.27:0/2607895053 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7feeac054e20 con 0x7feeb00d4f80 2024-04-20T10:55:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7feeb77fe640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7fee9c074bc0 0x7fee9c076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7feeb77fe640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7fee9c074bc0 0x7fee9c076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7feeb7fff640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee9c06c9c0 0x7fee9c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7fe5917fa640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe57806c940 0x7fe57806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f6629ee7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f661c005e80 0x7f661c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f6629ee7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f661c005e80 0x7f661c0caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48634/0 (socket says 172.21.15.27:48634) 2024-04-20T10:55:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f6629ee7640 1 -- 172.21.15.27:0/993239369 learned_addr learned my addr 172.21.15.27:0/993239369 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.393+0000 7f44a759e640 1 -- 172.21.15.27:0/4191619435 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44a0005390 con 0x7f44a004b090 2024-04-20T10:55:49.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7fe593fff640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe57806c940 0x7fe57806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f44a659c640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f449006c970 0x7f449006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f66296e6640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c1707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7fe5917fa640 1 -- 172.21.15.27:0/2425314965 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe5880ae2c0 con 0x7fe58c0452c0 2024-04-20T10:55:49.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f71b508f640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71a80dd6c0 0x7f71a80e4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f449effd640 1 -- 172.21.15.27:0/4191619435 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4498079b60 con 0x7f44a004b090 2024-04-20T10:55:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7feeb4ff9640 1 -- 172.21.15.27:0/2607895053 <== osd.1 v2:172.21.15.27:6800/600824511 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feeac009e50 con 0x7fee9c074bc0 2024-04-20T10:55:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f8a9e21d640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8a7c06c9f0 0x7f8a7c06eeb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009870 tx=0x7f8a88008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f449effd640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7f4490074b10 0x7f4490076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f449effd640 1 -- 172.21.15.27:0/4191619435 --> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f449802ee50 con 0x7f4490074b10 2024-04-20T10:55:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7fe5917fa640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7fe578074b60 0x7fe578076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7fe593fff640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe57806c940 0x7fe57806ee00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0dada0 tx=0x7fe580009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7ff11affd640 1 -- 172.21.15.27:0/779274262 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1240c3030 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7fe5917fa640 1 -- 172.21.15.27:0/2425314965 --> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe588006810 con 0x7fe578074b60 2024-04-20T10:55:49.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7fe5917fa640 1 -- 172.21.15.27:0/2425314965 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe5880ae7a0 con 0x7fe58c0452c0 2024-04-20T10:55:49.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7feeb7fff640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee9c06c9c0 0x7fee9c06ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feeb806b720 tx=0x7feeb80ac000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f449effd640 1 -- 172.21.15.27:0/4191619435 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f44980b3060 con 0x7f44a004b090 2024-04-20T10:55:49.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f44a659c640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f449006c970 0x7f449006ee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4494009930 tx=0x7f4494009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f44a6d9d640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7f4490074b10 0x7f4490076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f71affff640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f71a8047bf0 0x7f71a80e4680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f719c00d550 tx=0x7f719c00da10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f535e7fc640 1 -- 172.21.15.27:0/1709528732 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f53580557e0 con 0x7f53680736d0 2024-04-20T10:55:49.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.394+0000 7f662a6e8640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f661c0d51f0 0x7f661c170d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f8aa0481640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f8a90185460 0x7f8a900dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7ff11affd640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff10806c9c0 0x7ff10806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 --> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a900ded20 con 0x7f8a90185460 2024-04-20T10:55:49.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f536e3e0640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f534c074cc0 0x7f534c0770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7fe5997d7640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7fe578074b60 0x7fe578076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7ff11affd640 1 -- 172.21.15.27:0/779274262 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff12412afa0 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f6629ee7640 1 -- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 msgr2=0x7f661c1707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f6629ee7640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c1707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f6629ee7640 1 -- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f661c0d51f0 msgr2=0x7f661c170d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f8a9da1c640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f8a90185460 0x7f8a900dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f6629ee7640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f661c0d51f0 0x7f661c170d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7ff11affd640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7ff108074c40 0x7ff108077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7ff11affd640 1 -- 172.21.15.27:0/779274262 --> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff12406ee80 con 0x7ff108074c40 2024-04-20T10:55:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7ff12a3df640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff10806c9c0 0x7ff10806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7ff11affd640 1 -- 172.21.15.27:0/779274262 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff12412b390 con 0x7ff11c0dd3c0 2024-04-20T10:55:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f6629ee7640 1 -- 172.21.15.27:0/993239369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c173d20 con 0x7f661c005e80 2024-04-20T10:55:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c0125f0 con 0x7f71a8047bf0 2024-04-20T10:55:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a80d94f0 con 0x7f71a8047bf0 2024-04-20T10:55:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.396+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f719c019070 con 0x7f71a8047bf0 2024-04-20T10:55:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.396+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a80e5e90 con 0x7f71a8047bf0 2024-04-20T10:55:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f44a6d9d640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7f4490074b10 0x7f4490076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f536dbdf640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f534c06cb20 0x7f534c06efe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5364009870 tx=0x7f5364008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.396+0000 7ff12abe0640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7ff108074c40 0x7ff108077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.395+0000 7f66296e6640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f661c04b080 0x7f661c1707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.396+0000 7f8a9da1c640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f8a90185460 0x7f8a900dc3c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.396+0000 7f6629ee7640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f661c005e80 0x7f661c0caba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f661400d800 tx=0x7f661400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.396+0000 7ff12a3df640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff10806c9c0 0x7ff10806ee80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff120000f60 tx=0x7ff120007e00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.396+0000 7ff12abe0640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7ff108074c40 0x7ff108077060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.396+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c0129d0 con 0x7f71a8047bf0 2024-04-20T10:55:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.396+0000 7f535e7fc640 1 -- 172.21.15.27:0/1709528732 <== osd.4 v2:172.21.15.137:6800/3079865776 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f534c077790 con 0x7f534c074cc0 2024-04-20T10:55:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7f661a7fc640 1 -- 172.21.15.27:0/993239369 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661400b6c0 con 0x7f661c005e80 2024-04-20T10:55:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7f449effd640 1 -- 172.21.15.27:0/4191619435 <== osd.3 v2:172.21.15.27:6820/574866103 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f449802ee50 con 0x7f4490074b10 2024-04-20T10:55:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71a8037630 con 0x7f71a8047bf0 2024-04-20T10:55:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7f661a7fc640 1 -- 172.21.15.27:0/993239369 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6614019070 con 0x7f661c005e80 2024-04-20T10:55:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7f661a7fc640 1 -- 172.21.15.27:0/993239369 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661400ba80 con 0x7f661c005e80 2024-04-20T10:55:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c174000 con 0x7f661c005e80 2024-04-20T10:55:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7f8a8effd640 1 -- 172.21.15.27:0/1893082912 <== osd.2 v2:172.21.15.27:6816/3544986990 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a900ded20 con 0x7f8a90185460 2024-04-20T10:55:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7ff11affd640 1 -- 172.21.15.27:0/779274262 <== osd.6 v2:172.21.15.137:6808/1373372098 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff12406ee80 con 0x7ff108074c40 2024-04-20T10:55:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c1745c0 con 0x7f661c005e80 2024-04-20T10:55:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.398+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f719c012b70 con 0x7f71a8047bf0 2024-04-20T10:55:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.399+0000 7f71ad7fa640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f719406c9c0 0x7f719406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.399+0000 7f71b488e640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f719406c9c0 0x7f719406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.399+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f719c08bef0 con 0x7f71a8047bf0 2024-04-20T10:55:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.400+0000 7f661a7fc640 1 -- 172.21.15.27:0/993239369 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f661400bc20 con 0x7f661c005e80 2024-04-20T10:55:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.400+0000 7f71b488e640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f719406c9c0 0x7f719406ee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71a402b7d0 tx=0x7f71a402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.400+0000 7f71ad7fa640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194074be0 0x7f7194077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.400+0000 7f71b508f640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194074be0 0x7f7194077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.400+0000 7f661a7fc640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f660806c970 0x7f660806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.400+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71940776b0 con 0x7f7194074be0 2024-04-20T10:55:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.400+0000 7f66296e6640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f660806c970 0x7f660806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.400+0000 7f661a7fc640 1 -- 172.21.15.27:0/993239369 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6614057d00 con 0x7f661c005e80 2024-04-20T10:55:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.401+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f719c08e550 con 0x7f71a8047bf0 2024-04-20T10:55:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.401+0000 7f71b508f640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194074be0 0x7f7194077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.401+0000 7f66296e6640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f660806c970 0x7f660806ee30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f661c0470a0 tx=0x7f66240af000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.402+0000 7f662aee9640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7f661c0c7510 0x7f661c003180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.402+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 --> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f661c005380 con 0x7f661c0c7510 2024-04-20T10:55:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.397+0000 7fe5997d7640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7fe578074b60 0x7fe578076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.398+0000 7fe5917fa640 1 -- 172.21.15.27:0/2425314965 <== osd.5 v2:172.21.15.137:6824/673751184 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe588006810 con 0x7fe578074b60 2024-04-20T10:55:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.402+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 <== osd.0 v2:172.21.15.27:6808/2513808316 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71940776b0 con 0x7f7194074be0 2024-04-20T10:55:49.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.402+0000 7f662a6e8640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7f661c0c7510 0x7f661c003180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.403+0000 7f662a6e8640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7f661c0c7510 0x7f661c003180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.404+0000 7f661a7fc640 1 -- 172.21.15.27:0/993239369 <== osd.7 v2:172.21.15.137:6816/460941503 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f661c005380 con 0x7f661c0c7510 2024-04-20T10:55:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.427+0000 7feebe88c640 1 -- 172.21.15.27:0/2607895053 --> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feeb004bc50 con 0x7fee9c074bc0 2024-04-20T10:55:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.428+0000 7feeb4ff9640 1 -- 172.21.15.27:0/2607895053 <== osd.1 v2:172.21.15.27:6800/600824511 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feeb004bc50 con 0x7fee9c074bc0 2024-04-20T10:55:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.429+0000 7feea27fc640 1 -- 172.21.15.27:0/2607895053 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7fee9c074bc0 msgr2=0x7fee9c076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.429+0000 7feea27fc640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7fee9c074bc0 0x7fee9c076fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.429+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 --> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff11c182090 con 0x7ff108074c40 2024-04-20T10:55:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.429+0000 7feea27fc640 1 -- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee9c06c9c0 msgr2=0x7fee9c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.429+0000 7feea27fc640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee9c06c9c0 0x7fee9c06ee80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feeb806b720 tx=0x7feeb80ac000 comp rx=0 tx=0).stop 2024-04-20T10:55:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.429+0000 7feea27fc640 1 -- 172.21.15.27:0/2607895053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00d4f80 msgr2=0x7feeb004f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.429+0000 7feea27fc640 1 --2- 172.21.15.27:0/2607895053 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00d4f80 0x7feeb004f8e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c3c0 tx=0x7feeac00c880 comp rx=0 tx=0).stop 2024-04-20T10:55:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.430+0000 7feebce29640 1 -- 172.21.15.27:0/2607895053 reap_dead start 2024-04-20T10:55:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.430+0000 7feea27fc640 1 -- 172.21.15.27:0/2607895053 shutdown_connections 2024-04-20T10:55:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.430+0000 7feea27fc640 1 -- 172.21.15.27:0/2607895053 >> 172.21.15.27:0/2607895053 conn(0x7feeb0050170 msgr2=0x7feeb0045110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.430+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 --> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5368076610 con 0x7f534c074cc0 2024-04-20T10:55:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.430+0000 7ff11affd640 1 -- 172.21.15.27:0/779274262 <== osd.6 v2:172.21.15.137:6808/1373372098 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff11c182090 con 0x7ff108074c40 2024-04-20T10:55:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.430+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7ff108074c40 msgr2=0x7ff108077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.430+0000 7ff12b3e1640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7ff108074c40 0x7ff108077060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.430+0000 7feea27fc640 1 -- 172.21.15.27:0/2607895053 shutdown_connections 2024-04-20T10:55:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.430+0000 7feea27fc640 1 -- 172.21.15.27:0/2607895053 wait complete. 2024-04-20T10:55:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7f535e7fc640 1 -- 172.21.15.27:0/1709528732 <== osd.4 v2:172.21.15.137:6800/3079865776 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5368076610 con 0x7f534c074cc0 2024-04-20T10:55:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff10806c9c0 msgr2=0x7ff10806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7ff12b3e1640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff10806c9c0 0x7ff10806ee80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff120000f60 tx=0x7ff120007e00 comp rx=0 tx=0).stop 2024-04-20T10:55:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff11c0dd3c0 msgr2=0x7ff11c04e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7ff12b3e1640 1 --2- 172.21.15.27:0/779274262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff11c0dd3c0 0x7ff11c04e210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff12406adb0 tx=0x7ff12406a0b0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7ff12abe0640 1 -- 172.21.15.27:0/779274262 reap_dead start 2024-04-20T10:55:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7f44a759e640 1 -- 172.21.15.27:0/4191619435 --> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f44a0005840 con 0x7f4490074b10 2024-04-20T10:55:49.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 --> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a90181d30 con 0x7f8a90185460 2024-04-20T10:55:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 shutdown_connections 2024-04-20T10:55:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 >> 172.21.15.27:0/779274262 conn(0x7ff11c050180 msgr2=0x7ff11c006650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f534c074cc0 msgr2=0x7f534c0770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.431+0000 7f536ebe1640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f534c074cc0 0x7f534c0770e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f534c06cb20 msgr2=0x7f534c06efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7f536ebe1640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f534c06cb20 0x7f534c06efe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5364009870 tx=0x7f5364008030 comp rx=0 tx=0).stop 2024-04-20T10:55:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7f449effd640 1 -- 172.21.15.27:0/4191619435 <== osd.3 v2:172.21.15.27:6820/574866103 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f44a0005840 con 0x7f4490074b10 2024-04-20T10:55:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 msgr2=0x7f53681f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7f536ebe1640 1 --2- 172.21.15.27:0/1709528732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f53680736d0 0x7f53681f2f80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f535800e7e0 tx=0x7f535800eca0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7f536e3e0640 1 -- 172.21.15.27:0/1709528732 reap_dead start 2024-04-20T10:55:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 shutdown_connections 2024-04-20T10:55:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7fe599fd8640 1 -- 172.21.15.27:0/2425314965 --> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe574002dd0 con 0x7fe578074b60 2024-04-20T10:55:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7f8a8effd640 1 -- 172.21.15.27:0/1893082912 <== osd.2 v2:172.21.15.27:6816/3544986990 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8a980b6040 con 0x7f8a90185460 2024-04-20T10:55:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7ff12b3e1640 1 -- 172.21.15.27:0/779274262 wait complete. 2024-04-20T10:55:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 shutdown_connections 2024-04-20T10:55:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7fe5917fa640 1 -- 172.21.15.27:0/2425314965 <== osd.5 v2:172.21.15.137:6824/673751184 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe574002dd0 con 0x7fe578074b60 2024-04-20T10:55:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.432+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 >> 172.21.15.27:0/1709528732 conn(0x7f536806f990 msgr2=0x7f536806c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f8a90185460 msgr2=0x7f8a900dc3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f8aa0481640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f8a90185460 0x7f8a900dc3c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 -- 172.21.15.27:0/4191619435 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7f4490074b10 msgr2=0x7f4490076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7f4490074b10 0x7f4490076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7fe57effd640 1 -- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7fe578074b60 msgr2=0x7fe578076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7fe57effd640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7fe578074b60 0x7fe578076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 -- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f449006c970 msgr2=0x7f449006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f449006c970 0x7f449006ee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4494009930 tx=0x7f4494009190 comp rx=0 tx=0).stop 2024-04-20T10:55:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 shutdown_connections 2024-04-20T10:55:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f536ebe1640 1 -- 172.21.15.27:0/1709528732 wait complete. 2024-04-20T10:55:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8a7c06c9f0 msgr2=0x7f8a7c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7fe57effd640 1 -- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe57806c940 msgr2=0x7fe57806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f8aa0481640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8a7c06c9f0 0x7f8a7c06eeb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009870 tx=0x7f8a88008040 comp rx=0 tx=0).stop 2024-04-20T10:55:49.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7fe57effd640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe57806c940 0x7fe57806ee00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0dada0 tx=0x7fe580009040 comp rx=0 tx=0).stop 2024-04-20T10:55:49.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a900e6690 msgr2=0x7f8a900de040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f8aa0481640 1 --2- 172.21.15.27:0/1893082912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a900e6690 0x7f8a900de040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e7f0 tx=0x7f8a9400ecb0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7fe57effd640 1 -- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe58c0452c0 msgr2=0x7fe58c0c4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7fe57effd640 1 --2- 172.21.15.27:0/2425314965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe58c0452c0 0x7fe58c0c4570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe58802c020 tx=0x7fe5880046c0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 -- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 msgr2=0x7f44a00cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 --2- 172.21.15.27:0/4191619435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f44a004b090 0x7f44a00cd850 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f44980098c0 tx=0x7f4498004730 comp rx=0 tx=0).stop 2024-04-20T10:55:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f44a6d9d640 1 -- 172.21.15.27:0/4191619435 reap_dead start 2024-04-20T10:55:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 -- 172.21.15.27:0/4191619435 shutdown_connections 2024-04-20T10:55:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 -- 172.21.15.27:0/4191619435 >> 172.21.15.27:0/4191619435 conn(0x7f44a0050180 msgr2=0x7f44a00d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7fe5997d7640 1 -- 172.21.15.27:0/2425314965 reap_dead start 2024-04-20T10:55:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 -- 172.21.15.27:0/4191619435 shutdown_connections 2024-04-20T10:55:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f8a9ea1e640 1 -- 172.21.15.27:0/1893082912 reap_dead start 2024-04-20T10:55:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 shutdown_connections 2024-04-20T10:55:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 >> 172.21.15.27:0/1893082912 conn(0x7f8a90050a00 msgr2=0x7f8a9004b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.433+0000 7f449cff9640 1 -- 172.21.15.27:0/4191619435 wait complete. 2024-04-20T10:55:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.434+0000 7fe57effd640 1 -- 172.21.15.27:0/2425314965 shutdown_connections 2024-04-20T10:55:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.434+0000 7fe57effd640 1 -- 172.21.15.27:0/2425314965 >> 172.21.15.27:0/2425314965 conn(0x7fe58c050170 msgr2=0x7fe58c0c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.434+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 shutdown_connections 2024-04-20T10:55:49.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.434+0000 7f8aa0481640 1 -- 172.21.15.27:0/1893082912 wait complete. 2024-04-20T10:55:49.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.434+0000 7fe57effd640 1 -- 172.21.15.27:0/2425314965 shutdown_connections 2024-04-20T10:55:49.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.434+0000 7fe57effd640 1 -- 172.21.15.27:0/2425314965 wait complete. 2024-04-20T10:55:49.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.435+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71a80057e0 con 0x7f7194074be0 2024-04-20T10:55:49.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.435+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 --> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f661c04bc50 con 0x7f661c0c7510 2024-04-20T10:55:49.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f661a7fc640 1 -- 172.21.15.27:0/993239369 <== osd.7 v2:172.21.15.137:6816/460941503 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f661c04bc50 con 0x7f661c0c7510 2024-04-20T10:55:49.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f71b508f640 1 -- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194074be0 msgr2=0x7f7194077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-20T10:55:49.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f71b508f640 1 -- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194074be0 msgr2=0x7f7194077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T10:55:49.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f71b508f640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194074be0 0x7f7194077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:55:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f71b508f640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194074be0 0x7f7194077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194074be0 msgr2=0x7f7194077000 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-20T10:55:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f71ad7fa640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194074be0 0x7f7194077000 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f71ad7fa640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194079320 0x7f719407b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71940776b0 con 0x7f7194079320 2024-04-20T10:55:49.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.436+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=21}) -- 0x7f719407bd20 con 0x7f71a8047bf0 2024-04-20T10:55:49.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.437+0000 7f71b508f640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194079320 0x7f719407b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.438+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7f661c0c7510 msgr2=0x7f661c003180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.438+0000 7f662aee9640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7f661c0c7510 0x7f661c003180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.438+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f660806c970 msgr2=0x7f660806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.438+0000 7f662aee9640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f660806c970 0x7f660806ee30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f661c0470a0 tx=0x7f66240af000 comp rx=0 tx=0).stop 2024-04-20T10:55:49.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.437+0000 7f71b508f640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194079320 0x7f719407b710 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.438+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f661c005e80 msgr2=0x7f661c0caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.438+0000 7f662aee9640 1 --2- 172.21.15.27:0/993239369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f661c005e80 0x7f661c0caba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f661400d800 tx=0x7f661400dcc0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.438+0000 7f662a6e8640 1 -- 172.21.15.27:0/993239369 reap_dead start 2024-04-20T10:55:49.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.438+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 shutdown_connections 2024-04-20T10:55:49.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.438+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 >> 172.21.15.27:0/993239369 conn(0x7f661c050170 msgr2=0x7f661c006b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.439+0000 7f71ad7fa640 1 -- 172.21.15.27:0/892027529 <== osd.0 v2:172.21.15.27:6808/2513808316 1 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f71940776b0 con 0x7f7194079320 2024-04-20T10:55:49.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.439+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 shutdown_connections 2024-04-20T10:55:49.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.439+0000 7f662aee9640 1 -- 172.21.15.27:0/993239369 wait complete. 2024-04-20T10:55:49.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.439+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194079320 msgr2=0x7f719407b710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.439+0000 7f71b5890640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f7194079320 0x7f719407b710 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.440+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f719406c9c0 msgr2=0x7f719406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.440+0000 7f71b5890640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f719406c9c0 0x7f719406ee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71a402b7d0 tx=0x7f71a402f000 comp rx=0 tx=0).stop 2024-04-20T10:55:49.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.440+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f71a8047bf0 msgr2=0x7f71a80e4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.440+0000 7f71b5890640 1 --2- 172.21.15.27:0/892027529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f71a8047bf0 0x7f71a80e4680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f719c00d550 tx=0x7f719c00da10 comp rx=0 tx=0).stop 2024-04-20T10:55:49.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.440+0000 7f71b508f640 1 -- 172.21.15.27:0/892027529 reap_dead start 2024-04-20T10:55:49.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.441+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 shutdown_connections 2024-04-20T10:55:49.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.441+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 >> 172.21.15.27:0/892027529 conn(0x7f71a804dd30 msgr2=0x7f71a804f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.441+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 shutdown_connections 2024-04-20T10:55:49.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.441+0000 7f71b5890640 1 -- 172.21.15.27:0/892027529 wait complete. 2024-04-20T10:55:49.502 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:49.503 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T10:55:49.504 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:49.504 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T10:55:49.505 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:49.505 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T10:55:49.506 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2024-04-20T10:55:49.506 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T10:55:49.507 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:49.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T10:55:49.508 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:49.508 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T10:55:49.509 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2024-04-20T10:55:49.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T10:55:49.510 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2024-04-20T10:55:49.510 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T10:55:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.714+0000 7f7c30225640 1 Processor -- start 2024-04-20T10:55:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.714+0000 7f7c30225640 1 -- start start 2024-04-20T10:55:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.715+0000 7f7c30225640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7c20047690 0x7f7c200e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.715+0000 7f7c30225640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c200480f0 con 0x7f7c20047c80 2024-04-20T10:55:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.715+0000 7f7c30225640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c200def70 con 0x7f7c200d9090 2024-04-20T10:55:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.715+0000 7f7c30225640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f7c200df250 con 0x7f7c20047690 2024-04-20T10:55:49.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.716+0000 7f7c2e7c2640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7c20047c80 0x7f7c2004bd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53456/0 (socket says 172.21.15.27:53456) 2024-04-20T10:55:49.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.716+0000 7f7c2e7c2640 1 -- 172.21.15.27:0/1066518225 learned_addr learned my addr 172.21.15.27:0/1066518225 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:49.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.716+0000 7f7c2d7c0640 1 --2- 172.21.15.27:0/1066518225 >> v2:172.21.15.137:3300/0 conn(0x7f7c20047690 0x7f7c200e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1372359653 0 0) 0x7f7c200480f0 con 0x7f7c20047c80 2024-04-20T10:55:49.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c14003520 con 0x7f7c20047c80 2024-04-20T10:55:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f4ef73fa640 1 Processor -- start 2024-04-20T10:55:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 936770562 0 0) 0x7f7c14003520 con 0x7f7c20047c80 2024-04-20T10:55:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 >> v2:172.21.15.137:3300/0 conn(0x7f7c20047690 msgr2=0x7f7c200e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 --2- 172.21.15.27:0/1066518225 >> v2:172.21.15.137:3300/0 conn(0x7f7c20047690 0x7f7c200e2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 >> v1:172.21.15.137:6789/0 conn(0x7f7c200d9090 legacy=0x7f7c20046350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c200df530 con 0x7f7c20047c80 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f7c280763b0 con 0x7f7c20047c80 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 >> v1:172.21.15.27:6789/0 conn(0x7f7c20047c80 legacy=0x7f7c2004bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c140039b0 0x7f7c14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c14004350 0x7f7c14004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c14004fd0 0x7f7c140141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7c200df250 con 0x7f7c14004fd0 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14014710 con 0x7f7c140039b0 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7c14014910 con 0x7f7c14004350 2024-04-20T10:55:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.717+0000 7f4ef73fa640 1 -- start start 2024-04-20T10:55:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f7c2d7c0640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c14004350 0x7f7c14004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f798a68c640 1 Processor -- start 2024-04-20T10:55:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f798a68c640 1 -- start start 2024-04-20T10:55:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f7c2e7c2640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c14004fd0 0x7f7c140141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f7c2dfc1640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c140039b0 0x7f7c14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f7c2d7c0640 1 -- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c140039b0 msgr2=0x7f7c14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f7c2d7c0640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c140039b0 0x7f7c14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f7c2d7c0640 1 -- 172.21.15.27:0/1066518225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c14004fd0 msgr2=0x7f7c140141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f47fd8ca640 1 Processor -- start 2024-04-20T10:55:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f47fd8ca640 1 -- start start 2024-04-20T10:55:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f7e8e1e1640 1 Processor -- start 2024-04-20T10:55:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f4ef73fa640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80dd3c0 0x7f4ee8046460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f4ef73fa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ee80e37a0 0x7f4ee804be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f7c2d7c0640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c14004fd0 0x7f7c140141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.718+0000 7f7c2d7c0640 1 -- 172.21.15.27:0/1066518225 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c200df530 con 0x7f7c14004350 2024-04-20T10:55:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f7c2e7c2640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c14004fd0 0x7f7c140141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f7c2e7c2640 1 -- 172.21.15.27:0/1066518225 reap_dead start 2024-04-20T10:55:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f2fea47e640 1 Processor -- start 2024-04-20T10:55:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f4ef6bf9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ee80e37a0 0x7f4ee804be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f4ef73fa640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee80dd7a0 con 0x7f4ee804ba60 2024-04-20T10:55:49.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f4ef73fa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee8045450 con 0x7f4ee80e37a0 2024-04-20T10:55:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f4ef73fa640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee8045fd0 con 0x7f4ee80dd3c0 2024-04-20T10:55:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f4ef63f8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80dd3c0 0x7f4ee8046460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f4ef63f8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80dd3c0 0x7f4ee8046460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60810/0 (socket says 172.21.15.27:60810) 2024-04-20T10:55:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f4ef63f8640 1 -- 172.21.15.27:0/2366834365 learned_addr learned my addr 172.21.15.27:0/2366834365 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f798a68c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f79840ceb10 0x7f79840cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f7983fff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f7984153730 0x7f79840c8530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:46674/0 (socket says 172.21.15.27:46674) 2024-04-20T10:55:49.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f7983fff640 1 -- 172.21.15.27:0/899457725 learned_addr learned my addr 172.21.15.27:0/899457725 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:49.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47fd8ca640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f806c750 0x7f47f8151be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47fd8ca640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f47f80c26b0 0x7f47f8070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47fd8ca640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f47f80c95a0 0x7f47f806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47fd8ca640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f47f80c24f0 con 0x7f47f80c95a0 2024-04-20T10:55:49.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47fd8ca640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f47f80c1f30 con 0x7f47f80c26b0 2024-04-20T10:55:49.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47fd8ca640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f47f80c2210 con 0x7f47f806c750 2024-04-20T10:55:49.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f798a68c640 1 -- 172.21.15.27:0/899457725 --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f798415d900 con 0x7f79840b48d0 2024-04-20T10:55:49.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f4ef63f8640 1 -- 172.21.15.27:0/2366834365 >> v2:172.21.15.27:3300/0 conn(0x7f4ee80e37a0 msgr2=0x7f4ee804be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f4ef63f8640 1 --2- 172.21.15.27:0/2366834365 >> v2:172.21.15.27:3300/0 conn(0x7f4ee80e37a0 0x7f4ee804be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f4ef63f8640 1 -- 172.21.15.27:0/2366834365 >> v1:172.21.15.137:6789/0 conn(0x7f4ee804ba60 legacy=0x7f4ee80e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f4ef63f8640 1 -- 172.21.15.27:0/2366834365 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee80e3e90 con 0x7f4ee80dd3c0 2024-04-20T10:55:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f798a68c640 1 -- 172.21.15.27:0/899457725 --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79840c8e50 con 0x7f7984153730 2024-04-20T10:55:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47f7fff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f47f80c26b0 0x7f47f8070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f798a68c640 1 -- 172.21.15.27:0/899457725 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f7984153b10 con 0x7f79840ceb10 2024-04-20T10:55:49.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47f7fff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f47f80c26b0 0x7f47f8070fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47660/0 (socket says 172.21.15.27:47660) 2024-04-20T10:55:49.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47f7fff640 1 -- 172.21.15.27:0/4228737892 learned_addr learned my addr 172.21.15.27:0/4228737892 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f2fea47e640 1 -- start start 2024-04-20T10:55:49.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.719+0000 7f7e8e1e1640 1 -- start start 2024-04-20T10:55:49.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f7e8e1e1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 0x7f7e8815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f7c2d7c0640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c14004350 0x7f7c14004820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009ac0 tx=0x7f7c14005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870f8993fde7189 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f7982ffd640 1 -- 172.21.15.27:0/899457725 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 616114527 0 0) 0x7f79840c8e50 con 0x7f7984153730 2024-04-20T10:55:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f7982ffd640 1 -- 172.21.15.27:0/899457725 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f796c003520 con 0x7f7984153730 2024-04-20T10:55:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f7988c29640 1 --2- 172.21.15.27:0/899457725 >> v2:172.21.15.137:3300/0 conn(0x7f79840ceb10 0x7f79840cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f7e8e1e1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7e880cc9c0 0x7f7e880cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18039070 con 0x7f7c14004350 2024-04-20T10:55:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1802c6d0 con 0x7f7c14004350 2024-04-20T10:55:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f7c2cfbf640 1 -- 172.21.15.27:0/1066518225 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18009690 con 0x7f7c14004350 2024-04-20T10:55:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47fc8c8640 1 --2- 172.21.15.27:0/4228737892 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f806c750 0x7f47f8151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f47f7fff640 1 -- 172.21.15.27:0/4228737892 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f806c750 msgr2=0x7f47f8151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f47f7fff640 1 --2- 172.21.15.27:0/4228737892 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f806c750 0x7f47f8151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f47f7fff640 1 -- 172.21.15.27:0/4228737892 >> v2:172.21.15.27:3300/0 conn(0x7f47f80c95a0 msgr2=0x7f47f806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f47fd0c9640 1 --2- 172.21.15.27:0/4228737892 >> v2:172.21.15.27:3300/0 conn(0x7f47f80c95a0 0x7f47f806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f47f7fff640 1 --2- 172.21.15.27:0/4228737892 >> v2:172.21.15.27:3300/0 conn(0x7f47f80c95a0 0x7f47f806b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f47f7fff640 1 -- 172.21.15.27:0/4228737892 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f80c8f50 con 0x7f47f80c26b0 2024-04-20T10:55:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f2fea47e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2fe4156100 0x7f2fe4162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f2fea47e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fea47e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe415f6a0 con 0x7f2fe4160160 2024-04-20T10:55:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fea47e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe41605f0 con 0x7f2fe4156100 2024-04-20T10:55:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fea47e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe40c3880 con 0x7f2fe40c87a0 2024-04-20T10:55:49.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f383c767640 1 Processor -- start 2024-04-20T10:55:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f7982ffd640 1 -- 172.21.15.27:0/899457725 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3536828204 0 0) 0x7f798415d900 con 0x7f79840b48d0 2024-04-20T10:55:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.720+0000 7f7982ffd640 1 -- 172.21.15.27:0/899457725 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79840c8e50 con 0x7f79840b48d0 2024-04-20T10:55:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f7988c29640 1 -- 172.21.15.27:0/899457725 >> v1:172.21.15.137:6789/0 conn(0x7f7984153730 legacy=0x7f79840c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f7988c29640 1 -- 172.21.15.27:0/899457725 >> v1:172.21.15.27:6789/0 conn(0x7f79840b48d0 legacy=0x7f79841593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f7988c29640 1 -- 172.21.15.27:0/899457725 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798415cb40 con 0x7f79840ceb10 2024-04-20T10:55:49.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f383c767640 1 -- start start 2024-04-20T10:55:49.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fe8c7b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fe947c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2fe4156100 0x7f2fe4162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fe947c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2fe4156100 0x7f2fe4162c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48664/0 (socket says 172.21.15.27:48664) 2024-04-20T10:55:49.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fe947c640 1 -- 172.21.15.27:0/2463030082 learned_addr learned my addr 172.21.15.27:0/2463030082 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fe947c640 1 -- 172.21.15.27:0/2463030082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 msgr2=0x7f2fe40c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fe947c640 1 --2- 172.21.15.27:0/2463030082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fe947c640 1 -- 172.21.15.27:0/2463030082 >> v1:172.21.15.137:6789/0 conn(0x7f2fe4160160 legacy=0x7f2fe406bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f2fe947c640 1 -- 172.21.15.27:0/2463030082 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe40c42c0 con 0x7f2fe4156100 2024-04-20T10:55:49.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f47fd0c9640 1 --2- 172.21.15.27:0/4228737892 >> v2:172.21.15.27:3300/0 conn(0x7f47f80c95a0 0x7f47f806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:49.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f4ef63f8640 1 --2- 172.21.15.27:0/2366834365 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80dd3c0 0x7f4ee8046460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009870 tx=0x7f4eec02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebfa8662e93345cf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7c30225640 1 -- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c14004350 msgr2=0x7f7c14004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7c30225640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c14004350 0x7f7c14004820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009ac0 tx=0x7f7c14005830 comp rx=0 tx=0).stop 2024-04-20T10:55:49.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7c30225640 1 -- 172.21.15.27:0/1066518225 shutdown_connections 2024-04-20T10:55:49.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7c30225640 1 --2- 172.21.15.27:0/1066518225 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c14004350 0x7f7c14004820 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7c30225640 1 -- 172.21.15.27:0/1066518225 >> 172.21.15.27:0/1066518225 conn(0x7f7c20051290 msgr2=0x7f7c200516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f383c767640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 0x7f3834162e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f383c767640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f38340c75d0 0x7f38340c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f383c767640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3834163560 con 0x7f3834071a60 2024-04-20T10:55:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f383c767640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3834074340 con 0x7f38340c75d0 2024-04-20T10:55:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f383c767640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f3834163840 con 0x7f3834159c00 2024-04-20T10:55:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f4ef53f6640 1 -- 172.21.15.27:0/2366834365 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec034070 con 0x7f4ee80dd3c0 2024-04-20T10:55:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f4ef53f6640 1 -- 172.21.15.27:0/2366834365 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eec030840 con 0x7f4ee80dd3c0 2024-04-20T10:55:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f47fc8c8640 1 --2- 172.21.15.27:0/4228737892 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f806c750 0x7f47f8151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f4ef53f6640 1 -- 172.21.15.27:0/2366834365 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec034070 con 0x7f4ee80dd3c0 2024-04-20T10:55:49.737 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7c30225640 1 -- 172.21.15.27:0/1066518225 shutdown_connections 2024-04-20T10:55:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f383a503640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 0x7f3834162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f2fe947c640 1 --2- 172.21.15.27:0/2463030082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4156100 0x7f2fe4162c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00098a0 tx=0x7f2fe002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0d573ca380e61e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f3839d02640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f3834071a60 0x7f3834160240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:46704/0 (socket says 172.21.15.27:46704) 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f7e8d1df640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 0x7f7e8815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f7e8d1df640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 0x7f7e8815f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60818/0 (socket says 172.21.15.27:60818) 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.721+0000 7f7e8c9de640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f7e88163870 0x7f7e880cbca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:46684/0 (socket says 172.21.15.27:46684) 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e8d1df640 1 -- 172.21.15.27:0/3535488181 learned_addr learned my addr 172.21.15.27:0/3535488181 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3535488181 --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e880b5080 con 0x7f7e88163870 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3535488181 --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7e880cc3b0 con 0x7f7e880cc9c0 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3535488181 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7e88162e20 con 0x7f7e880c7380 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2370239905 0 0) 0x7f7e880b5080 con 0x7f7e88163870 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e74003520 con 0x7f7e88163870 2024-04-20T10:55:49.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e8d9e0640 1 --2- 172.21.15.27:0/3535488181 >> v2:172.21.15.27:3300/0 conn(0x7f7e880cc9c0 0x7f7e880cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f3839d02640 1 -- 172.21.15.27:0/2229708702 learned_addr learned my addr 172.21.15.27:0/2229708702 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f383ad04640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f38340c75d0 0x7f38340c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 <== mon.1 v1:172.21.15.137:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 831797072 0 0) 0x7f7e74003520 con 0x7f7e88163870 2024-04-20T10:55:49.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f2fdbfff640 1 -- 172.21.15.27:0/2463030082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0034070 con 0x7f2fe4156100 2024-04-20T10:55:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f2fdbfff640 1 -- 172.21.15.27:0/2463030082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe0030840 con 0x7f2fe4156100 2024-04-20T10:55:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f2fdbfff640 1 -- 172.21.15.27:0/2463030082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0030c10 con 0x7f2fe4156100 2024-04-20T10:55:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f47f7fff640 1 --2- 172.21.15.27:0/4228737892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f80c26b0 0x7f47f8070fe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47f806fb00 tx=0x7f47ec02b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3083650f443fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 msgr2=0x7f7e8815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e7ffff640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 0x7f7e8815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 >> v2:172.21.15.27:3300/0 conn(0x7f7e880cc9c0 msgr2=0x7f7e880cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 --2- 172.21.15.27:0/3535488181 >> v2:172.21.15.27:3300/0 conn(0x7f7e880cc9c0 0x7f7e880cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 --> v1:172.21.15.137:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e880b5360 con 0x7f7e88163870 2024-04-20T10:55:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e8d9e0640 1 --2- 172.21.15.27:0/3535488181 >> v2:172.21.15.27:3300/0 conn(0x7f7e880cc9c0 0x7f7e880cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:55:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f3839501640 1 -- 172.21.15.27:0/2229708702 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914034237 0 0) 0x7f3834163560 con 0x7f3834071a60 2024-04-20T10:55:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7988c29640 1 --2- 172.21.15.27:0/899457725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 0x7f79840cef10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f79740097a0 tx=0x7f797402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31a0eb23b0d7d8f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f3839501640 1 -- 172.21.15.27:0/2229708702 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3820003520 con 0x7f3834071a60 2024-04-20T10:55:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 <== mon.1 v1:172.21.15.137:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f7e78001670 con 0x7f7e88163870 2024-04-20T10:55:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 >> v1:172.21.15.137:6789/0 conn(0x7f7e88163870 legacy=0x7f7e880cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e740039b0 0x7f7e74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 0x7f7e74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e74004d40 0x7f7e740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88162e20 con 0x7f7e74004d40 2024-04-20T10:55:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7400ba70 con 0x7f7e740039b0 2024-04-20T10:55:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7e7400bc70 con 0x7f7e880c7380 2024-04-20T10:55:49.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f4ef73fa640 1 -- 172.21.15.27:0/2366834365 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80dd3c0 msgr2=0x7f4ee8046460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f4ef73fa640 1 --2- 172.21.15.27:0/2366834365 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80dd3c0 0x7f4ee8046460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009870 tx=0x7f4eec02b980 comp rx=0 tx=0).stop 2024-04-20T10:55:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f383ad04640 1 -- 172.21.15.27:0/2229708702 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 msgr2=0x7f3834162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f383ad04640 1 --2- 172.21.15.27:0/2229708702 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 0x7f3834162e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f383ad04640 1 -- 172.21.15.27:0/2229708702 >> v1:172.21.15.137:6789/0 conn(0x7f3834071a60 legacy=0x7f3834160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f383ad04640 1 -- 172.21.15.27:0/2229708702 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3834163b20 con 0x7f38340c75d0 2024-04-20T10:55:49.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7e8d9e0640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e74004d40 0x7f7e740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f7982ffd640 1 -- 172.21.15.27:0/899457725 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974034070 con 0x7f79840ceb10 2024-04-20T10:55:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7982ffd640 1 -- 172.21.15.27:0/899457725 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7974030610 con 0x7f79840ceb10 2024-04-20T10:55:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7982ffd640 1 -- 172.21.15.27:0/899457725 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974034070 con 0x7f79840ceb10 2024-04-20T10:55:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7e8d1df640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e740039b0 0x7f7e74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.722+0000 7f7c30225640 1 -- 172.21.15.27:0/1066518225 wait complete. 2024-04-20T10:55:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f0135550640 1 Processor -- start 2024-04-20T10:55:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.723+0000 7f0135550640 1 -- start start 2024-04-20T10:55:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7e8c9de640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 0x7f7e74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7e8d1df640 1 -- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 msgr2=0x7f7e74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7e8d1df640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 0x7f7e74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f47f77fe640 1 -- 172.21.15.27:0/4228737892 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec034070 con 0x7f47f80c26b0 2024-04-20T10:55:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f47f77fe640 1 -- 172.21.15.27:0/4228737892 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec030890 con 0x7f47f80c26b0 2024-04-20T10:55:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f47f77fe640 1 -- 172.21.15.27:0/4228737892 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec030ca0 con 0x7f47f80c26b0 2024-04-20T10:55:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7c30225640 1 Processor -- start 2024-04-20T10:55:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f0135550640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f01300d1070 0x7f01300d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f0135550640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f01300c5e40 0x7f013015fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7e8d1df640 1 -- 172.21.15.27:0/3535488181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e74004d40 msgr2=0x7f7e740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7e8d1df640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e74004d40 0x7f7e740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7e8d1df640 1 -- 172.21.15.27:0/3535488181 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e74003520 con 0x7f7e740039b0 2024-04-20T10:55:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7e8d9e0640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e74004d40 0x7f7e740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f2fea47e640 1 -- 172.21.15.27:0/2463030082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4156100 msgr2=0x7f2fe4162c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f2fea47e640 1 --2- 172.21.15.27:0/2463030082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4156100 0x7f2fe4162c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00098a0 tx=0x7f2fe002b980 comp rx=0 tx=0).stop 2024-04-20T10:55:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f4ef73fa640 1 -- 172.21.15.27:0/2366834365 shutdown_connections 2024-04-20T10:55:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f4ef73fa640 1 --2- 172.21.15.27:0/2366834365 >> v2:172.21.15.27:3300/0 conn(0x7f4ee80e37a0 0x7f4ee804be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f4ef73fa640 1 --2- 172.21.15.27:0/2366834365 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80dd3c0 0x7f4ee8046460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f4ef73fa640 1 -- 172.21.15.27:0/2366834365 >> 172.21.15.27:0/2366834365 conn(0x7f4ee8050180 msgr2=0x7f4ee8050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f7e8c9de640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e880c7380 0x7f7e74004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:49.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f7c30225640 1 -- start start 2024-04-20T10:55:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f2fea47e640 1 -- 172.21.15.27:0/2463030082 shutdown_connections 2024-04-20T10:55:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f2fea47e640 1 --2- 172.21.15.27:0/2463030082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f2fea47e640 1 --2- 172.21.15.27:0/2463030082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4156100 0x7f2fe4162c90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f2fea47e640 1 -- 172.21.15.27:0/2463030082 >> 172.21.15.27:0/2463030082 conn(0x7f2fe40cc920 msgr2=0x7f2fe40ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f0135550640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300c6320 0x7f01300c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f47fd8ca640 1 -- 172.21.15.27:0/4228737892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f80c26b0 msgr2=0x7f47f8070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f0135550640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f01300b6db0 con 0x7f01300d1070 2024-04-20T10:55:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f0135550640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f01300c6870 con 0x7f01300c5e40 2024-04-20T10:55:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f0135550640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f013015ff80 con 0x7f01300c6320 2024-04-20T10:55:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f012effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f01300d1070 0x7f01300d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f012effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f01300d1070 0x7f01300d1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48688/0 (socket says 172.21.15.27:48688) 2024-04-20T10:55:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f012effd640 1 -- 172.21.15.27:0/4250305460 learned_addr learned my addr 172.21.15.27:0/4250305460 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.724+0000 7f012f7fe640 1 --2- 172.21.15.27:0/4250305460 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300c6320 0x7f01300c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f012effd640 1 -- 172.21.15.27:0/4250305460 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300c6320 msgr2=0x7f01300c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:49.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f012effd640 1 --2- 172.21.15.27:0/4250305460 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300c6320 0x7f01300c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f012effd640 1 -- 172.21.15.27:0/4250305460 >> v2:172.21.15.137:3300/0 conn(0x7f01300c5e40 msgr2=0x7f013015fa40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:55:49.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f012effd640 1 --2- 172.21.15.27:0/4250305460 >> v2:172.21.15.137:3300/0 conn(0x7f01300c5e40 0x7f013015fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f012effd640 1 -- 172.21.15.27:0/4250305460 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0130160210 con 0x7f01300d1070 2024-04-20T10:55:49.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f47fd8ca640 1 --2- 172.21.15.27:0/4228737892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f80c26b0 0x7f47f8070fe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47f806fb00 tx=0x7f47ec02b9d0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f383ad04640 1 --2- 172.21.15.27:0/2229708702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 0x7f38340c86c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f382c009d90 tx=0x7f382c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dbd7c392f923d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f3839501640 1 -- 172.21.15.27:0/2229708702 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c039070 con 0x7f38340c75d0 2024-04-20T10:55:49.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f3839501640 1 -- 172.21.15.27:0/2229708702 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f382c02cd10 con 0x7f38340c75d0 2024-04-20T10:55:49.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f3839501640 1 -- 172.21.15.27:0/2229708702 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c034660 con 0x7f38340c75d0 2024-04-20T10:55:49.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f4ef73fa640 1 -- 172.21.15.27:0/2366834365 shutdown_connections 2024-04-20T10:55:49.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f4ef73fa640 1 -- 172.21.15.27:0/2366834365 wait complete. 2024-04-20T10:55:49.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f7c30225640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c14004350 0x7f7c200d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f47fd8ca640 1 -- 172.21.15.27:0/4228737892 shutdown_connections 2024-04-20T10:55:49.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f47fd8ca640 1 --2- 172.21.15.27:0/4228737892 >> v2:172.21.15.27:3300/0 conn(0x7f47f80c95a0 0x7f47f806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f47fd8ca640 1 --2- 172.21.15.27:0/4228737892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f80c26b0 0x7f47f8070fe0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f47fd8ca640 1 --2- 172.21.15.27:0/4228737892 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f806c750 0x7f47f8151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f47fd8ca640 1 -- 172.21.15.27:0/4228737892 >> 172.21.15.27:0/4228737892 conn(0x7f47f80681b0 msgr2=0x7f47f806ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f7c30225640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c200d0f40 0x7f7c200d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f47fd8ca640 1 -- 172.21.15.27:0/4228737892 shutdown_connections 2024-04-20T10:55:49.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f7e8d1df640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e740039b0 0x7f7e74003dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098d0 tx=0x7f7e74005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f0f254c7a78b830 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f47fd8ca640 1 -- 172.21.15.27:0/4228737892 wait complete. 2024-04-20T10:55:49.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f7c2dfc1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c14004350 0x7f7c200d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f7c2dfc1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c14004350 0x7f7c200d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47682/0 (socket says 172.21.15.27:47682) 2024-04-20T10:55:49.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84034070 con 0x7f7e740039b0 2024-04-20T10:55:49.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e84030670 con 0x7f7e740039b0 2024-04-20T10:55:49.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f7e7ffff640 1 -- 172.21.15.27:0/3535488181 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400f040 con 0x7f7e740039b0 2024-04-20T10:55:49.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f7c2dfc1640 1 -- 172.21.15.27:0/3637665743 learned_addr learned my addr 172.21.15.27:0/3637665743 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f2fea47e640 1 -- 172.21.15.27:0/2463030082 shutdown_connections 2024-04-20T10:55:49.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.725+0000 7f2fea47e640 1 -- 172.21.15.27:0/2463030082 wait complete. 2024-04-20T10:55:49.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f798a68c640 1 -- 172.21.15.27:0/899457725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 msgr2=0x7f79840cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f798a68c640 1 --2- 172.21.15.27:0/899457725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 0x7f79840cef10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f79740097a0 tx=0x7f797402b960 comp rx=0 tx=0).stop 2024-04-20T10:55:49.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f383c767640 1 -- 172.21.15.27:0/2229708702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 msgr2=0x7f38340c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f383c767640 1 --2- 172.21.15.27:0/2229708702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 0x7f38340c86c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f382c009d90 tx=0x7f382c02bf60 comp rx=0 tx=0).stop 2024-04-20T10:55:49.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f012effd640 1 --2- 172.21.15.27:0/4250305460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300d1070 0x7f01300d1450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f013008e910 tx=0x7f012002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b01f9059132cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f7c30225640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c200d2370 0x7f7c200d9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f798a68c640 1 -- 172.21.15.27:0/899457725 shutdown_connections 2024-04-20T10:55:49.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f798a68c640 1 --2- 172.21.15.27:0/899457725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 0x7f79840cef10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f798a68c640 1 -- 172.21.15.27:0/899457725 >> 172.21.15.27:0/899457725 conn(0x7f79840676c0 msgr2=0x7f7984067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f383c767640 1 -- 172.21.15.27:0/2229708702 shutdown_connections 2024-04-20T10:55:49.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f383c767640 1 --2- 172.21.15.27:0/2229708702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 0x7f38340c86c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f798a68c640 1 -- 172.21.15.27:0/899457725 shutdown_connections 2024-04-20T10:55:49.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f383c767640 1 --2- 172.21.15.27:0/2229708702 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 0x7f3834162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.759 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f798a68c640 1 -- 172.21.15.27:0/899457725 wait complete. 2024-04-20T10:55:49.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f012dffb640 1 -- 172.21.15.27:0/4250305460 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120035020 con 0x7f01300d1070 2024-04-20T10:55:49.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f012dffb640 1 -- 172.21.15.27:0/4250305460 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0120031890 con 0x7f01300d1070 2024-04-20T10:55:49.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f012dffb640 1 -- 172.21.15.27:0/4250305460 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120031c60 con 0x7f01300d1070 2024-04-20T10:55:49.760 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f383c767640 1 -- 172.21.15.27:0/2229708702 >> 172.21.15.27:0/2229708702 conn(0x7f38340ccbd0 msgr2=0x7f38340ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.726+0000 7f7c2d7c0640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c200d0f40 0x7f7c200d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f7c30225640 1 -- 172.21.15.27:0/3637665743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7c200d9470 con 0x7f7c200d2370 2024-04-20T10:55:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f7c30225640 1 -- 172.21.15.27:0/3637665743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7c200dfaf0 con 0x7f7c14004350 2024-04-20T10:55:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f7c2e7c2640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c200d2370 0x7f7c200d9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f4ef73fa640 1 Processor -- start 2024-04-20T10:55:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f7c30225640 1 -- 172.21.15.27:0/3637665743 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7c200e2d10 con 0x7f7c200d0f40 2024-04-20T10:55:49.761 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f383c767640 1 -- 172.21.15.27:0/2229708702 shutdown_connections 2024-04-20T10:55:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f4ef73fa640 1 -- start start 2024-04-20T10:55:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f47fd8ca640 1 Processor -- start 2024-04-20T10:55:49.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f383c767640 1 -- 172.21.15.27:0/2229708702 wait complete. 2024-04-20T10:55:49.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f47fd8ca640 1 -- start start 2024-04-20T10:55:49.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f0135550640 1 -- 172.21.15.27:0/4250305460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300d1070 msgr2=0x7f01300d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f0135550640 1 --2- 172.21.15.27:0/4250305460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300d1070 0x7f01300d1450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f013008e910 tx=0x7f012002b930 comp rx=0 tx=0).stop 2024-04-20T10:55:49.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f0135550640 1 -- 172.21.15.27:0/4250305460 shutdown_connections 2024-04-20T10:55:49.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f0135550640 1 --2- 172.21.15.27:0/4250305460 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300c6320 0x7f01300c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f0135550640 1 --2- 172.21.15.27:0/4250305460 >> v2:172.21.15.137:3300/0 conn(0x7f01300c5e40 0x7f013015fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f0135550640 1 --2- 172.21.15.27:0/4250305460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300d1070 0x7f01300d1450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f0135550640 1 -- 172.21.15.27:0/4250305460 >> 172.21.15.27:0/4250305460 conn(0x7f01300678a0 msgr2=0x7f0130067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f7c2e7c2640 1 -- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c200d0f40 msgr2=0x7f7c200d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f7c2e7c2640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c200d0f40 0x7f7c200d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f7c2e7c2640 1 -- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c14004350 msgr2=0x7f7c200d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f7c2e7c2640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c14004350 0x7f7c200d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f7c2e7c2640 1 -- 172.21.15.27:0/3637665743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c200d27f0 con 0x7f7c200d2370 2024-04-20T10:55:49.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f0135550640 1 -- 172.21.15.27:0/4250305460 shutdown_connections 2024-04-20T10:55:49.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f0135550640 1 -- 172.21.15.27:0/4250305460 wait complete. 2024-04-20T10:55:49.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f7c2e7c2640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c200d2370 0x7f7c200d9940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7c2802db60 tx=0x7f7c28069770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f47fd8ca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f806c750 0x7f47f80cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f798a68c640 1 Processor -- start 2024-04-20T10:55:49.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e740039b0 msgr2=0x7f7e74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f7e8e1e1640 1 --2- 172.21.15.27:0/3535488181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e740039b0 0x7f7e74003dd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098d0 tx=0x7f7e74005830 comp rx=0 tx=0).stop 2024-04-20T10:55:49.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f7e8d9e0640 1 -- 172.21.15.27:0/3535488181 reap_dead start 2024-04-20T10:55:49.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f7c1e7fc640 1 -- 172.21.15.27:0/3637665743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2806b930 con 0x7f7c200d2370 2024-04-20T10:55:49.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f7c30225640 1 -- 172.21.15.27:0/3637665743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c200cff30 con 0x7f7c200d2370 2024-04-20T10:55:49.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f7c30225640 1 -- 172.21.15.27:0/3637665743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c200dabf0 con 0x7f7c200d2370 2024-04-20T10:55:49.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47fd8ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f80c26b0 0x7f47f81f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47fd8ca640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f80c95a0 0x7f47f813be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47fc8c8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f806c750 0x7f47f80cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47fc8c8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f806c750 0x7f47f80cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47684/0 (socket says 172.21.15.27:47684) 2024-04-20T10:55:49.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47fc8c8640 1 -- 172.21.15.27:0/2209601312 learned_addr learned my addr 172.21.15.27:0/2209601312 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47fd0c9640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f80c95a0 0x7f47f813be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47fd8ca640 1 -- 172.21.15.27:0/2209601312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47f80c9b50 con 0x7f47f80c26b0 2024-04-20T10:55:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47fd8ca640 1 -- 172.21.15.27:0/2209601312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47f80c99d0 con 0x7f47f806c750 2024-04-20T10:55:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47fd8ca640 1 -- 172.21.15.27:0/2209601312 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f47f80c9cd0 con 0x7f47f80c95a0 2024-04-20T10:55:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f47f7fff640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f80c26b0 0x7f47f81f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f2fea47e640 1 Processor -- start 2024-04-20T10:55:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.727+0000 7f2fea47e640 1 -- start start 2024-04-20T10:55:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f2fea47e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f2fea47e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe4156100 0x7f2fe4073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f2fea47e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4160160 0x7f2fe41516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f2fea47e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe415f960 con 0x7f2fe4160160 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f2fea47e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe40c4880 con 0x7f2fe4156100 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f2fea47e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe415fc40 con 0x7f2fe40c87a0 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f2fe947c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.728+0000 7f2fe9c7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4160160 0x7f2fe41516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f2fe947c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60860/0 (socket says 172.21.15.27:60860) 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f2fe947c640 1 -- 172.21.15.27:0/2950665224 learned_addr learned my addr 172.21.15.27:0/2950665224 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f2fe947c640 1 -- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe4156100 msgr2=0x7f2fe4073830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f2fe947c640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe4156100 0x7f2fe4073830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f2fe947c640 1 -- 172.21.15.27:0/2950665224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4160160 msgr2=0x7f2fe41516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f2fe947c640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4160160 0x7f2fe41516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f2fe947c640 1 -- 172.21.15.27:0/2950665224 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe4152f40 con 0x7f2fe40c87a0 2024-04-20T10:55:49.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f2fe9c7d640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4160160 0x7f2fe41516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f4ef73fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ee804ba60 0x7f4ee80e31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f4ef73fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee80dd3c0 0x7f4ee8049c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f4ef73fa640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80e37a0 0x7f4ee80df450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f4ef73fa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8045980 con 0x7f4ee80dd3c0 2024-04-20T10:55:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f4ef73fa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee80e3c30 con 0x7f4ee804ba60 2024-04-20T10:55:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f4ef73fa640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee8045c60 con 0x7f4ee80e37a0 2024-04-20T10:55:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f383c767640 1 Processor -- start 2024-04-20T10:55:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f383c767640 1 -- start start 2024-04-20T10:55:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3535488181 shutdown_connections 2024-04-20T10:55:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3535488181 >> 172.21.15.27:0/3535488181 conn(0x7f7e8806cee0 msgr2=0x7f7e880676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f2fe947c640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c98d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0000c00 tx=0x7f2fe0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f47fc8c8640 1 -- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f80c95a0 msgr2=0x7f47f813be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f47fc8c8640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f80c95a0 0x7f47f813be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f47fc8c8640 1 -- 172.21.15.27:0/2209601312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f80c26b0 msgr2=0x7f47f81f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f47fc8c8640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f80c26b0 0x7f47f81f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f47fc8c8640 1 -- 172.21.15.27:0/2209601312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f80cab60 con 0x7f47f806c750 2024-04-20T10:55:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f47f7fff640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f80c26b0 0x7f47f81f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f4ef63f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ee804ba60 0x7f4ee80e31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f4ef63f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ee804ba60 0x7f4ee80e31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47694/0 (socket says 172.21.15.27:47694) 2024-04-20T10:55:49.771 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f4ef63f8640 1 -- 172.21.15.27:0/841918071 learned_addr learned my addr 172.21.15.27:0/841918071 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.729+0000 7f798a68c640 1 -- start start 2024-04-20T10:55:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f798a68c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79840b48d0 0x7f79841582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f798a68c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 0x7f79840ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f798a68c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7984153730 0x7f79840caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f798a68c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f798415d3e0 con 0x7f79840b48d0 2024-04-20T10:55:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f798a68c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f798415d100 con 0x7f79840ceb10 2024-04-20T10:55:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3535488181 shutdown_connections 2024-04-20T10:55:49.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f2fd9ffb640 1 -- 172.21.15.27:0/2950665224 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0048070 con 0x7f2fe40c87a0 2024-04-20T10:55:49.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f798a68c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f79840c9b70 con 0x7f7984153730 2024-04-20T10:55:49.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f2fd9ffb640 1 -- 172.21.15.27:0/2950665224 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe0030640 con 0x7f2fe40c87a0 2024-04-20T10:55:49.773 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3535488181 wait complete. 2024-04-20T10:55:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f47fc8c8640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f806c750 0x7f47f80cd100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c7c0 tx=0x7f47e400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f47f57fa640 1 -- 172.21.15.27:0/2209601312 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4015070 con 0x7f47f806c750 2024-04-20T10:55:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f7c1e7fc640 1 -- 172.21.15.27:0/3637665743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c28068900 con 0x7f7c200d2370 2024-04-20T10:55:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f47f57fa640 1 -- 172.21.15.27:0/2209601312 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e4007a30 con 0x7f47f806c750 2024-04-20T10:55:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.731+0000 7f4ef6bf9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80e37a0 0x7f4ee80df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f7c1e7fc640 1 -- 172.21.15.27:0/3637665743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2806bd10 con 0x7f7c200d2370 2024-04-20T10:55:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.731+0000 7f4ef5bf7640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee80dd3c0 0x7f4ee8049c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.731+0000 7f7c0bfff640 1 -- 172.21.15.27:0/3637665743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c200057e0 con 0x7f7c200d2370 2024-04-20T10:55:49.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.731+0000 7f47fd8ca640 1 -- 172.21.15.27:0/2209601312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f80cae40 con 0x7f47f806c750 2024-04-20T10:55:49.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.731+0000 7f79837fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 0x7f79840ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.731+0000 7f79837fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 0x7f79840ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47698/0 (socket says 172.21.15.27:47698) 2024-04-20T10:55:49.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.731+0000 7f79837fe640 1 -- 172.21.15.27:0/3269285048 learned_addr learned my addr 172.21.15.27:0/3269285048 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.731+0000 7f47fd8ca640 1 -- 172.21.15.27:0/2209601312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f80cb400 con 0x7f47f806c750 2024-04-20T10:55:49.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.731+0000 7f47f57fa640 1 -- 172.21.15.27:0/2209601312 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4007e00 con 0x7f47f806c750 2024-04-20T10:55:49.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f2fd9ffb640 1 -- 172.21.15.27:0/2950665224 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0034040 con 0x7f2fe40c87a0 2024-04-20T10:55:49.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe4072630 con 0x7f2fe40c87a0 2024-04-20T10:55:49.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe4072bf0 con 0x7f2fe40c87a0 2024-04-20T10:55:49.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f79837fe640 1 -- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7984153730 msgr2=0x7f79840caee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T10:55:49.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f79837fe640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7984153730 0x7f79840caee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f79837fe640 1 -- 172.21.15.27:0/3269285048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79840b48d0 msgr2=0x7f79841582b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:55:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f0135550640 1 Processor -- start 2024-04-20T10:55:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.730+0000 7f0135550640 1 -- start start 2024-04-20T10:55:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f0135550640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01300c5e40 0x7f013020f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f0135550640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300c6320 0x7f0130213080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f0135550640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300d1070 0x7f0130150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f0135550640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f01300c6ac0 con 0x7f01300c6320 2024-04-20T10:55:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f0135550640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01301607d0 con 0x7f01300c5e40 2024-04-20T10:55:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f0135550640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f01300c6da0 con 0x7f01300d1070 2024-04-20T10:55:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f7e8e1e1640 1 Processor -- start 2024-04-20T10:55:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f012e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300c6320 0x7f0130213080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f012e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300c6320 0x7f0130213080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48744/0 (socket says 172.21.15.27:48744) 2024-04-20T10:55:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f012e7fc640 1 -- 172.21.15.27:0/3238808265 learned_addr learned my addr 172.21.15.27:0/3238808265 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f012f7fe640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300d1070 0x7f0130150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f012effd640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01300c5e40 0x7f013020f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f4ef6bf9640 1 -- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ee804ba60 msgr2=0x7f4ee80e31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f4ef6bf9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ee804ba60 0x7f4ee80e31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f4ef6bf9640 1 -- 172.21.15.27:0/841918071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee80dd3c0 msgr2=0x7f4ee8049c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f4ef6bf9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee80dd3c0 0x7f4ee8049c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f4ef6bf9640 1 -- 172.21.15.27:0/841918071 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee8179130 con 0x7f4ee80e37a0 2024-04-20T10:55:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f4ef5bf7640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee80dd3c0 0x7f4ee8049c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f4ef6bf9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80e37a0 0x7f4ee80df450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0065480 tx=0x7f4ef0077440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f383c767640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3834071a60 0x7f38340cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f383c767640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 0x7f38340c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f79837fe640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79840b48d0 0x7f79841582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f79837fe640 1 -- 172.21.15.27:0/3269285048 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7984159150 con 0x7f79840ceb10 2024-04-20T10:55:49.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f383c767640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 0x7f38340c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f383c767640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3834070fb0 con 0x7f38340c75d0 2024-04-20T10:55:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f383c767640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f383415fd20 con 0x7f3834071a60 2024-04-20T10:55:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f3839d02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 0x7f38340c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f4ee6ffd640 1 -- 172.21.15.27:0/841918071 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00695b0 con 0x7f4ee80e37a0 2024-04-20T10:55:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f4ef73fa640 1 -- 172.21.15.27:0/841918071 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee817d120 con 0x7f4ee80e37a0 2024-04-20T10:55:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f4ef73fa640 1 -- 172.21.15.27:0/841918071 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81796b0 con 0x7f4ee80e37a0 2024-04-20T10:55:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f3839d02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 0x7f38340c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48768/0 (socket says 172.21.15.27:48768) 2024-04-20T10:55:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f3839d02640 1 -- 172.21.15.27:0/1951983964 learned_addr learned my addr 172.21.15.27:0/1951983964 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f38340c8c00 con 0x7f3834159c00 2024-04-20T10:55:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f7e8e1e1640 1 -- start start 2024-04-20T10:55:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f4ee6ffd640 1 -- 172.21.15.27:0/841918071 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef0069750 con 0x7f4ee80e37a0 2024-04-20T10:55:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f4ee6ffd640 1 -- 172.21.15.27:0/841918071 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00744e0 con 0x7f4ee80e37a0 2024-04-20T10:55:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f7c1e7fc640 1 -- 172.21.15.27:0/3637665743 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c28097480 con 0x7f7c200d2370 2024-04-20T10:55:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f7e8e1e1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e740039b0 0x7f7e881f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f383ad04640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 0x7f38340c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f383a503640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3834071a60 0x7f38340cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f7e8e1e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e8815c260 0x7f7e881e9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f47fd8ca640 1 -- 172.21.15.27:0/2209601312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f80696d0 con 0x7f47f806c750 2024-04-20T10:55:49.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe4076610 con 0x7f2fe40c87a0 2024-04-20T10:55:49.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7e8e1e1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e88158840 0x7f7e88158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f3839d02640 1 -- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 msgr2=0x7f38340c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f3839d02640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 0x7f38340c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f3839d02640 1 -- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3834071a60 msgr2=0x7f38340cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f012e7fc640 1 -- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300d1070 msgr2=0x7f0130150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.732+0000 7f012e7fc640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300d1070 0x7f0130150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f3839d02640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3834071a60 0x7f38340cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f012e7fc640 1 -- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01300c5e40 msgr2=0x7f013020f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f012e7fc640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01300c5e40 0x7f013020f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f012e7fc640 1 -- 172.21.15.27:0/3238808265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01300cac70 con 0x7f01300c6320 2024-04-20T10:55:49.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.733+0000 7f012effd640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01300c5e40 0x7f013020f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:49.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f012e7fc640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300c6320 0x7f0130213080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f011c00b410 tx=0x7f011c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f3839d02640 1 -- 172.21.15.27:0/1951983964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38341f8f70 con 0x7f38340c75d0 2024-04-20T10:55:49.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7e8d1df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e740039b0 0x7f7e881f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f383ad04640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 0x7f38340c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7e8d1df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e740039b0 0x7f7e881f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48770/0 (socket says 172.21.15.27:48770) 2024-04-20T10:55:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7e8c9de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e8815c260 0x7f7e881e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7e8c9de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e8815c260 0x7f7e881e9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:47720/0 (socket says 172.21.15.27:47720) 2024-04-20T10:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f3839d02640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 0x7f38340c5f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f383000c500 tx=0x7f383000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f79837fe640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 0x7f79840ca9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f797800c530 tx=0x7f797800c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7e8d1df640 1 -- 172.21.15.27:0/3794760230 learned_addr learned my addr 172.21.15.27:0/3794760230 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:49.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7c1e7fc640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7c0c06c9c0 0x7f7c0c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7c2dfc1640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7c0c06c9c0 0x7f7c0c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7c1e7fc640 1 -- 172.21.15.27:0/3637665743 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c28107e60 con 0x7f7c200d2370 2024-04-20T10:55:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7e8d9e0640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e88158840 0x7f7e88158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7c2dfc1640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7c0c06c9c0 0x7f7c0c06ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c24007aa0 tx=0x7f7c24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f7980ff9640 1 -- 172.21.15.27:0/3269285048 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978017070 con 0x7f79840ceb10 2024-04-20T10:55:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7980ff9640 1 -- 172.21.15.27:0/3269285048 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79780035f0 con 0x7f79840ceb10 2024-04-20T10:55:49.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7980ff9640 1 -- 172.21.15.27:0/3269285048 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79780039e0 con 0x7f79840ceb10 2024-04-20T10:55:49.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.734+0000 7f47f57fa640 1 -- 172.21.15.27:0/2209601312 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47e4018070 con 0x7f47f806c750 2024-04-20T10:55:49.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f47f57fa640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47dc06c9c0 0x7f47dc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f47f57fa640 1 -- 172.21.15.27:0/2209601312 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f47e408bab0 con 0x7f47f806c750 2024-04-20T10:55:49.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f47f7fff640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47dc06c9c0 0x7f47dc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79841f6980 con 0x7f79840ceb10 2024-04-20T10:55:49.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79840ca1a0 con 0x7f79840ceb10 2024-04-20T10:55:49.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f4ee6ffd640 1 -- 172.21.15.27:0/841918071 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ef0096440 con 0x7f4ee80e37a0 2024-04-20T10:55:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88163cd0 con 0x7f7e740039b0 2024-04-20T10:55:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881633e0 con 0x7f7e8815c260 2024-04-20T10:55:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7e88163070 con 0x7f7e88158840 2024-04-20T10:55:49.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f4ef73fa640 1 -- 172.21.15.27:0/841918071 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed8005290 con 0x7f4ee80e37a0 2024-04-20T10:55:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7e8d1df640 1 -- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e88158840 msgr2=0x7f7e88158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7e8d1df640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e88158840 0x7f7e88158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7e8d1df640 1 -- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e8815c260 msgr2=0x7f7e881e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7e8d1df640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e8815c260 0x7f7e881e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f7e8d1df640 1 -- 172.21.15.27:0/3794760230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e881f3620 con 0x7f7e740039b0 2024-04-20T10:55:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.735+0000 7f2fd9ffb640 1 -- 172.21.15.27:0/2950665224 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fe004b070 con 0x7f2fe40c87a0 2024-04-20T10:55:49.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.736+0000 7f47f7fff640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47dc06c9c0 0x7f47dc06ee80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47ec036000 tx=0x7f47ec008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.736+0000 7f4ee6ffd640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ed406c8f0 0x7f4ed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.736+0000 7f7e8d1df640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e740039b0 0x7f7e881f30e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7e8402bfd0 tx=0x7f7e84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.736+0000 7f4ee6ffd640 1 -- 172.21.15.27:0/841918071 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4ef01082e0 con 0x7f4ee80e37a0 2024-04-20T10:55:49.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.736+0000 7f2fd9ffb640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2fc806c9c0 0x7f2fc806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.736+0000 7f4ef63f8640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ed406c8f0 0x7f4ed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.736+0000 7f2fd9ffb640 1 -- 172.21.15.27:0/2950665224 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2fe00add20 con 0x7f2fe40c87a0 2024-04-20T10:55:49.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.736+0000 7f2fe8c7b640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2fc806c9c0 0x7f2fc806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.737+0000 7f4ef63f8640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ed406c8f0 0x7f4ed406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009870 tx=0x7f4eec008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.737+0000 7f2fe8c7b640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2fc806c9c0 0x7f2fc806ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2fe41625c0 tx=0x7f2fd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.737+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798404fcf0 con 0x7f79840ceb10 2024-04-20T10:55:49.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f2fd9ffb640 1 -- 172.21.15.27:0/2950665224 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2fe007bae0 con 0x7f2fe40c87a0 2024-04-20T10:55:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f01277fe640 1 -- 172.21.15.27:0/3238808265 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c015020 con 0x7f01300c6320 2024-04-20T10:55:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f382affd640 1 -- 172.21.15.27:0/1951983964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830017070 con 0x7f38340c75d0 2024-04-20T10:55:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f7e7dffb640 1 -- 172.21.15.27:0/3794760230 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84034040 con 0x7f7e740039b0 2024-04-20T10:55:49.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f7c1e7fc640 1 -- 172.21.15.27:0/3637665743 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c200d27f0 con 0x7f7c200d2370 2024-04-20T10:55:49.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e881f93a0 con 0x7f7e740039b0 2024-04-20T10:55:49.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f7e7dffb640 1 -- 172.21.15.27:0/3794760230 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8400f000 con 0x7f7e740039b0 2024-04-20T10:55:49.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01300caf50 con 0x7f01300c6320 2024-04-20T10:55:49.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0130212b60 con 0x7f01300c6320 2024-04-20T10:55:49.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f47f57fa640 1 -- 172.21.15.27:0/2209601312 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f47f80696d0 con 0x7f47f806c750 2024-04-20T10:55:49.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38341f9250 con 0x7f38340c75d0 2024-04-20T10:55:49.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38341f9810 con 0x7f38340c75d0 2024-04-20T10:55:49.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f01277fe640 1 -- 172.21.15.27:0/3238808265 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f011c011830 con 0x7f01300c6320 2024-04-20T10:55:49.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f01277fe640 1 -- 172.21.15.27:0/3238808265 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c015020 con 0x7f01300c6320 2024-04-20T10:55:49.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f7e7dffb640 1 -- 172.21.15.27:0/3794760230 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8403a9e0 con 0x7f7e740039b0 2024-04-20T10:55:49.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.740+0000 7f382affd640 1 -- 172.21.15.27:0/1951983964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3830005ce0 con 0x7f38340c75d0 2024-04-20T10:55:49.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.741+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e880c8270 con 0x7f7e740039b0 2024-04-20T10:55:49.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.741+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c005290 con 0x7f01300c6320 2024-04-20T10:55:49.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.741+0000 7f382affd640 1 -- 172.21.15.27:0/1951983964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300072f0 con 0x7f38340c75d0 2024-04-20T10:55:49.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.742+0000 7f7980ff9640 1 -- 172.21.15.27:0/3269285048 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f797801a070 con 0x7f79840ceb10 2024-04-20T10:55:49.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.742+0000 7f7980ff9640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f797006c8f0 0x7f797006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.743+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381c005290 con 0x7f38340c75d0 2024-04-20T10:55:49.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.743+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e880b5080 con 0x7f7e740039b0 2024-04-20T10:55:49.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.742+0000 7f7980ff9640 1 -- 172.21.15.27:0/3269285048 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f797808b400 con 0x7f79840ceb10 2024-04-20T10:55:49.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.743+0000 7f7983fff640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f797006c8f0 0x7f797006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.745+0000 7f382affd640 1 -- 172.21.15.27:0/1951983964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f383001a070 con 0x7f38340c75d0 2024-04-20T10:55:49.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.744+0000 7f4ee6ffd640 1 -- 172.21.15.27:0/841918071 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4ef00d65e0 con 0x7f4ee80e37a0 2024-04-20T10:55:49.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.745+0000 7f01277fe640 1 -- 172.21.15.27:0/3238808265 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f011c018020 con 0x7f01300c6320 2024-04-20T10:55:49.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.745+0000 7f7983fff640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f797006c8f0 0x7f797006edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f797c002000 tx=0x7f797c0015f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.745+0000 7f7e7dffb640 1 -- 172.21.15.27:0/3794760230 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e84037070 con 0x7f7e740039b0 2024-04-20T10:55:49.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.745+0000 7f01277fe640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f011806caf0 0x7f011806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.746+0000 7f012effd640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f011806caf0 0x7f011806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.746+0000 7f01277fe640 1 -- 172.21.15.27:0/3238808265 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f011c08bca0 con 0x7f01300c6320 2024-04-20T10:55:49.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.746+0000 7f012effd640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f011806caf0 0x7f011806efb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0120036000 tx=0x7f0120008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.745+0000 7f7e7dffb640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7e6c06c920 0x7f7e6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.746+0000 7f7e7dffb640 1 -- 172.21.15.27:0/3794760230 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e840ade60 con 0x7f7e740039b0 2024-04-20T10:55:49.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.746+0000 7f7e8c9de640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7e6c06c920 0x7f7e6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.746+0000 7f382affd640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f381806c920 0x7f381806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:49.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.746+0000 7f7980ff9640 1 -- 172.21.15.27:0/3269285048 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7978021020 con 0x7f79840ceb10 2024-04-20T10:55:49.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.747+0000 7f382affd640 1 -- 172.21.15.27:0/1951983964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f383008b6c0 con 0x7f38340c75d0 2024-04-20T10:55:49.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.747+0000 7f383a503640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f381806c920 0x7f381806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:49.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.747+0000 7f383a503640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f381806c920 0x7f381806ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3824009a30 tx=0x7f3824008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.747+0000 7f7e8c9de640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7e6c06c920 0x7f7e6c06ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e78001eb0 tx=0x7f7e78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:49.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.753+0000 7f01277fe640 1 -- 172.21.15.27:0/3238808265 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f011c01f020 con 0x7f01300c6320 2024-04-20T10:55:49.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.762+0000 7f7e7dffb640 1 -- 172.21.15.27:0/3794760230 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7e880b5080 con 0x7f7e740039b0 2024-04-20T10:55:49.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:49.771+0000 7f382affd640 1 -- 172.21.15.27:0/1951983964 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3830059480 con 0x7f38340c75d0 2024-04-20T10:55:50.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.159+0000 7f47fd8ca640 1 -- 172.21.15.27:0/2209601312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f47f804fcf0 con 0x7f47f806c750 2024-04-20T10:55:50.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.160+0000 7f47f57fa640 1 -- 172.21.15.27:0/2209601312 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47f804fcf0 con 0x7f47f806c750 2024-04-20T10:55:50.161 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:50.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.163+0000 7f47e2ffd640 1 -- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47dc06c9c0 msgr2=0x7f47dc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.164+0000 7f47e2ffd640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47dc06c9c0 0x7f47dc06ee80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47ec036000 tx=0x7f47ec008030 comp rx=0 tx=0).stop 2024-04-20T10:55:50.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.164+0000 7f47e2ffd640 1 -- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f806c750 msgr2=0x7f47f80cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.164+0000 7f47e2ffd640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f806c750 0x7f47f80cd100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c7c0 tx=0x7f47e400cc80 comp rx=0 tx=0).stop 2024-04-20T10:55:50.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.164+0000 7f47e2ffd640 1 -- 172.21.15.27:0/2209601312 shutdown_connections 2024-04-20T10:55:50.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.164+0000 7f47e2ffd640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47dc06c9c0 0x7f47dc06ee80 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47ec036000 tx=0x7f47ec008030 comp rx=0 tx=0).stop 2024-04-20T10:55:50.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.164+0000 7f47e2ffd640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47f80c95a0 0x7f47f813be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.164+0000 7f47e2ffd640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47f80c26b0 0x7f47f81f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.164+0000 7f47e2ffd640 1 --2- 172.21.15.27:0/2209601312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47f806c750 0x7f47f80cd100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.164+0000 7f47e2ffd640 1 -- 172.21.15.27:0/2209601312 >> 172.21.15.27:0/2209601312 conn(0x7f47f80681b0 msgr2=0x7f47f814fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.166+0000 7f47e2ffd640 1 -- 172.21.15.27:0/2209601312 shutdown_connections 2024-04-20T10:55:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.166+0000 7f47e2ffd640 1 -- 172.21.15.27:0/2209601312 wait complete. 2024-04-20T10:55:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.176+0000 7f4ef73fa640 1 -- 172.21.15.27:0/841918071 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4ed8005580 con 0x7f4ee80e37a0 2024-04-20T10:55:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.177+0000 7f4ee6ffd640 1 -- 172.21.15.27:0/841918071 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4ef00d5ee0 con 0x7f4ee80e37a0 2024-04-20T10:55:50.178 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2024-04-20T10:55:50.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.181+0000 7f4ee4ff9640 1 -- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ed406c8f0 msgr2=0x7f4ed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.181+0000 7f4ee4ff9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ed406c8f0 0x7f4ed406edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009870 tx=0x7f4eec008030 comp rx=0 tx=0).stop 2024-04-20T10:55:50.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.181+0000 7f4ee4ff9640 1 -- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80e37a0 msgr2=0x7f4ee80df450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.181+0000 7f4ee4ff9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80e37a0 0x7f4ee80df450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0065480 tx=0x7f4ef0077440 comp rx=0 tx=0).stop 2024-04-20T10:55:50.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.181+0000 7f4ee4ff9640 1 -- 172.21.15.27:0/841918071 shutdown_connections 2024-04-20T10:55:50.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.181+0000 7f4ee4ff9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ed406c8f0 0x7f4ed406edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.182+0000 7f4ee4ff9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4ee80e37a0 0x7f4ee80df450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.182+0000 7f4ee4ff9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ee80dd3c0 0x7f4ee8049c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.182+0000 7f7c0bfff640 1 -- 172.21.15.27:0/3637665743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7c20033f60 con 0x7f7c200d2370 2024-04-20T10:55:50.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.182+0000 7f4ee4ff9640 1 --2- 172.21.15.27:0/841918071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ee804ba60 0x7f4ee80e31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.182+0000 7f4ee4ff9640 1 -- 172.21.15.27:0/841918071 >> 172.21.15.27:0/841918071 conn(0x7f4ee8050180 msgr2=0x7f4ee80d85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.182+0000 7f4ee4ff9640 1 -- 172.21.15.27:0/841918071 shutdown_connections 2024-04-20T10:55:50.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.182+0000 7f4ee4ff9640 1 -- 172.21.15.27:0/841918071 wait complete. 2024-04-20T10:55:50.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.182+0000 7f7c1e7fc640 1 -- 172.21.15.27:0/3637665743 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7c280d5cb0 con 0x7f7c200d2370 2024-04-20T10:55:50.185 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:50.186 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-20T10:55:50.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.184+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7984156dd0 con 0x7f79840ceb10 2024-04-20T10:55:50.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.184+0000 7f7980ff9640 1 -- 172.21.15.27:0/3269285048 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7978059960 con 0x7f79840ceb10 2024-04-20T10:55:50.186 DEBUG:teuthology.parallel:result is None 2024-04-20T10:55:50.187 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:50.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.184+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2fe40c8580 con 0x7f2fe40c87a0 2024-04-20T10:55:50.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.185+0000 7f2fd9ffb640 1 -- 172.21.15.27:0/2950665224 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2fe007fde0 con 0x7f2fe40c87a0 2024-04-20T10:55:50.187 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:50.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 -- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7c0c06c9c0 msgr2=0x7f7c0c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7c0c06c9c0 0x7f7c0c06ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c24007aa0 tx=0x7f7c24009040 comp rx=0 tx=0).stop 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 -- 172.21.15.27:0/3637665743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c200d2370 msgr2=0x7f7c200d9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c200d2370 0x7f7c200d9940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7c2802db60 tx=0x7f7c28069770 comp rx=0 tx=0).stop 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 -- 172.21.15.27:0/3637665743 shutdown_connections 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7c0c06c9c0 0x7f7c0c06ee80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c200d2370 0x7f7c200d9940 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7c200d0f40 0x7f7c200d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 --2- 172.21.15.27:0/3637665743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c14004350 0x7f7c200d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 -- 172.21.15.27:0/3637665743 >> 172.21.15.27:0/3637665743 conn(0x7f7c20051290 msgr2=0x7f7c200070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 -- 172.21.15.27:0/3637665743 shutdown_connections 2024-04-20T10:55:50.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.186+0000 7f7c0bfff640 1 -- 172.21.15.27:0/3637665743 wait complete. 2024-04-20T10:55:50.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.187+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f010c005850 con 0x7f01300c6320 2024-04-20T10:55:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f797006c8f0 msgr2=0x7f797006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f797006c8f0 0x7f797006edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f797c002000 tx=0x7f797c0015f0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 msgr2=0x7f79840ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 0x7f79840ca9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f797800c530 tx=0x7f797800c9f0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 shutdown_connections 2024-04-20T10:55:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f797006c8f0 0x7f797006edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7984153730 0x7f79840caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79840ceb10 0x7f79840ca9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 --2- 172.21.15.27:0/3269285048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79840b48d0 0x7f79841582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 >> 172.21.15.27:0/3269285048 conn(0x7f79840676c0 msgr2=0x7f79841506d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f01277fe640 1 -- 172.21.15.27:0/3238808265 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f011c05a140 con 0x7f01300c6320 2024-04-20T10:55:50.192 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2024-04-20T10:55:50.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 shutdown_connections 2024-04-20T10:55:50.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.188+0000 7f798a68c640 1 -- 172.21.15.27:0/3269285048 wait complete. 2024-04-20T10:55:50.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2fc806c9c0 msgr2=0x7f2fc806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2fc806c9c0 0x7f2fc806ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2fe41625c0 tx=0x7f2fd4008040 comp rx=0 tx=0).stop 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 msgr2=0x7f2fe40c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c98d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0000c00 tx=0x7f2fe0004730 comp rx=0 tx=0).stop 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 shutdown_connections 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2fc806c9c0 0x7f2fc806ee80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2fe4160160 0x7f2fe41516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fe4156100 0x7f2fe4073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 --2- 172.21.15.27:0/2950665224 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2fe40c87a0 0x7f2fe40c98d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 >> 172.21.15.27:0/2950665224 conn(0x7f2fe40cc920 msgr2=0x7f2fe415dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 shutdown_connections 2024-04-20T10:55:50.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7e881627a0 con 0x7f7e740039b0 2024-04-20T10:55:50.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.189+0000 7f2fea47e640 1 -- 172.21.15.27:0/2950665224 wait complete. 2024-04-20T10:55:50.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.190+0000 7f7e7dffb640 1 -- 172.21.15.27:0/3794760230 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e881627a0 con 0x7f7e740039b0 2024-04-20T10:55:50.196 INFO:teuthology.orchestra.run.smithi027.stdout:55834574856 2024-04-20T10:55:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.191+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f011806caf0 msgr2=0x7f011806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.191+0000 7f0135550640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f011806caf0 0x7f011806efb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0120036000 tx=0x7f0120008030 comp rx=0 tx=0).stop 2024-04-20T10:55:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.191+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300c6320 msgr2=0x7f0130213080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.191+0000 7f0135550640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300c6320 0x7f0130213080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f011c00b410 tx=0x7f011c00b8d0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.192+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 shutdown_connections 2024-04-20T10:55:50.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.192+0000 7f0135550640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f011806caf0 0x7f011806efb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.192+0000 7f0135550640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f01300d1070 0x7f0130150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.192+0000 7f0135550640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01300c6320 0x7f0130213080 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.192+0000 7f0135550640 1 --2- 172.21.15.27:0/3238808265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01300c5e40 0x7f013020f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.192+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 >> 172.21.15.27:0/3238808265 conn(0x7f01300678a0 msgr2=0x7f01301eb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.192+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 shutdown_connections 2024-04-20T10:55:50.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.192+0000 7f0135550640 1 -- 172.21.15.27:0/3238808265 wait complete. 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.193+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7e6c06c920 msgr2=0x7f7e6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.193+0000 7f7e8e1e1640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7e6c06c920 0x7f7e6c06ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e78001eb0 tx=0x7f7e78009040 comp rx=0 tx=0).stop 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e740039b0 msgr2=0x7f7e881f30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e740039b0 0x7f7e881f30e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7e8402bfd0 tx=0x7f7e84004730 comp rx=0 tx=0).stop 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 shutdown_connections 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7e6c06c920 0x7f7e6c06ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7e88158840 0x7f7e88158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e8815c260 0x7f7e881e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 --2- 172.21.15.27:0/3794760230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e740039b0 0x7f7e881f30e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 >> 172.21.15.27:0/3794760230 conn(0x7f7e8806cee0 msgr2=0x7f7e880676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 shutdown_connections 2024-04-20T10:55:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.194+0000 7f7e8e1e1640 1 -- 172.21.15.27:0/3794760230 wait complete. 2024-04-20T10:55:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.199+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f381c005850 con 0x7f38340c75d0 2024-04-20T10:55:50.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.200+0000 7f382affd640 1 -- 172.21.15.27:0/1951983964 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f383005d780 con 0x7f38340c75d0 2024-04-20T10:55:50.201 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2024-04-20T10:55:50.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-20T10:55:50.202 DEBUG:teuthology.parallel:result is None 2024-04-20T10:55:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f381806c920 msgr2=0x7f381806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f381806c920 0x7f381806ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3824009a30 tx=0x7f3824008040 comp rx=0 tx=0).stop 2024-04-20T10:55:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 msgr2=0x7f38340c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 0x7f38340c5f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f383000c500 tx=0x7f383000c9c0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 shutdown_connections 2024-04-20T10:55:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f381806c920 0x7f381806ede0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3834159c00 0x7f38340c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38340c75d0 0x7f38340c5f00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 --2- 172.21.15.27:0/1951983964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3834071a60 0x7f38340cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 >> 172.21.15.27:0/1951983964 conn(0x7f38340ccbd0 msgr2=0x7f38340c79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 shutdown_connections 2024-04-20T10:55:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.203+0000 7f383c767640 1 -- 172.21.15.27:0/1951983964 wait complete. 2024-04-20T10:55:50.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-20T10:55:50.212 DEBUG:teuthology.parallel:result is None 2024-04-20T10:55:50.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-20T10:55:50.213 DEBUG:teuthology.parallel:result is None 2024-04-20T10:55:50.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-20T10:55:50.214 DEBUG:teuthology.parallel:result is None 2024-04-20T10:55:50.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-04-20T10:55:50.215 DEBUG:teuthology.parallel:result is None 2024-04-20T10:55:50.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2024-04-20T10:55:50.216 DEBUG:teuthology.parallel:result is None 2024-04-20T10:55:50.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-20T10:55:50.218 DEBUG:teuthology.parallel:result is None 2024-04-20T10:55:50.218 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T10:55:50.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:55:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.357+0000 7f3bc4681640 1 Processor -- start 2024-04-20T10:55:50.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.357+0000 7f3bc4681640 1 -- start start 2024-04-20T10:55:50.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc4681640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3bbc15b580 0x7f3bbc070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc4681640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15f580 0x7f3bbc0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc4681640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bbc0c9e60 con 0x7f3bbc1556f0 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc4681640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3bbc06ead0 con 0x7f3bbc15b580 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc4681640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f3bbc15b9c0 con 0x7f3bbc15f580 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc1c1c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3bbc15b580 0x7f3bbc070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc241d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3bbc1556f0 0x7f3bbc0714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53468/0 (socket says 172.21.15.27:53468) 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc241d640 1 -- 172.21.15.27:0/1655948560 learned_addr learned my addr 172.21.15.27:0/1655948560 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc1c1c640 1 -- 172.21.15.27:0/1655948560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15f580 msgr2=0x7f3bbc0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc1c1c640 1 --2- 172.21.15.27:0/1655948560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15f580 0x7f3bbc0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc1c1c640 1 -- 172.21.15.27:0/1655948560 >> v1:172.21.15.27:6789/0 conn(0x7f3bbc1556f0 legacy=0x7f3bbc0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc1c1c640 1 -- 172.21.15.27:0/1655948560 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc06cce0 con 0x7f3bbc15b580 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.358+0000 7f3bc141b640 1 -- 172.21.15.27:0/1655948560 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3771710719 0 0) 0x7f3bbc0c9e60 con 0x7f3bbc1556f0 2024-04-20T10:55:50.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.359+0000 7f3bc1c1c640 1 --2- 172.21.15.27:0/1655948560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15b580 0x7f3bbc070ca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0097a0 tx=0x7f3bac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911a164dcaf6bcee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:50.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.359+0000 7f3bc141b640 1 -- 172.21.15.27:0/1655948560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0315a0 con 0x7f3bbc15b580 2024-04-20T10:55:50.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.359+0000 7f3bc141b640 1 -- 172.21.15.27:0/1655948560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bac037070 con 0x7f3bbc15b580 2024-04-20T10:55:50.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.359+0000 7f3bc141b640 1 -- 172.21.15.27:0/1655948560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac031960 con 0x7f3bbc15b580 2024-04-20T10:55:50.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.360+0000 7f3bc4681640 1 -- 172.21.15.27:0/1655948560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15b580 msgr2=0x7f3bbc070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.360+0000 7f3bc4681640 1 --2- 172.21.15.27:0/1655948560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15b580 0x7f3bbc070ca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0097a0 tx=0x7f3bac02b840 comp rx=0 tx=0).stop 2024-04-20T10:55:50.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.360+0000 7f3bc4681640 1 -- 172.21.15.27:0/1655948560 shutdown_connections 2024-04-20T10:55:50.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.360+0000 7f3bc4681640 1 --2- 172.21.15.27:0/1655948560 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15f580 0x7f3bbc0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.360+0000 7f3bc4681640 1 --2- 172.21.15.27:0/1655948560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15b580 0x7f3bbc070ca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.360+0000 7f3bc4681640 1 -- 172.21.15.27:0/1655948560 >> 172.21.15.27:0/1655948560 conn(0x7f3bbc067ae0 msgr2=0x7f3bbc067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.360+0000 7f3bc4681640 1 -- 172.21.15.27:0/1655948560 shutdown_connections 2024-04-20T10:55:50.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.360+0000 7f3bc4681640 1 -- 172.21.15.27:0/1655948560 wait complete. 2024-04-20T10:55:50.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.361+0000 7f3bc4681640 1 Processor -- start 2024-04-20T10:55:50.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.361+0000 7f3bc4681640 1 -- start start 2024-04-20T10:55:50.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc4681640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bbc1556f0 0x7f3bbc1f1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc4681640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15b580 0x7f3bbc1f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc4681640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15f580 0x7f3bbc0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc4681640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc06e480 con 0x7f3bbc15f580 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc4681640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc0c9b80 con 0x7f3bbc1556f0 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc4681640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f3bbc06e760 con 0x7f3bbc15b580 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc2c1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15f580 0x7f3bbc0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc2c1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15f580 0x7f3bbc0cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48796/0 (socket says 172.21.15.27:48796) 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc2c1e640 1 -- 172.21.15.27:0/1419804717 learned_addr learned my addr 172.21.15.27:0/1419804717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc241d640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bbc1556f0 0x7f3bbc1f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc1c1c640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15b580 0x7f3bbc1f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc2c1e640 1 -- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15b580 msgr2=0x7f3bbc1f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc2c1e640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15b580 0x7f3bbc1f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc2c1e640 1 -- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bbc1556f0 msgr2=0x7f3bbc1f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc2c1e640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bbc1556f0 0x7f3bbc1f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc2c1e640 1 -- 172.21.15.27:0/1419804717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc14fde0 con 0x7f3bbc15f580 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc1c1c640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15b580 0x7f3bbc1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.362+0000 7f3bc2c1e640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15f580 0x7f3bbc0cf510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400b9a0 tx=0x7f3bb400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.363+0000 7f3bb2ffd640 1 -- 172.21.15.27:0/1419804717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4019070 con 0x7f3bbc15f580 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.363+0000 7f3bb2ffd640 1 -- 172.21.15.27:0/1419804717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb400cbc0 con 0x7f3bbc15f580 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.363+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bbc1500c0 con 0x7f3bbc15f580 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.363+0000 7f3bb2ffd640 1 -- 172.21.15.27:0/1419804717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4014440 con 0x7f3bbc15f580 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.363+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bbc1f4950 con 0x7f3bbc15f580 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.364+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bbc04fcf0 con 0x7f3bbc15f580 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.364+0000 7f3bb2ffd640 1 -- 172.21.15.27:0/1419804717 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3bb401c070 con 0x7f3bbc15f580 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.364+0000 7f3bb2ffd640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3ba006c8f0 0x7f3ba006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.364+0000 7f3bb2ffd640 1 -- 172.21.15.27:0/1419804717 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3bb408b900 con 0x7f3bbc15f580 2024-04-20T10:55:50.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.365+0000 7f3bc241d640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3ba006c8f0 0x7f3ba006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.365+0000 7f3bc241d640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3ba006c8f0 0x7f3ba006edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000f60 tx=0x7f3bb8007c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:50.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.369+0000 7f3bb2ffd640 1 -- 172.21.15.27:0/1419804717 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3bb4059ca0 con 0x7f3bbc15f580 2024-04-20T10:55:50.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.550+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3bbc0712c0 con 0x7f3ba006c8f0 2024-04-20T10:55:50.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.556+0000 7f3bb2ffd640 1 -- 172.21.15.27:0/1419804717 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7f3bbc0712c0 con 0x7f3ba006c8f0 2024-04-20T10:55:50.560 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:50.562 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3ba006c8f0 msgr2=0x7f3ba006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3ba006c8f0 0x7f3ba006edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000f60 tx=0x7f3bb8007c00 comp rx=0 tx=0).stop 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15f580 msgr2=0x7f3bbc0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15f580 0x7f3bbc0cf510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400b9a0 tx=0x7f3bb400be60 comp rx=0 tx=0).stop 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 shutdown_connections 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3ba006c8f0 0x7f3ba006edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bbc15f580 0x7f3bbc0cf510 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3bbc15b580 0x7f3bbc1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 --2- 172.21.15.27:0/1419804717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3bbc1556f0 0x7f3bbc1f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 >> 172.21.15.27:0/1419804717 conn(0x7f3bbc067ae0 msgr2=0x7f3bbc0c9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 shutdown_connections 2024-04-20T10:55:50.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.560+0000 7f3bc4681640 1 -- 172.21.15.27:0/1419804717 wait complete. 2024-04-20T10:55:50.571 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-20T10:55:49.703733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218952,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755768,"statfs":{"total":773094113280,"available":772869906432,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.926413"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287393+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T10:55:44.287393+0000","last_peered":"2024-04-20T10:55:44.287393+0000","last_clean":"2024-04-20T10:55:44.287393+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T10:55:44.287393+0000","last_undegraded":"2024-04-20T10:55:44.287393+0000","last_fullsized":"2024-04-20T10:55:44.287393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797629+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T10:55:43.797629+0000","last_peered":"2024-04-20T10:55:43.797629+0000","last_clean":"2024-04-20T10:55:43.797629+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T10:55:43.797629+0000","last_undegraded":"2024-04-20T10:55:43.797629+0000","last_fullsized":"2024-04-20T10:55:43.797629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285486+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T10:55:44.285486+0000","last_peered":"2024-04-20T10:55:44.285486+0000","last_clean":"2024-04-20T10:55:44.285486+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T10:55:44.285486+0000","last_undegraded":"2024-04-20T10:55:44.285486+0000","last_fullsized":"2024-04-20T10:55:44.285486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286383+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T10:55:44.286383+0000","last_peered":"2024-04-20T10:55:44.286383+0000","last_clean":"2024-04-20T10:55:44.286383+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T10:55:44.286383+0000","last_undegraded":"2024-04-20T10:55:44.286383+0000","last_fullsized":"2024-04-20T10:55:44.286383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286289+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T10:55:44.286289+0000","last_peered":"2024-04-20T10:55:44.286289+0000","last_clean":"2024-04-20T10:55:44.286289+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T10:55:44.286289+0000","last_undegraded":"2024-04-20T10:55:44.286289+0000","last_fullsized":"2024-04-20T10:55:44.286289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287329+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T10:55:44.287329+0000","last_peered":"2024-04-20T10:55:44.287329+0000","last_clean":"2024-04-20T10:55:44.287329+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T10:55:44.287329+0000","last_undegraded":"2024-04-20T10:55:44.287329+0000","last_fullsized":"2024-04-20T10:55:44.287329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287439+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T10:55:44.287439+0000","last_peered":"2024-04-20T10:55:44.287439+0000","last_clean":"2024-04-20T10:55:44.287439+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T10:55:44.287439+0000","last_undegraded":"2024-04-20T10:55:44.287439+0000","last_fullsized":"2024-04-20T10:55:44.287439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792859+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T10:55:43.792859+0000","last_peered":"2024-04-20T10:55:43.792859+0000","last_clean":"2024-04-20T10:55:43.792859+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T10:55:43.792859+0000","last_undegraded":"2024-04-20T10:55:43.792859+0000","last_fullsized":"2024-04-20T10:55:43.792859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286907+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T10:55:44.286907+0000","last_peered":"2024-04-20T10:55:44.286907+0000","last_clean":"2024-04-20T10:55:44.286907+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T10:55:44.286907+0000","last_undegraded":"2024-04-20T10:55:44.286907+0000","last_fullsized":"2024-04-20T10:55:44.286907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286541+0000","last_change":"2024-04-20T10:55:42.802716+0000","last_active":"2024-04-20T10:55:44.286541+0000","last_peered":"2024-04-20T10:55:44.286541+0000","last_clean":"2024-04-20T10:55:44.286541+0000","last_became_active":"2024-04-20T10:55:42.802418+0000","last_became_peered":"2024-04-20T10:55:42.802418+0000","last_unstale":"2024-04-20T10:55:44.286541+0000","last_undegraded":"2024-04-20T10:55:44.286541+0000","last_fullsized":"2024-04-20T10:55:44.286541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:12:22.186243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286228+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T10:55:44.286228+0000","last_peered":"2024-04-20T10:55:44.286228+0000","last_clean":"2024-04-20T10:55:44.286228+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T10:55:44.286228+0000","last_undegraded":"2024-04-20T10:55:44.286228+0000","last_fullsized":"2024-04-20T10:55:44.286228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801002+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T10:55:43.801002+0000","last_peered":"2024-04-20T10:55:43.801002+0000","last_clean":"2024-04-20T10:55:43.801002+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T10:55:43.801002+0000","last_undegraded":"2024-04-20T10:55:43.801002+0000","last_fullsized":"2024-04-20T10:55:43.801002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793050+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T10:55:43.793050+0000","last_peered":"2024-04-20T10:55:43.793050+0000","last_clean":"2024-04-20T10:55:43.793050+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T10:55:43.793050+0000","last_undegraded":"2024-04-20T10:55:43.793050+0000","last_fullsized":"2024-04-20T10:55:43.793050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797332+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T10:55:43.797332+0000","last_peered":"2024-04-20T10:55:43.797332+0000","last_clean":"2024-04-20T10:55:43.797332+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T10:55:43.797332+0000","last_undegraded":"2024-04-20T10:55:43.797332+0000","last_fullsized":"2024-04-20T10:55:43.797332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285851+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T10:55:44.285851+0000","last_peered":"2024-04-20T10:55:44.285851+0000","last_clean":"2024-04-20T10:55:44.285851+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T10:55:44.285851+0000","last_undegraded":"2024-04-20T10:55:44.285851+0000","last_fullsized":"2024-04-20T10:55:44.285851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286491+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T10:55:44.286491+0000","last_peered":"2024-04-20T10:55:44.286491+0000","last_clean":"2024-04-20T10:55:44.286491+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T10:55:44.286491+0000","last_undegraded":"2024-04-20T10:55:44.286491+0000","last_fullsized":"2024-04-20T10:55:44.286491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285217+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T10:55:44.285217+0000","last_peered":"2024-04-20T10:55:44.285217+0000","last_clean":"2024-04-20T10:55:44.285217+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T10:55:44.285217+0000","last_undegraded":"2024-04-20T10:55:44.285217+0000","last_fullsized":"2024-04-20T10:55:44.285217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287175+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T10:55:44.287175+0000","last_peered":"2024-04-20T10:55:44.287175+0000","last_clean":"2024-04-20T10:55:44.287175+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T10:55:44.287175+0000","last_undegraded":"2024-04-20T10:55:44.287175+0000","last_fullsized":"2024-04-20T10:55:44.287175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286994+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T10:55:44.286994+0000","last_peered":"2024-04-20T10:55:44.286994+0000","last_clean":"2024-04-20T10:55:44.286994+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T10:55:44.286994+0000","last_undegraded":"2024-04-20T10:55:44.286994+0000","last_fullsized":"2024-04-20T10:55:44.286994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285751+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T10:55:44.285751+0000","last_peered":"2024-04-20T10:55:44.285751+0000","last_clean":"2024-04-20T10:55:44.285751+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T10:55:44.285751+0000","last_undegraded":"2024-04-20T10:55:44.285751+0000","last_fullsized":"2024-04-20T10:55:44.285751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287484+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T10:55:44.287484+0000","last_peered":"2024-04-20T10:55:44.287484+0000","last_clean":"2024-04-20T10:55:44.287484+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T10:55:44.287484+0000","last_undegraded":"2024-04-20T10:55:44.287484+0000","last_fullsized":"2024-04-20T10:55:44.287484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792921+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T10:55:43.792921+0000","last_peered":"2024-04-20T10:55:43.792921+0000","last_clean":"2024-04-20T10:55:43.792921+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T10:55:43.792921+0000","last_undegraded":"2024-04-20T10:55:43.792921+0000","last_fullsized":"2024-04-20T10:55:43.792921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793499+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T10:55:43.793499+0000","last_peered":"2024-04-20T10:55:43.793499+0000","last_clean":"2024-04-20T10:55:43.793499+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T10:55:43.793499+0000","last_undegraded":"2024-04-20T10:55:43.793499+0000","last_fullsized":"2024-04-20T10:55:43.793499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286683+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T10:55:44.286683+0000","last_peered":"2024-04-20T10:55:44.286683+0000","last_clean":"2024-04-20T10:55:44.286683+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T10:55:44.286683+0000","last_undegraded":"2024-04-20T10:55:44.286683+0000","last_fullsized":"2024-04-20T10:55:44.286683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797439+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T10:55:43.797439+0000","last_peered":"2024-04-20T10:55:43.797439+0000","last_clean":"2024-04-20T10:55:43.797439+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T10:55:43.797439+0000","last_undegraded":"2024-04-20T10:55:43.797439+0000","last_fullsized":"2024-04-20T10:55:43.797439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793232+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T10:55:43.793232+0000","last_peered":"2024-04-20T10:55:43.793232+0000","last_clean":"2024-04-20T10:55:43.793232+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T10:55:43.793232+0000","last_undegraded":"2024-04-20T10:55:43.793232+0000","last_fullsized":"2024-04-20T10:55:43.793232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285757+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T10:55:44.285757+0000","last_peered":"2024-04-20T10:55:44.285757+0000","last_clean":"2024-04-20T10:55:44.285757+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T10:55:44.285757+0000","last_undegraded":"2024-04-20T10:55:44.285757+0000","last_fullsized":"2024-04-20T10:55:44.285757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287493+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T10:55:44.287493+0000","last_peered":"2024-04-20T10:55:44.287493+0000","last_clean":"2024-04-20T10:55:44.287493+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T10:55:44.287493+0000","last_undegraded":"2024-04-20T10:55:44.287493+0000","last_fullsized":"2024-04-20T10:55:44.287493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285309+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T10:55:44.285309+0000","last_peered":"2024-04-20T10:55:44.285309+0000","last_clean":"2024-04-20T10:55:44.285309+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T10:55:44.285309+0000","last_undegraded":"2024-04-20T10:55:44.285309+0000","last_fullsized":"2024-04-20T10:55:44.285309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285746+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T10:55:44.285746+0000","last_peered":"2024-04-20T10:55:44.285746+0000","last_clean":"2024-04-20T10:55:44.285746+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T10:55:44.285746+0000","last_undegraded":"2024-04-20T10:55:44.285746+0000","last_fullsized":"2024-04-20T10:55:44.285746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800661+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T10:55:43.800661+0000","last_peered":"2024-04-20T10:55:43.800661+0000","last_clean":"2024-04-20T10:55:43.800661+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T10:55:43.800661+0000","last_undegraded":"2024-04-20T10:55:43.800661+0000","last_fullsized":"2024-04-20T10:55:43.800661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.722181+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T10:55:44.722181+0000","last_peered":"2024-04-20T10:55:44.722181+0000","last_clean":"2024-04-20T10:55:44.722181+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T10:55:44.722181+0000","last_undegraded":"2024-04-20T10:55:44.722181+0000","last_fullsized":"2024-04-20T10:55:44.722181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285840+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T10:55:44.285840+0000","last_peered":"2024-04-20T10:55:44.285840+0000","last_clean":"2024-04-20T10:55:44.285840+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T10:55:44.285840+0000","last_undegraded":"2024-04-20T10:55:44.285840+0000","last_fullsized":"2024-04-20T10:55:44.285840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.724680+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T10:55:44.724680+0000","last_peered":"2024-04-20T10:55:44.724680+0000","last_clean":"2024-04-20T10:55:44.724680+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T10:55:44.724680+0000","last_undegraded":"2024-04-20T10:55:44.724680+0000","last_fullsized":"2024-04-20T10:55:44.724680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793178+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T10:55:43.793178+0000","last_peered":"2024-04-20T10:55:43.793178+0000","last_clean":"2024-04-20T10:55:43.793178+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T10:55:43.793178+0000","last_undegraded":"2024-04-20T10:55:43.793178+0000","last_fullsized":"2024-04-20T10:55:43.793178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792877+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T10:55:43.792877+0000","last_peered":"2024-04-20T10:55:43.792877+0000","last_clean":"2024-04-20T10:55:43.792877+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T10:55:43.792877+0000","last_undegraded":"2024-04-20T10:55:43.792877+0000","last_fullsized":"2024-04-20T10:55:43.792877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286112+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T10:55:44.286112+0000","last_peered":"2024-04-20T10:55:44.286112+0000","last_clean":"2024-04-20T10:55:44.286112+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T10:55:44.286112+0000","last_undegraded":"2024-04-20T10:55:44.286112+0000","last_fullsized":"2024-04-20T10:55:44.286112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797544+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T10:55:43.797544+0000","last_peered":"2024-04-20T10:55:43.797544+0000","last_clean":"2024-04-20T10:55:43.797544+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T10:55:43.797544+0000","last_undegraded":"2024-04-20T10:55:43.797544+0000","last_fullsized":"2024-04-20T10:55:43.797544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286640+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T10:55:44.286640+0000","last_peered":"2024-04-20T10:55:44.286640+0000","last_clean":"2024-04-20T10:55:44.286640+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T10:55:44.286640+0000","last_undegraded":"2024-04-20T10:55:44.286640+0000","last_fullsized":"2024-04-20T10:55:44.286640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287062+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T10:55:44.287062+0000","last_peered":"2024-04-20T10:55:44.287062+0000","last_clean":"2024-04-20T10:55:44.287062+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T10:55:44.287062+0000","last_undegraded":"2024-04-20T10:55:44.287062+0000","last_fullsized":"2024-04-20T10:55:44.287062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287172+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T10:55:44.287172+0000","last_peered":"2024-04-20T10:55:44.287172+0000","last_clean":"2024-04-20T10:55:44.287172+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T10:55:44.287172+0000","last_undegraded":"2024-04-20T10:55:44.287172+0000","last_fullsized":"2024-04-20T10:55:44.287172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285669+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T10:55:44.285669+0000","last_peered":"2024-04-20T10:55:44.285669+0000","last_clean":"2024-04-20T10:55:44.285669+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T10:55:44.285669+0000","last_undegraded":"2024-04-20T10:55:44.285669+0000","last_fullsized":"2024-04-20T10:55:44.285669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.786854+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T10:55:47.786854+0000","last_peered":"2024-04-20T10:55:47.786854+0000","last_clean":"2024-04-20T10:55:47.786854+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T10:55:47.786854+0000","last_undegraded":"2024-04-20T10:55:47.786854+0000","last_fullsized":"2024-04-20T10:55:47.786854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801052+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T10:55:43.801052+0000","last_peered":"2024-04-20T10:55:43.801052+0000","last_clean":"2024-04-20T10:55:43.801052+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T10:55:43.801052+0000","last_undegraded":"2024-04-20T10:55:43.801052+0000","last_fullsized":"2024-04-20T10:55:43.801052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.794272+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T10:55:47.794272+0000","last_peered":"2024-04-20T10:55:47.794272+0000","last_clean":"2024-04-20T10:55:47.794272+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T10:55:47.794272+0000","last_undegraded":"2024-04-20T10:55:47.794272+0000","last_fullsized":"2024-04-20T10:55:47.794272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287336+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T10:55:44.287336+0000","last_peered":"2024-04-20T10:55:44.287336+0000","last_clean":"2024-04-20T10:55:44.287336+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T10:55:44.287336+0000","last_undegraded":"2024-04-20T10:55:44.287336+0000","last_fullsized":"2024-04-20T10:55:44.287336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.786016+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T10:55:47.786016+0000","last_peered":"2024-04-20T10:55:47.786016+0000","last_clean":"2024-04-20T10:55:47.786016+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T10:55:47.786016+0000","last_undegraded":"2024-04-20T10:55:47.786016+0000","last_fullsized":"2024-04-20T10:55:47.786016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285622+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T10:55:44.285622+0000","last_peered":"2024-04-20T10:55:44.285622+0000","last_clean":"2024-04-20T10:55:44.285622+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T10:55:44.285622+0000","last_undegraded":"2024-04-20T10:55:44.285622+0000","last_fullsized":"2024-04-20T10:55:44.285622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287576+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T10:55:44.287576+0000","last_peered":"2024-04-20T10:55:44.287576+0000","last_clean":"2024-04-20T10:55:44.287576+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T10:55:44.287576+0000","last_undegraded":"2024-04-20T10:55:44.287576+0000","last_fullsized":"2024-04-20T10:55:44.287576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286631+0000","last_change":"2024-04-20T10:55:42.805169+0000","last_active":"2024-04-20T10:55:44.286631+0000","last_peered":"2024-04-20T10:55:44.286631+0000","last_clean":"2024-04-20T10:55:44.286631+0000","last_became_active":"2024-04-20T10:55:42.804573+0000","last_became_peered":"2024-04-20T10:55:42.804573+0000","last_unstale":"2024-04-20T10:55:44.286631+0000","last_undegraded":"2024-04-20T10:55:44.286631+0000","last_fullsized":"2024-04-20T10:55:44.286631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:16.103447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287223+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T10:55:44.287223+0000","last_peered":"2024-04-20T10:55:44.287223+0000","last_clean":"2024-04-20T10:55:44.287223+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T10:55:44.287223+0000","last_undegraded":"2024-04-20T10:55:44.287223+0000","last_fullsized":"2024-04-20T10:55:44.287223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285762+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T10:55:44.285762+0000","last_peered":"2024-04-20T10:55:44.285762+0000","last_clean":"2024-04-20T10:55:44.285762+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T10:55:44.285762+0000","last_undegraded":"2024-04-20T10:55:44.285762+0000","last_fullsized":"2024-04-20T10:55:44.285762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.791669+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T10:55:47.791669+0000","last_peered":"2024-04-20T10:55:47.791669+0000","last_clean":"2024-04-20T10:55:47.791669+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T10:55:47.791669+0000","last_undegraded":"2024-04-20T10:55:47.791669+0000","last_fullsized":"2024-04-20T10:55:47.791669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287518+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T10:55:44.287518+0000","last_peered":"2024-04-20T10:55:44.287518+0000","last_clean":"2024-04-20T10:55:44.287518+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T10:55:44.287518+0000","last_undegraded":"2024-04-20T10:55:44.287518+0000","last_fullsized":"2024-04-20T10:55:44.287518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.722087+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T10:55:44.722087+0000","last_peered":"2024-04-20T10:55:44.722087+0000","last_clean":"2024-04-20T10:55:44.722087+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T10:55:44.722087+0000","last_undegraded":"2024-04-20T10:55:44.722087+0000","last_fullsized":"2024-04-20T10:55:44.722087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286460+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T10:55:44.286460+0000","last_peered":"2024-04-20T10:55:44.286460+0000","last_clean":"2024-04-20T10:55:44.286460+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T10:55:44.286460+0000","last_undegraded":"2024-04-20T10:55:44.286460+0000","last_fullsized":"2024-04-20T10:55:44.286460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.789279+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T10:55:47.789279+0000","last_peered":"2024-04-20T10:55:47.789279+0000","last_clean":"2024-04-20T10:55:47.789279+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T10:55:47.789279+0000","last_undegraded":"2024-04-20T10:55:47.789279+0000","last_fullsized":"2024-04-20T10:55:47.789279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287067+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T10:55:44.287067+0000","last_peered":"2024-04-20T10:55:44.287067+0000","last_clean":"2024-04-20T10:55:44.287067+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T10:55:44.287067+0000","last_undegraded":"2024-04-20T10:55:44.287067+0000","last_fullsized":"2024-04-20T10:55:44.287067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.731055+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T10:55:44.731055+0000","last_peered":"2024-04-20T10:55:44.731055+0000","last_clean":"2024-04-20T10:55:44.731055+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T10:55:44.731055+0000","last_undegraded":"2024-04-20T10:55:44.731055+0000","last_fullsized":"2024-04-20T10:55:44.731055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286721+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T10:55:44.286721+0000","last_peered":"2024-04-20T10:55:44.286721+0000","last_clean":"2024-04-20T10:55:44.286721+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T10:55:44.286721+0000","last_undegraded":"2024-04-20T10:55:44.286721+0000","last_fullsized":"2024-04-20T10:55:44.286721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285213+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T10:55:44.285213+0000","last_peered":"2024-04-20T10:55:44.285213+0000","last_clean":"2024-04-20T10:55:44.285213+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T10:55:44.285213+0000","last_undegraded":"2024-04-20T10:55:44.285213+0000","last_fullsized":"2024-04-20T10:55:44.285213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286934+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T10:55:44.286934+0000","last_peered":"2024-04-20T10:55:44.286934+0000","last_clean":"2024-04-20T10:55:44.286934+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T10:55:44.286934+0000","last_undegraded":"2024-04-20T10:55:44.286934+0000","last_fullsized":"2024-04-20T10:55:44.286934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286525+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T10:55:44.286525+0000","last_peered":"2024-04-20T10:55:44.286525+0000","last_clean":"2024-04-20T10:55:44.286525+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T10:55:44.286525+0000","last_undegraded":"2024-04-20T10:55:44.286525+0000","last_fullsized":"2024-04-20T10:55:44.286525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793252+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T10:55:43.793252+0000","last_peered":"2024-04-20T10:55:43.793252+0000","last_clean":"2024-04-20T10:55:43.793252+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T10:55:43.793252+0000","last_undegraded":"2024-04-20T10:55:43.793252+0000","last_fullsized":"2024-04-20T10:55:43.793252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287230+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T10:55:44.287230+0000","last_peered":"2024-04-20T10:55:44.287230+0000","last_clean":"2024-04-20T10:55:44.287230+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T10:55:44.287230+0000","last_undegraded":"2024-04-20T10:55:44.287230+0000","last_fullsized":"2024-04-20T10:55:44.287230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285651+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T10:55:44.285651+0000","last_peered":"2024-04-20T10:55:44.285651+0000","last_clean":"2024-04-20T10:55:44.285651+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T10:55:44.285651+0000","last_undegraded":"2024-04-20T10:55:44.285651+0000","last_fullsized":"2024-04-20T10:55:44.285651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285287+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T10:55:44.285287+0000","last_peered":"2024-04-20T10:55:44.285287+0000","last_clean":"2024-04-20T10:55:44.285287+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T10:55:44.285287+0000","last_undegraded":"2024-04-20T10:55:44.285287+0000","last_fullsized":"2024-04-20T10:55:44.285287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797319+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T10:55:43.797319+0000","last_peered":"2024-04-20T10:55:43.797319+0000","last_clean":"2024-04-20T10:55:43.797319+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T10:55:43.797319+0000","last_undegraded":"2024-04-20T10:55:43.797319+0000","last_fullsized":"2024-04-20T10:55:43.797319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287600+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T10:55:44.287600+0000","last_peered":"2024-04-20T10:55:44.287600+0000","last_clean":"2024-04-20T10:55:44.287600+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T10:55:44.287600+0000","last_undegraded":"2024-04-20T10:55:44.287600+0000","last_fullsized":"2024-04-20T10:55:44.287600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800914+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T10:55:43.800914+0000","last_peered":"2024-04-20T10:55:43.800914+0000","last_clean":"2024-04-20T10:55:43.800914+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T10:55:43.800914+0000","last_undegraded":"2024-04-20T10:55:43.800914+0000","last_fullsized":"2024-04-20T10:55:43.800914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797497+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T10:55:43.797497+0000","last_peered":"2024-04-20T10:55:43.797497+0000","last_clean":"2024-04-20T10:55:43.797497+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T10:55:43.797497+0000","last_undegraded":"2024-04-20T10:55:43.797497+0000","last_fullsized":"2024-04-20T10:55:43.797497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287087+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T10:55:44.287087+0000","last_peered":"2024-04-20T10:55:44.287087+0000","last_clean":"2024-04-20T10:55:44.287087+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T10:55:44.287087+0000","last_undegraded":"2024-04-20T10:55:44.287087+0000","last_fullsized":"2024-04-20T10:55:44.287087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287263+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T10:55:44.287263+0000","last_peered":"2024-04-20T10:55:44.287263+0000","last_clean":"2024-04-20T10:55:44.287263+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T10:55:44.287263+0000","last_undegraded":"2024-04-20T10:55:44.287263+0000","last_fullsized":"2024-04-20T10:55:44.287263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797411+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T10:55:43.797411+0000","last_peered":"2024-04-20T10:55:43.797411+0000","last_clean":"2024-04-20T10:55:43.797411+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T10:55:43.797411+0000","last_undegraded":"2024-04-20T10:55:43.797411+0000","last_fullsized":"2024-04-20T10:55:43.797411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797512+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T10:55:43.797512+0000","last_peered":"2024-04-20T10:55:43.797512+0000","last_clean":"2024-04-20T10:55:43.797512+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T10:55:43.797512+0000","last_undegraded":"2024-04-20T10:55:43.797512+0000","last_fullsized":"2024-04-20T10:55:43.797512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793562+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T10:55:43.793562+0000","last_peered":"2024-04-20T10:55:43.793562+0000","last_clean":"2024-04-20T10:55:43.793562+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T10:55:43.793562+0000","last_undegraded":"2024-04-20T10:55:43.793562+0000","last_fullsized":"2024-04-20T10:55:43.793562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.787910+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T10:55:47.787910+0000","last_peered":"2024-04-20T10:55:47.787910+0000","last_clean":"2024-04-20T10:55:47.787910+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T10:55:47.787910+0000","last_undegraded":"2024-04-20T10:55:47.787910+0000","last_fullsized":"2024-04-20T10:55:47.787910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285397+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T10:55:44.285397+0000","last_peered":"2024-04-20T10:55:44.285397+0000","last_clean":"2024-04-20T10:55:44.285397+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T10:55:44.285397+0000","last_undegraded":"2024-04-20T10:55:44.285397+0000","last_fullsized":"2024-04-20T10:55:44.285397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285923+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T10:55:44.285923+0000","last_peered":"2024-04-20T10:55:44.285923+0000","last_clean":"2024-04-20T10:55:44.285923+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T10:55:44.285923+0000","last_undegraded":"2024-04-20T10:55:44.285923+0000","last_fullsized":"2024-04-20T10:55:44.285923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800994+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T10:55:43.800994+0000","last_peered":"2024-04-20T10:55:43.800994+0000","last_clean":"2024-04-20T10:55:43.800994+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T10:55:43.800994+0000","last_undegraded":"2024-04-20T10:55:43.800994+0000","last_fullsized":"2024-04-20T10:55:43.800994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797429+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T10:55:43.797429+0000","last_peered":"2024-04-20T10:55:43.797429+0000","last_clean":"2024-04-20T10:55:43.797429+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T10:55:43.797429+0000","last_undegraded":"2024-04-20T10:55:43.797429+0000","last_fullsized":"2024-04-20T10:55:43.797429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793024+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T10:55:43.793024+0000","last_peered":"2024-04-20T10:55:43.793024+0000","last_clean":"2024-04-20T10:55:43.793024+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T10:55:43.793024+0000","last_undegraded":"2024-04-20T10:55:43.793024+0000","last_fullsized":"2024-04-20T10:55:43.793024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797381+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T10:55:43.797381+0000","last_peered":"2024-04-20T10:55:43.797381+0000","last_clean":"2024-04-20T10:55:43.797381+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T10:55:43.797381+0000","last_undegraded":"2024-04-20T10:55:43.797381+0000","last_fullsized":"2024-04-20T10:55:43.797381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793105+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T10:55:43.793105+0000","last_peered":"2024-04-20T10:55:43.793105+0000","last_clean":"2024-04-20T10:55:43.793105+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T10:55:43.793105+0000","last_undegraded":"2024-04-20T10:55:43.793105+0000","last_fullsized":"2024-04-20T10:55:43.793105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.787788+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T10:55:47.787788+0000","last_peered":"2024-04-20T10:55:47.787788+0000","last_clean":"2024-04-20T10:55:47.787788+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T10:55:47.787788+0000","last_undegraded":"2024-04-20T10:55:47.787788+0000","last_fullsized":"2024-04-20T10:55:47.787788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286369+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T10:55:44.286369+0000","last_peered":"2024-04-20T10:55:44.286369+0000","last_clean":"2024-04-20T10:55:44.286369+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T10:55:44.286369+0000","last_undegraded":"2024-04-20T10:55:44.286369+0000","last_fullsized":"2024-04-20T10:55:44.286369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286936+0000","last_change":"2024-04-20T10:55:42.792905+0000","last_active":"2024-04-20T10:55:44.286936+0000","last_peered":"2024-04-20T10:55:44.286936+0000","last_clean":"2024-04-20T10:55:44.286936+0000","last_became_active":"2024-04-20T10:55:42.792587+0000","last_became_peered":"2024-04-20T10:55:42.792587+0000","last_unstale":"2024-04-20T10:55:44.286936+0000","last_undegraded":"2024-04-20T10:55:44.286936+0000","last_fullsized":"2024-04-20T10:55:44.286936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:02.987448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.787455+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T10:55:47.787455+0000","last_peered":"2024-04-20T10:55:47.787455+0000","last_clean":"2024-04-20T10:55:47.787455+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T10:55:47.787455+0000","last_undegraded":"2024-04-20T10:55:47.787455+0000","last_fullsized":"2024-04-20T10:55:47.787455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286560+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T10:55:44.286560+0000","last_peered":"2024-04-20T10:55:44.286560+0000","last_clean":"2024-04-20T10:55:44.286560+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T10:55:44.286560+0000","last_undegraded":"2024-04-20T10:55:44.286560+0000","last_fullsized":"2024-04-20T10:55:44.286560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.787552+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T10:55:47.787552+0000","last_peered":"2024-04-20T10:55:47.787552+0000","last_clean":"2024-04-20T10:55:47.787552+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T10:55:47.787552+0000","last_undegraded":"2024-04-20T10:55:47.787552+0000","last_fullsized":"2024-04-20T10:55:47.787552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.722204+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T10:55:44.722204+0000","last_peered":"2024-04-20T10:55:44.722204+0000","last_clean":"2024-04-20T10:55:44.722204+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T10:55:44.722204+0000","last_undegraded":"2024-04-20T10:55:44.722204+0000","last_fullsized":"2024-04-20T10:55:44.722204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793370+0000","last_change":"2024-04-20T10:55:42.802227+0000","last_active":"2024-04-20T10:55:43.793370+0000","last_peered":"2024-04-20T10:55:43.793370+0000","last_clean":"2024-04-20T10:55:43.793370+0000","last_became_active":"2024-04-20T10:55:42.801944+0000","last_became_peered":"2024-04-20T10:55:42.801944+0000","last_unstale":"2024-04-20T10:55:43.793370+0000","last_undegraded":"2024-04-20T10:55:43.793370+0000","last_fullsized":"2024-04-20T10:55:43.793370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:29.862643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.788592+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T10:55:47.788592+0000","last_peered":"2024-04-20T10:55:47.788592+0000","last_clean":"2024-04-20T10:55:47.788592+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T10:55:47.788592+0000","last_undegraded":"2024-04-20T10:55:47.788592+0000","last_fullsized":"2024-04-20T10:55:47.788592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:55:50.576 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:55:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.711+0000 7fbe4d8f2640 1 Processor -- start 2024-04-20T10:55:50.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.711+0000 7fbe4d8f2640 1 -- start start 2024-04-20T10:55:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.712+0000 7fbe4d8f2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbe48071f60 0x7fbe480730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.712+0000 7fbe4d8f2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe4815c180 con 0x7fbe48073620 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.712+0000 7fbe4d8f2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe4806c920 con 0x7fbe48155720 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.712+0000 7fbe4d8f2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbe4806b930 con 0x7fbe48071f60 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.712+0000 7fbe467fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbe48071f60 0x7fbe480730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.712+0000 7fbe477fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbe48073620 0x7fbe48165c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53478/0 (socket says 172.21.15.27:53478) 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.712+0000 7fbe477fe640 1 -- 172.21.15.27:0/3896479431 learned_addr learned my addr 172.21.15.27:0/3896479431 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.712+0000 7fbe467fc640 1 -- 172.21.15.27:0/3896479431 >> v1:172.21.15.137:6789/0 conn(0x7fbe48155720 legacy=0x7fbe480c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.713+0000 7fbe467fc640 1 -- 172.21.15.27:0/3896479431 >> v1:172.21.15.27:6789/0 conn(0x7fbe48073620 legacy=0x7fbe48165c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.713+0000 7fbe467fc640 1 -- 172.21.15.27:0/3896479431 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4806bb60 con 0x7fbe48071f60 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.713+0000 7fbe45ffb640 1 -- 172.21.15.27:0/3896479431 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726720938 0 0) 0x7fbe4815c180 con 0x7fbe48073620 2024-04-20T10:55:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.713+0000 7fbe467fc640 1 --2- 172.21.15.27:0/3896479431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48071f60 0x7fbe480730e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe300097a0 tx=0x7fbe3002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706557c3d9366631 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.713+0000 7fbe45ffb640 1 -- 172.21.15.27:0/3896479431 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30034070 con 0x7fbe48071f60 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.713+0000 7fbe45ffb640 1 -- 172.21.15.27:0/3896479431 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe30030610 con 0x7fbe48071f60 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.713+0000 7fbe45ffb640 1 -- 172.21.15.27:0/3896479431 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300309e0 con 0x7fbe48071f60 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.714+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/3896479431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48071f60 msgr2=0x7fbe480730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.714+0000 7fbe4d8f2640 1 --2- 172.21.15.27:0/3896479431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48071f60 0x7fbe480730e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe300097a0 tx=0x7fbe3002b960 comp rx=0 tx=0).stop 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.714+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/3896479431 shutdown_connections 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.714+0000 7fbe4d8f2640 1 --2- 172.21.15.27:0/3896479431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48071f60 0x7fbe480730e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.714+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/3896479431 >> 172.21.15.27:0/3896479431 conn(0x7fbe4806f950 msgr2=0x7fbe4806fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.714+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/3896479431 shutdown_connections 2024-04-20T10:55:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.714+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/3896479431 wait complete. 2024-04-20T10:55:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.715+0000 7fbe4d8f2640 1 Processor -- start 2024-04-20T10:55:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.715+0000 7fbe4d8f2640 1 -- start start 2024-04-20T10:55:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe4d8f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe48071f60 0x7fbe4815f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe4d8f2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe48073620 0x7fbe4815d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe4d8f2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48155720 0x7fbe4814e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe4d8f2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4806c120 con 0x7fbe48155720 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe4d8f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4806c490 con 0x7fbe48071f60 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe4d8f2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbe48072370 con 0x7fbe48073620 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48155720 0x7fbe4814e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48155720 0x7fbe4814e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48808/0 (socket says 172.21.15.27:48808) 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe477fe640 1 -- 172.21.15.27:0/1331995586 learned_addr learned my addr 172.21.15.27:0/1331995586 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe46ffd640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe48071f60 0x7fbe4815f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe477fe640 1 -- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe48073620 msgr2=0x7fbe4815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe467fc640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe48073620 0x7fbe4815d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe477fe640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe48073620 0x7fbe4815d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe477fe640 1 -- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe48071f60 msgr2=0x7fbe4815f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe477fe640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe48071f60 0x7fbe4815f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe477fe640 1 -- 172.21.15.27:0/1331995586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4815b910 con 0x7fbe48155720 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.716+0000 7fbe46ffd640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe48071f60 0x7fbe4815f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:50.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.717+0000 7fbe477fe640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48155720 0x7fbe4814e4d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800c450 tx=0x7fbe3800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.717+0000 7fbe3ffff640 1 -- 172.21.15.27:0/1331995586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38018070 con 0x7fbe48155720 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.717+0000 7fbe3ffff640 1 -- 172.21.15.27:0/1331995586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe38007500 con 0x7fbe48155720 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.717+0000 7fbe3ffff640 1 -- 172.21.15.27:0/1331995586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38004a20 con 0x7fbe48155720 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.717+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4815d9d0 con 0x7fbe48155720 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.717+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe481f5250 con 0x7fbe48155720 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.718+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe4815f800 con 0x7fbe48155720 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.718+0000 7fbe3ffff640 1 -- 172.21.15.27:0/1331995586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe3801b070 con 0x7fbe48155720 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.719+0000 7fbe3ffff640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe2806c8f0 0x7fbe2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.719+0000 7fbe3ffff640 1 -- 172.21.15.27:0/1331995586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe3808ad80 con 0x7fbe48155720 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.719+0000 7fbe46ffd640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe2806c8f0 0x7fbe2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:50.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.719+0000 7fbe46ffd640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe2806c8f0 0x7fbe2806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe40007e90 tx=0x7fbe400073a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:50.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.723+0000 7fbe3ffff640 1 -- 172.21.15.27:0/1331995586 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbe4815f800 con 0x7fbe48155720 2024-04-20T10:55:50.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.908+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe481523d0 con 0x7fbe2806c8f0 2024-04-20T10:55:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.915+0000 7fbe3ffff640 1 -- 172.21.15.27:0/1331995586 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7fbe481523d0 con 0x7fbe2806c8f0 2024-04-20T10:55:50.918 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T10:55:50.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe2806c8f0 msgr2=0x7fbe2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe2806c8f0 0x7fbe2806edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe40007e90 tx=0x7fbe400073a0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48155720 msgr2=0x7fbe4814e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:50.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48155720 0x7fbe4814e4d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800c450 tx=0x7fbe3800c910 comp rx=0 tx=0).stop 2024-04-20T10:55:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 shutdown_connections 2024-04-20T10:55:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe2806c8f0 0x7fbe2806edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe48155720 0x7fbe4814e4d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe48073620 0x7fbe4815d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 --2- 172.21.15.27:0/1331995586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe48071f60 0x7fbe4815f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 >> 172.21.15.27:0/1331995586 conn(0x7fbe4806f950 msgr2=0x7fbe480c5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 shutdown_connections 2024-04-20T10:55:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:50.918+0000 7fbe4d8f2640 1 -- 172.21.15.27:0/1331995586 wait complete. 2024-04-20T10:55:50.929 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-20T10:55:49.703733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218952,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755768,"statfs":{"total":773094113280,"available":772869906432,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.926413"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287393+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T10:55:44.287393+0000","last_peered":"2024-04-20T10:55:44.287393+0000","last_clean":"2024-04-20T10:55:44.287393+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T10:55:44.287393+0000","last_undegraded":"2024-04-20T10:55:44.287393+0000","last_fullsized":"2024-04-20T10:55:44.287393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797629+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T10:55:43.797629+0000","last_peered":"2024-04-20T10:55:43.797629+0000","last_clean":"2024-04-20T10:55:43.797629+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T10:55:43.797629+0000","last_undegraded":"2024-04-20T10:55:43.797629+0000","last_fullsized":"2024-04-20T10:55:43.797629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285486+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T10:55:44.285486+0000","last_peered":"2024-04-20T10:55:44.285486+0000","last_clean":"2024-04-20T10:55:44.285486+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T10:55:44.285486+0000","last_undegraded":"2024-04-20T10:55:44.285486+0000","last_fullsized":"2024-04-20T10:55:44.285486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286383+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T10:55:44.286383+0000","last_peered":"2024-04-20T10:55:44.286383+0000","last_clean":"2024-04-20T10:55:44.286383+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T10:55:44.286383+0000","last_undegraded":"2024-04-20T10:55:44.286383+0000","last_fullsized":"2024-04-20T10:55:44.286383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286289+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T10:55:44.286289+0000","last_peered":"2024-04-20T10:55:44.286289+0000","last_clean":"2024-04-20T10:55:44.286289+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T10:55:44.286289+0000","last_undegraded":"2024-04-20T10:55:44.286289+0000","last_fullsized":"2024-04-20T10:55:44.286289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287329+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T10:55:44.287329+0000","last_peered":"2024-04-20T10:55:44.287329+0000","last_clean":"2024-04-20T10:55:44.287329+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T10:55:44.287329+0000","last_undegraded":"2024-04-20T10:55:44.287329+0000","last_fullsized":"2024-04-20T10:55:44.287329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287439+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T10:55:44.287439+0000","last_peered":"2024-04-20T10:55:44.287439+0000","last_clean":"2024-04-20T10:55:44.287439+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T10:55:44.287439+0000","last_undegraded":"2024-04-20T10:55:44.287439+0000","last_fullsized":"2024-04-20T10:55:44.287439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792859+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T10:55:43.792859+0000","last_peered":"2024-04-20T10:55:43.792859+0000","last_clean":"2024-04-20T10:55:43.792859+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T10:55:43.792859+0000","last_undegraded":"2024-04-20T10:55:43.792859+0000","last_fullsized":"2024-04-20T10:55:43.792859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286907+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T10:55:44.286907+0000","last_peered":"2024-04-20T10:55:44.286907+0000","last_clean":"2024-04-20T10:55:44.286907+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T10:55:44.286907+0000","last_undegraded":"2024-04-20T10:55:44.286907+0000","last_fullsized":"2024-04-20T10:55:44.286907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286541+0000","last_change":"2024-04-20T10:55:42.802716+0000","last_active":"2024-04-20T10:55:44.286541+0000","last_peered":"2024-04-20T10:55:44.286541+0000","last_clean":"2024-04-20T10:55:44.286541+0000","last_became_active":"2024-04-20T10:55:42.802418+0000","last_became_peered":"2024-04-20T10:55:42.802418+0000","last_unstale":"2024-04-20T10:55:44.286541+0000","last_undegraded":"2024-04-20T10:55:44.286541+0000","last_fullsized":"2024-04-20T10:55:44.286541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:12:22.186243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286228+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T10:55:44.286228+0000","last_peered":"2024-04-20T10:55:44.286228+0000","last_clean":"2024-04-20T10:55:44.286228+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T10:55:44.286228+0000","last_undegraded":"2024-04-20T10:55:44.286228+0000","last_fullsized":"2024-04-20T10:55:44.286228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801002+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T10:55:43.801002+0000","last_peered":"2024-04-20T10:55:43.801002+0000","last_clean":"2024-04-20T10:55:43.801002+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T10:55:43.801002+0000","last_undegraded":"2024-04-20T10:55:43.801002+0000","last_fullsized":"2024-04-20T10:55:43.801002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793050+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T10:55:43.793050+0000","last_peered":"2024-04-20T10:55:43.793050+0000","last_clean":"2024-04-20T10:55:43.793050+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T10:55:43.793050+0000","last_undegraded":"2024-04-20T10:55:43.793050+0000","last_fullsized":"2024-04-20T10:55:43.793050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797332+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T10:55:43.797332+0000","last_peered":"2024-04-20T10:55:43.797332+0000","last_clean":"2024-04-20T10:55:43.797332+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T10:55:43.797332+0000","last_undegraded":"2024-04-20T10:55:43.797332+0000","last_fullsized":"2024-04-20T10:55:43.797332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285851+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T10:55:44.285851+0000","last_peered":"2024-04-20T10:55:44.285851+0000","last_clean":"2024-04-20T10:55:44.285851+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T10:55:44.285851+0000","last_undegraded":"2024-04-20T10:55:44.285851+0000","last_fullsized":"2024-04-20T10:55:44.285851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286491+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T10:55:44.286491+0000","last_peered":"2024-04-20T10:55:44.286491+0000","last_clean":"2024-04-20T10:55:44.286491+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T10:55:44.286491+0000","last_undegraded":"2024-04-20T10:55:44.286491+0000","last_fullsized":"2024-04-20T10:55:44.286491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285217+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T10:55:44.285217+0000","last_peered":"2024-04-20T10:55:44.285217+0000","last_clean":"2024-04-20T10:55:44.285217+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T10:55:44.285217+0000","last_undegraded":"2024-04-20T10:55:44.285217+0000","last_fullsized":"2024-04-20T10:55:44.285217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287175+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T10:55:44.287175+0000","last_peered":"2024-04-20T10:55:44.287175+0000","last_clean":"2024-04-20T10:55:44.287175+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T10:55:44.287175+0000","last_undegraded":"2024-04-20T10:55:44.287175+0000","last_fullsized":"2024-04-20T10:55:44.287175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286994+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T10:55:44.286994+0000","last_peered":"2024-04-20T10:55:44.286994+0000","last_clean":"2024-04-20T10:55:44.286994+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T10:55:44.286994+0000","last_undegraded":"2024-04-20T10:55:44.286994+0000","last_fullsized":"2024-04-20T10:55:44.286994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285751+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T10:55:44.285751+0000","last_peered":"2024-04-20T10:55:44.285751+0000","last_clean":"2024-04-20T10:55:44.285751+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T10:55:44.285751+0000","last_undegraded":"2024-04-20T10:55:44.285751+0000","last_fullsized":"2024-04-20T10:55:44.285751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287484+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T10:55:44.287484+0000","last_peered":"2024-04-20T10:55:44.287484+0000","last_clean":"2024-04-20T10:55:44.287484+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T10:55:44.287484+0000","last_undegraded":"2024-04-20T10:55:44.287484+0000","last_fullsized":"2024-04-20T10:55:44.287484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792921+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T10:55:43.792921+0000","last_peered":"2024-04-20T10:55:43.792921+0000","last_clean":"2024-04-20T10:55:43.792921+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T10:55:43.792921+0000","last_undegraded":"2024-04-20T10:55:43.792921+0000","last_fullsized":"2024-04-20T10:55:43.792921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793499+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T10:55:43.793499+0000","last_peered":"2024-04-20T10:55:43.793499+0000","last_clean":"2024-04-20T10:55:43.793499+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T10:55:43.793499+0000","last_undegraded":"2024-04-20T10:55:43.793499+0000","last_fullsized":"2024-04-20T10:55:43.793499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286683+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T10:55:44.286683+0000","last_peered":"2024-04-20T10:55:44.286683+0000","last_clean":"2024-04-20T10:55:44.286683+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T10:55:44.286683+0000","last_undegraded":"2024-04-20T10:55:44.286683+0000","last_fullsized":"2024-04-20T10:55:44.286683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797439+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T10:55:43.797439+0000","last_peered":"2024-04-20T10:55:43.797439+0000","last_clean":"2024-04-20T10:55:43.797439+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T10:55:43.797439+0000","last_undegraded":"2024-04-20T10:55:43.797439+0000","last_fullsized":"2024-04-20T10:55:43.797439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793232+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T10:55:43.793232+0000","last_peered":"2024-04-20T10:55:43.793232+0000","last_clean":"2024-04-20T10:55:43.793232+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T10:55:43.793232+0000","last_undegraded":"2024-04-20T10:55:43.793232+0000","last_fullsized":"2024-04-20T10:55:43.793232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285757+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T10:55:44.285757+0000","last_peered":"2024-04-20T10:55:44.285757+0000","last_clean":"2024-04-20T10:55:44.285757+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T10:55:44.285757+0000","last_undegraded":"2024-04-20T10:55:44.285757+0000","last_fullsized":"2024-04-20T10:55:44.285757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287493+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T10:55:44.287493+0000","last_peered":"2024-04-20T10:55:44.287493+0000","last_clean":"2024-04-20T10:55:44.287493+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T10:55:44.287493+0000","last_undegraded":"2024-04-20T10:55:44.287493+0000","last_fullsized":"2024-04-20T10:55:44.287493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285309+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T10:55:44.285309+0000","last_peered":"2024-04-20T10:55:44.285309+0000","last_clean":"2024-04-20T10:55:44.285309+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T10:55:44.285309+0000","last_undegraded":"2024-04-20T10:55:44.285309+0000","last_fullsized":"2024-04-20T10:55:44.285309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285746+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T10:55:44.285746+0000","last_peered":"2024-04-20T10:55:44.285746+0000","last_clean":"2024-04-20T10:55:44.285746+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T10:55:44.285746+0000","last_undegraded":"2024-04-20T10:55:44.285746+0000","last_fullsized":"2024-04-20T10:55:44.285746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800661+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T10:55:43.800661+0000","last_peered":"2024-04-20T10:55:43.800661+0000","last_clean":"2024-04-20T10:55:43.800661+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T10:55:43.800661+0000","last_undegraded":"2024-04-20T10:55:43.800661+0000","last_fullsized":"2024-04-20T10:55:43.800661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.722181+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T10:55:44.722181+0000","last_peered":"2024-04-20T10:55:44.722181+0000","last_clean":"2024-04-20T10:55:44.722181+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T10:55:44.722181+0000","last_undegraded":"2024-04-20T10:55:44.722181+0000","last_fullsized":"2024-04-20T10:55:44.722181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285840+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T10:55:44.285840+0000","last_peered":"2024-04-20T10:55:44.285840+0000","last_clean":"2024-04-20T10:55:44.285840+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T10:55:44.285840+0000","last_undegraded":"2024-04-20T10:55:44.285840+0000","last_fullsized":"2024-04-20T10:55:44.285840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.724680+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T10:55:44.724680+0000","last_peered":"2024-04-20T10:55:44.724680+0000","last_clean":"2024-04-20T10:55:44.724680+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T10:55:44.724680+0000","last_undegraded":"2024-04-20T10:55:44.724680+0000","last_fullsized":"2024-04-20T10:55:44.724680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793178+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T10:55:43.793178+0000","last_peered":"2024-04-20T10:55:43.793178+0000","last_clean":"2024-04-20T10:55:43.793178+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T10:55:43.793178+0000","last_undegraded":"2024-04-20T10:55:43.793178+0000","last_fullsized":"2024-04-20T10:55:43.793178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792877+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T10:55:43.792877+0000","last_peered":"2024-04-20T10:55:43.792877+0000","last_clean":"2024-04-20T10:55:43.792877+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T10:55:43.792877+0000","last_undegraded":"2024-04-20T10:55:43.792877+0000","last_fullsized":"2024-04-20T10:55:43.792877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286112+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T10:55:44.286112+0000","last_peered":"2024-04-20T10:55:44.286112+0000","last_clean":"2024-04-20T10:55:44.286112+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T10:55:44.286112+0000","last_undegraded":"2024-04-20T10:55:44.286112+0000","last_fullsized":"2024-04-20T10:55:44.286112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797544+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T10:55:43.797544+0000","last_peered":"2024-04-20T10:55:43.797544+0000","last_clean":"2024-04-20T10:55:43.797544+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T10:55:43.797544+0000","last_undegraded":"2024-04-20T10:55:43.797544+0000","last_fullsized":"2024-04-20T10:55:43.797544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286640+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T10:55:44.286640+0000","last_peered":"2024-04-20T10:55:44.286640+0000","last_clean":"2024-04-20T10:55:44.286640+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T10:55:44.286640+0000","last_undegraded":"2024-04-20T10:55:44.286640+0000","last_fullsized":"2024-04-20T10:55:44.286640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287062+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T10:55:44.287062+0000","last_peered":"2024-04-20T10:55:44.287062+0000","last_clean":"2024-04-20T10:55:44.287062+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T10:55:44.287062+0000","last_undegraded":"2024-04-20T10:55:44.287062+0000","last_fullsized":"2024-04-20T10:55:44.287062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287172+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T10:55:44.287172+0000","last_peered":"2024-04-20T10:55:44.287172+0000","last_clean":"2024-04-20T10:55:44.287172+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T10:55:44.287172+0000","last_undegraded":"2024-04-20T10:55:44.287172+0000","last_fullsized":"2024-04-20T10:55:44.287172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285669+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T10:55:44.285669+0000","last_peered":"2024-04-20T10:55:44.285669+0000","last_clean":"2024-04-20T10:55:44.285669+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T10:55:44.285669+0000","last_undegraded":"2024-04-20T10:55:44.285669+0000","last_fullsized":"2024-04-20T10:55:44.285669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.786854+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T10:55:47.786854+0000","last_peered":"2024-04-20T10:55:47.786854+0000","last_clean":"2024-04-20T10:55:47.786854+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T10:55:47.786854+0000","last_undegraded":"2024-04-20T10:55:47.786854+0000","last_fullsized":"2024-04-20T10:55:47.786854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801052+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T10:55:43.801052+0000","last_peered":"2024-04-20T10:55:43.801052+0000","last_clean":"2024-04-20T10:55:43.801052+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T10:55:43.801052+0000","last_undegraded":"2024-04-20T10:55:43.801052+0000","last_fullsized":"2024-04-20T10:55:43.801052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.794272+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T10:55:47.794272+0000","last_peered":"2024-04-20T10:55:47.794272+0000","last_clean":"2024-04-20T10:55:47.794272+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T10:55:47.794272+0000","last_undegraded":"2024-04-20T10:55:47.794272+0000","last_fullsized":"2024-04-20T10:55:47.794272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287336+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T10:55:44.287336+0000","last_peered":"2024-04-20T10:55:44.287336+0000","last_clean":"2024-04-20T10:55:44.287336+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T10:55:44.287336+0000","last_undegraded":"2024-04-20T10:55:44.287336+0000","last_fullsized":"2024-04-20T10:55:44.287336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.786016+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T10:55:47.786016+0000","last_peered":"2024-04-20T10:55:47.786016+0000","last_clean":"2024-04-20T10:55:47.786016+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T10:55:47.786016+0000","last_undegraded":"2024-04-20T10:55:47.786016+0000","last_fullsized":"2024-04-20T10:55:47.786016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285622+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T10:55:44.285622+0000","last_peered":"2024-04-20T10:55:44.285622+0000","last_clean":"2024-04-20T10:55:44.285622+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T10:55:44.285622+0000","last_undegraded":"2024-04-20T10:55:44.285622+0000","last_fullsized":"2024-04-20T10:55:44.285622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287576+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T10:55:44.287576+0000","last_peered":"2024-04-20T10:55:44.287576+0000","last_clean":"2024-04-20T10:55:44.287576+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T10:55:44.287576+0000","last_undegraded":"2024-04-20T10:55:44.287576+0000","last_fullsized":"2024-04-20T10:55:44.287576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286631+0000","last_change":"2024-04-20T10:55:42.805169+0000","last_active":"2024-04-20T10:55:44.286631+0000","last_peered":"2024-04-20T10:55:44.286631+0000","last_clean":"2024-04-20T10:55:44.286631+0000","last_became_active":"2024-04-20T10:55:42.804573+0000","last_became_peered":"2024-04-20T10:55:42.804573+0000","last_unstale":"2024-04-20T10:55:44.286631+0000","last_undegraded":"2024-04-20T10:55:44.286631+0000","last_fullsized":"2024-04-20T10:55:44.286631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:16.103447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287223+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T10:55:44.287223+0000","last_peered":"2024-04-20T10:55:44.287223+0000","last_clean":"2024-04-20T10:55:44.287223+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T10:55:44.287223+0000","last_undegraded":"2024-04-20T10:55:44.287223+0000","last_fullsized":"2024-04-20T10:55:44.287223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285762+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T10:55:44.285762+0000","last_peered":"2024-04-20T10:55:44.285762+0000","last_clean":"2024-04-20T10:55:44.285762+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T10:55:44.285762+0000","last_undegraded":"2024-04-20T10:55:44.285762+0000","last_fullsized":"2024-04-20T10:55:44.285762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.791669+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T10:55:47.791669+0000","last_peered":"2024-04-20T10:55:47.791669+0000","last_clean":"2024-04-20T10:55:47.791669+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T10:55:47.791669+0000","last_undegraded":"2024-04-20T10:55:47.791669+0000","last_fullsized":"2024-04-20T10:55:47.791669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287518+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T10:55:44.287518+0000","last_peered":"2024-04-20T10:55:44.287518+0000","last_clean":"2024-04-20T10:55:44.287518+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T10:55:44.287518+0000","last_undegraded":"2024-04-20T10:55:44.287518+0000","last_fullsized":"2024-04-20T10:55:44.287518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.722087+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T10:55:44.722087+0000","last_peered":"2024-04-20T10:55:44.722087+0000","last_clean":"2024-04-20T10:55:44.722087+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T10:55:44.722087+0000","last_undegraded":"2024-04-20T10:55:44.722087+0000","last_fullsized":"2024-04-20T10:55:44.722087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286460+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T10:55:44.286460+0000","last_peered":"2024-04-20T10:55:44.286460+0000","last_clean":"2024-04-20T10:55:44.286460+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T10:55:44.286460+0000","last_undegraded":"2024-04-20T10:55:44.286460+0000","last_fullsized":"2024-04-20T10:55:44.286460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.789279+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T10:55:47.789279+0000","last_peered":"2024-04-20T10:55:47.789279+0000","last_clean":"2024-04-20T10:55:47.789279+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T10:55:47.789279+0000","last_undegraded":"2024-04-20T10:55:47.789279+0000","last_fullsized":"2024-04-20T10:55:47.789279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287067+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T10:55:44.287067+0000","last_peered":"2024-04-20T10:55:44.287067+0000","last_clean":"2024-04-20T10:55:44.287067+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T10:55:44.287067+0000","last_undegraded":"2024-04-20T10:55:44.287067+0000","last_fullsized":"2024-04-20T10:55:44.287067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.731055+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T10:55:44.731055+0000","last_peered":"2024-04-20T10:55:44.731055+0000","last_clean":"2024-04-20T10:55:44.731055+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T10:55:44.731055+0000","last_undegraded":"2024-04-20T10:55:44.731055+0000","last_fullsized":"2024-04-20T10:55:44.731055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286721+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T10:55:44.286721+0000","last_peered":"2024-04-20T10:55:44.286721+0000","last_clean":"2024-04-20T10:55:44.286721+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T10:55:44.286721+0000","last_undegraded":"2024-04-20T10:55:44.286721+0000","last_fullsized":"2024-04-20T10:55:44.286721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285213+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T10:55:44.285213+0000","last_peered":"2024-04-20T10:55:44.285213+0000","last_clean":"2024-04-20T10:55:44.285213+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T10:55:44.285213+0000","last_undegraded":"2024-04-20T10:55:44.285213+0000","last_fullsized":"2024-04-20T10:55:44.285213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286934+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T10:55:44.286934+0000","last_peered":"2024-04-20T10:55:44.286934+0000","last_clean":"2024-04-20T10:55:44.286934+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T10:55:44.286934+0000","last_undegraded":"2024-04-20T10:55:44.286934+0000","last_fullsized":"2024-04-20T10:55:44.286934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286525+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T10:55:44.286525+0000","last_peered":"2024-04-20T10:55:44.286525+0000","last_clean":"2024-04-20T10:55:44.286525+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T10:55:44.286525+0000","last_undegraded":"2024-04-20T10:55:44.286525+0000","last_fullsized":"2024-04-20T10:55:44.286525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793252+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T10:55:43.793252+0000","last_peered":"2024-04-20T10:55:43.793252+0000","last_clean":"2024-04-20T10:55:43.793252+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T10:55:43.793252+0000","last_undegraded":"2024-04-20T10:55:43.793252+0000","last_fullsized":"2024-04-20T10:55:43.793252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287230+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T10:55:44.287230+0000","last_peered":"2024-04-20T10:55:44.287230+0000","last_clean":"2024-04-20T10:55:44.287230+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T10:55:44.287230+0000","last_undegraded":"2024-04-20T10:55:44.287230+0000","last_fullsized":"2024-04-20T10:55:44.287230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285651+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T10:55:44.285651+0000","last_peered":"2024-04-20T10:55:44.285651+0000","last_clean":"2024-04-20T10:55:44.285651+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T10:55:44.285651+0000","last_undegraded":"2024-04-20T10:55:44.285651+0000","last_fullsized":"2024-04-20T10:55:44.285651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285287+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T10:55:44.285287+0000","last_peered":"2024-04-20T10:55:44.285287+0000","last_clean":"2024-04-20T10:55:44.285287+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T10:55:44.285287+0000","last_undegraded":"2024-04-20T10:55:44.285287+0000","last_fullsized":"2024-04-20T10:55:44.285287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797319+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T10:55:43.797319+0000","last_peered":"2024-04-20T10:55:43.797319+0000","last_clean":"2024-04-20T10:55:43.797319+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T10:55:43.797319+0000","last_undegraded":"2024-04-20T10:55:43.797319+0000","last_fullsized":"2024-04-20T10:55:43.797319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287600+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T10:55:44.287600+0000","last_peered":"2024-04-20T10:55:44.287600+0000","last_clean":"2024-04-20T10:55:44.287600+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T10:55:44.287600+0000","last_undegraded":"2024-04-20T10:55:44.287600+0000","last_fullsized":"2024-04-20T10:55:44.287600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800914+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T10:55:43.800914+0000","last_peered":"2024-04-20T10:55:43.800914+0000","last_clean":"2024-04-20T10:55:43.800914+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T10:55:43.800914+0000","last_undegraded":"2024-04-20T10:55:43.800914+0000","last_fullsized":"2024-04-20T10:55:43.800914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797497+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T10:55:43.797497+0000","last_peered":"2024-04-20T10:55:43.797497+0000","last_clean":"2024-04-20T10:55:43.797497+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T10:55:43.797497+0000","last_undegraded":"2024-04-20T10:55:43.797497+0000","last_fullsized":"2024-04-20T10:55:43.797497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287087+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T10:55:44.287087+0000","last_peered":"2024-04-20T10:55:44.287087+0000","last_clean":"2024-04-20T10:55:44.287087+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T10:55:44.287087+0000","last_undegraded":"2024-04-20T10:55:44.287087+0000","last_fullsized":"2024-04-20T10:55:44.287087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287263+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T10:55:44.287263+0000","last_peered":"2024-04-20T10:55:44.287263+0000","last_clean":"2024-04-20T10:55:44.287263+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T10:55:44.287263+0000","last_undegraded":"2024-04-20T10:55:44.287263+0000","last_fullsized":"2024-04-20T10:55:44.287263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797411+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T10:55:43.797411+0000","last_peered":"2024-04-20T10:55:43.797411+0000","last_clean":"2024-04-20T10:55:43.797411+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T10:55:43.797411+0000","last_undegraded":"2024-04-20T10:55:43.797411+0000","last_fullsized":"2024-04-20T10:55:43.797411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797512+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T10:55:43.797512+0000","last_peered":"2024-04-20T10:55:43.797512+0000","last_clean":"2024-04-20T10:55:43.797512+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T10:55:43.797512+0000","last_undegraded":"2024-04-20T10:55:43.797512+0000","last_fullsized":"2024-04-20T10:55:43.797512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793562+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T10:55:43.793562+0000","last_peered":"2024-04-20T10:55:43.793562+0000","last_clean":"2024-04-20T10:55:43.793562+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T10:55:43.793562+0000","last_undegraded":"2024-04-20T10:55:43.793562+0000","last_fullsized":"2024-04-20T10:55:43.793562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.787910+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T10:55:47.787910+0000","last_peered":"2024-04-20T10:55:47.787910+0000","last_clean":"2024-04-20T10:55:47.787910+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T10:55:47.787910+0000","last_undegraded":"2024-04-20T10:55:47.787910+0000","last_fullsized":"2024-04-20T10:55:47.787910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285397+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T10:55:44.285397+0000","last_peered":"2024-04-20T10:55:44.285397+0000","last_clean":"2024-04-20T10:55:44.285397+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T10:55:44.285397+0000","last_undegraded":"2024-04-20T10:55:44.285397+0000","last_fullsized":"2024-04-20T10:55:44.285397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285923+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T10:55:44.285923+0000","last_peered":"2024-04-20T10:55:44.285923+0000","last_clean":"2024-04-20T10:55:44.285923+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T10:55:44.285923+0000","last_undegraded":"2024-04-20T10:55:44.285923+0000","last_fullsized":"2024-04-20T10:55:44.285923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800994+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T10:55:43.800994+0000","last_peered":"2024-04-20T10:55:43.800994+0000","last_clean":"2024-04-20T10:55:43.800994+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T10:55:43.800994+0000","last_undegraded":"2024-04-20T10:55:43.800994+0000","last_fullsized":"2024-04-20T10:55:43.800994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797429+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T10:55:43.797429+0000","last_peered":"2024-04-20T10:55:43.797429+0000","last_clean":"2024-04-20T10:55:43.797429+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T10:55:43.797429+0000","last_undegraded":"2024-04-20T10:55:43.797429+0000","last_fullsized":"2024-04-20T10:55:43.797429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793024+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T10:55:43.793024+0000","last_peered":"2024-04-20T10:55:43.793024+0000","last_clean":"2024-04-20T10:55:43.793024+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T10:55:43.793024+0000","last_undegraded":"2024-04-20T10:55:43.793024+0000","last_fullsized":"2024-04-20T10:55:43.793024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797381+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T10:55:43.797381+0000","last_peered":"2024-04-20T10:55:43.797381+0000","last_clean":"2024-04-20T10:55:43.797381+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T10:55:43.797381+0000","last_undegraded":"2024-04-20T10:55:43.797381+0000","last_fullsized":"2024-04-20T10:55:43.797381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793105+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T10:55:43.793105+0000","last_peered":"2024-04-20T10:55:43.793105+0000","last_clean":"2024-04-20T10:55:43.793105+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T10:55:43.793105+0000","last_undegraded":"2024-04-20T10:55:43.793105+0000","last_fullsized":"2024-04-20T10:55:43.793105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.787788+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T10:55:47.787788+0000","last_peered":"2024-04-20T10:55:47.787788+0000","last_clean":"2024-04-20T10:55:47.787788+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T10:55:47.787788+0000","last_undegraded":"2024-04-20T10:55:47.787788+0000","last_fullsized":"2024-04-20T10:55:47.787788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286369+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T10:55:44.286369+0000","last_peered":"2024-04-20T10:55:44.286369+0000","last_clean":"2024-04-20T10:55:44.286369+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T10:55:44.286369+0000","last_undegraded":"2024-04-20T10:55:44.286369+0000","last_fullsized":"2024-04-20T10:55:44.286369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286936+0000","last_change":"2024-04-20T10:55:42.792905+0000","last_active":"2024-04-20T10:55:44.286936+0000","last_peered":"2024-04-20T10:55:44.286936+0000","last_clean":"2024-04-20T10:55:44.286936+0000","last_became_active":"2024-04-20T10:55:42.792587+0000","last_became_peered":"2024-04-20T10:55:42.792587+0000","last_unstale":"2024-04-20T10:55:44.286936+0000","last_undegraded":"2024-04-20T10:55:44.286936+0000","last_fullsized":"2024-04-20T10:55:44.286936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:02.987448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.787455+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T10:55:47.787455+0000","last_peered":"2024-04-20T10:55:47.787455+0000","last_clean":"2024-04-20T10:55:47.787455+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T10:55:47.787455+0000","last_undegraded":"2024-04-20T10:55:47.787455+0000","last_fullsized":"2024-04-20T10:55:47.787455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286560+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T10:55:44.286560+0000","last_peered":"2024-04-20T10:55:44.286560+0000","last_clean":"2024-04-20T10:55:44.286560+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T10:55:44.286560+0000","last_undegraded":"2024-04-20T10:55:44.286560+0000","last_fullsized":"2024-04-20T10:55:44.286560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.787552+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T10:55:47.787552+0000","last_peered":"2024-04-20T10:55:47.787552+0000","last_clean":"2024-04-20T10:55:47.787552+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T10:55:47.787552+0000","last_undegraded":"2024-04-20T10:55:47.787552+0000","last_fullsized":"2024-04-20T10:55:47.787552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.722204+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T10:55:44.722204+0000","last_peered":"2024-04-20T10:55:44.722204+0000","last_clean":"2024-04-20T10:55:44.722204+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T10:55:44.722204+0000","last_undegraded":"2024-04-20T10:55:44.722204+0000","last_fullsized":"2024-04-20T10:55:44.722204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793370+0000","last_change":"2024-04-20T10:55:42.802227+0000","last_active":"2024-04-20T10:55:43.793370+0000","last_peered":"2024-04-20T10:55:43.793370+0000","last_clean":"2024-04-20T10:55:43.793370+0000","last_became_active":"2024-04-20T10:55:42.801944+0000","last_became_peered":"2024-04-20T10:55:42.801944+0000","last_unstale":"2024-04-20T10:55:43.793370+0000","last_undegraded":"2024-04-20T10:55:43.793370+0000","last_fullsized":"2024-04-20T10:55:43.793370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:29.862643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.788592+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T10:55:47.788592+0000","last_peered":"2024-04-20T10:55:47.788592+0000","last_clean":"2024-04-20T10:55:47.788592+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T10:55:47.788592+0000","last_undegraded":"2024-04-20T10:55:47.788592+0000","last_fullsized":"2024-04-20T10:55:47.788592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:55:50.934 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T10:55:50.934 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T10:55:50.934 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-20T10:55:50.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:55:51.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.069+0000 7fc11ed2b640 1 Processor -- start 2024-04-20T10:55:51.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.069+0000 7fc11ed2b640 1 -- start start 2024-04-20T10:55:51.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.069+0000 7fc11ed2b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc118153480 0x7fc1180c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:51.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.069+0000 7fc11ed2b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc1180cc2a0 0x7fc1180c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:51.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.069+0000 7fc11ed2b640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc118070a60 con 0x7fc1181592e0 2024-04-20T10:55:51.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.069+0000 7fc11ed2b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc118071520 con 0x7fc118153480 2024-04-20T10:55:51.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.069+0000 7fc11ed2b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fc118071770 con 0x7fc1180cc2a0 2024-04-20T10:55:51.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.070+0000 7fc11cac7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc118153480 0x7fc1180c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:51.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.070+0000 7fc11cac7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc118153480 0x7fc1180c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48812/0 (socket says 172.21.15.27:48812) 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.070+0000 7fc11cac7640 1 -- 172.21.15.27:0/3152751598 learned_addr learned my addr 172.21.15.27:0/3152751598 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.070+0000 7fc11d2c8640 1 --2- 172.21.15.27:0/3152751598 >> v2:172.21.15.137:3300/0 conn(0x7fc1180cc2a0 0x7fc1180c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.070+0000 7fc11cac7640 1 -- 172.21.15.27:0/3152751598 >> v2:172.21.15.137:3300/0 conn(0x7fc1180cc2a0 msgr2=0x7fc1180c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.070+0000 7fc11cac7640 1 --2- 172.21.15.27:0/3152751598 >> v2:172.21.15.137:3300/0 conn(0x7fc1180cc2a0 0x7fc1180c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.070+0000 7fc11cac7640 1 -- 172.21.15.27:0/3152751598 >> v1:172.21.15.137:6789/0 conn(0x7fc1181592e0 legacy=0x7fc1180c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.070+0000 7fc11cac7640 1 -- 172.21.15.27:0/3152751598 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc118070d00 con 0x7fc118153480 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc11cac7640 1 --2- 172.21.15.27:0/3152751598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc118153480 0x7fc1180c3f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc11806f210 tx=0x7fc10c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee214e4ae0b4c914 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc1177fe640 1 -- 172.21.15.27:0/3152751598 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c035020 con 0x7fc118153480 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc1177fe640 1 -- 172.21.15.27:0/3152751598 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10c031870 con 0x7fc118153480 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc1177fe640 1 -- 172.21.15.27:0/3152751598 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c031c40 con 0x7fc118153480 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc11ed2b640 1 -- 172.21.15.27:0/3152751598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc118153480 msgr2=0x7fc1180c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc11ed2b640 1 --2- 172.21.15.27:0/3152751598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc118153480 0x7fc1180c3f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc11806f210 tx=0x7fc10c02b910 comp rx=0 tx=0).stop 2024-04-20T10:55:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc11ed2b640 1 -- 172.21.15.27:0/3152751598 shutdown_connections 2024-04-20T10:55:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc11ed2b640 1 --2- 172.21.15.27:0/3152751598 >> v2:172.21.15.137:3300/0 conn(0x7fc1180cc2a0 0x7fc1180c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc11ed2b640 1 --2- 172.21.15.27:0/3152751598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc118153480 0x7fc1180c3f20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc11ed2b640 1 -- 172.21.15.27:0/3152751598 >> 172.21.15.27:0/3152751598 conn(0x7fc118067ae0 msgr2=0x7fc118067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc11ed2b640 1 -- 172.21.15.27:0/3152751598 shutdown_connections 2024-04-20T10:55:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.071+0000 7fc11ed2b640 1 -- 172.21.15.27:0/3152751598 wait complete. 2024-04-20T10:55:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.072+0000 7fc11ed2b640 1 Processor -- start 2024-04-20T10:55:51.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.072+0000 7fc11ed2b640 1 -- start start 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.073+0000 7fc11ed2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1180cc2a0 0x7fc118156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.073+0000 7fc11ed2b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc118153480 0x7fc11814ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.073+0000 7fc11ed2b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1181592e0 0x7fc11806e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.073+0000 7fc11ed2b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc1180cf350 con 0x7fc1181592e0 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.073+0000 7fc11ed2b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc118071a50 con 0x7fc1180cc2a0 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.073+0000 7fc11ed2b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc1180cf4d0 con 0x7fc118153480 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.073+0000 7fc11d2c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1181592e0 0x7fc11806e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.073+0000 7fc11d2c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1181592e0 0x7fc11806e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48816/0 (socket says 172.21.15.27:48816) 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.073+0000 7fc11d2c8640 1 -- 172.21.15.27:0/969895226 learned_addr learned my addr 172.21.15.27:0/969895226 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11cac7640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1180cc2a0 0x7fc118156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc117fff640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc118153480 0x7fc11814ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11d2c8640 1 -- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc118153480 msgr2=0x7fc11814ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11d2c8640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc118153480 0x7fc11814ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11d2c8640 1 -- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1180cc2a0 msgr2=0x7fc118156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11d2c8640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1180cc2a0 0x7fc118156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11d2c8640 1 -- 172.21.15.27:0/969895226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc118156090 con 0x7fc1181592e0 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11cac7640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1180cc2a0 0x7fc118156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11d2c8640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1181592e0 0x7fc11806e1d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc11000cda0 tx=0x7fc11000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc1157fa640 1 -- 172.21.15.27:0/969895226 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110003b90 con 0x7fc1181592e0 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc1157fa640 1 -- 172.21.15.27:0/969895226 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc110003d30 con 0x7fc1181592e0 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc118156ab0 con 0x7fc1181592e0 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc1157fa640 1 -- 172.21.15.27:0/969895226 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110004240 con 0x7fc1181592e0 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.074+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc118155ad0 con 0x7fc1181592e0 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.075+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1180688e0 con 0x7fc1181592e0 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.076+0000 7fc1157fa640 1 -- 172.21.15.27:0/969895226 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1100043e0 con 0x7fc1181592e0 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.076+0000 7fc1157fa640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc0fc06c8f0 0x7fc0fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.076+0000 7fc1157fa640 1 -- 172.21.15.27:0/969895226 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc11008c600 con 0x7fc1181592e0 2024-04-20T10:55:51.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.076+0000 7fc11cac7640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc0fc06c8f0 0x7fc0fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:51.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.077+0000 7fc11cac7640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc0fc06c8f0 0x7fc0fc06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc10c036000 tx=0x7fc10c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:51.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.080+0000 7fc1157fa640 1 -- 172.21.15.27:0/969895226 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc11005aa30 con 0x7fc1181592e0 2024-04-20T10:55:51.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.304+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc11806f630 con 0x7fc1181592e0 2024-04-20T10:55:51.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.305+0000 7fc1157fa640 1 -- 172.21.15.27:0/969895226 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7fc11806f630 con 0x7fc1181592e0 2024-04-20T10:55:51.305 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:51.306 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-20T10:55:51.307 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.306+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc0fc06c8f0 msgr2=0x7fc0fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.306+0000 7fc11ed2b640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc0fc06c8f0 0x7fc0fc06edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc10c036000 tx=0x7fc10c008030 comp rx=0 tx=0).stop 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.306+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1181592e0 msgr2=0x7fc11806e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.306+0000 7fc11ed2b640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1181592e0 0x7fc11806e1d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc11000cda0 tx=0x7fc11000a660 comp rx=0 tx=0).stop 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.307+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 shutdown_connections 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.307+0000 7fc11ed2b640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc0fc06c8f0 0x7fc0fc06edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.307+0000 7fc11ed2b640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1181592e0 0x7fc11806e1d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.307+0000 7fc11ed2b640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc118153480 0x7fc11814ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.307+0000 7fc11ed2b640 1 --2- 172.21.15.27:0/969895226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1180cc2a0 0x7fc118156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.307+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 >> 172.21.15.27:0/969895226 conn(0x7fc118067ae0 msgr2=0x7fc1180ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.307+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 shutdown_connections 2024-04-20T10:55:51.308 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:51.307+0000 7fc11ed2b640 1 -- 172.21.15.27:0/969895226 wait complete. 2024-04-20T10:55:54.319 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:55:54.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.431+0000 7f660269a640 1 Processor -- start 2024-04-20T10:55:54.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.431+0000 7f660269a640 1 -- start start 2024-04-20T10:55:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f660269a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f65fc155800 0x7f65fc15bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f660269a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc0d1260 0x7f65fc0d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f660269a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65fc15c4d0 con 0x7f65fc0cbf50 2024-04-20T10:55:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f660269a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f65fc0ce610 con 0x7f65fc155800 2024-04-20T10:55:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f660269a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f65fc0ce180 con 0x7f65fc0d1260 2024-04-20T10:55:54.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f65fbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f65fc155800 0x7f65fc15bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:54.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f65fbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f65fc155800 0x7f65fc15bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48820/0 (socket says 172.21.15.27:48820) 2024-04-20T10:55:54.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f65fbfff640 1 -- 172.21.15.27:0/344324953 learned_addr learned my addr 172.21.15.27:0/344324953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:54.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f65fb7fe640 1 --1- 172.21.15.27:0/344324953 >> v1:172.21.15.27:6789/0 conn(0x7f65fc0cbf50 0x7f65fc0c5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53492/0 (socket says 172.21.15.27:53492) 2024-04-20T10:55:54.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f6600c37640 1 --2- 172.21.15.27:0/344324953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc0d1260 0x7f65fc0d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:54.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f65fbfff640 1 -- 172.21.15.27:0/344324953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc0d1260 msgr2=0x7f65fc0d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:54.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f65fbfff640 1 --2- 172.21.15.27:0/344324953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc0d1260 0x7f65fc0d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f65fbfff640 1 -- 172.21.15.27:0/344324953 >> v1:172.21.15.27:6789/0 conn(0x7f65fc0cbf50 legacy=0x7f65fc0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:54.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.432+0000 7f65fbfff640 1 -- 172.21.15.27:0/344324953 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc0cd290 con 0x7f65fc155800 2024-04-20T10:55:54.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.433+0000 7f65fbfff640 1 --2- 172.21.15.27:0/344324953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc155800 0x7f65fc15bf90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009870 tx=0x7f65ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7093e6d639e1f794 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:54.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.433+0000 7f65faffd640 1 -- 172.21.15.27:0/344324953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec035020 con 0x7f65fc155800 2024-04-20T10:55:54.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.433+0000 7f65faffd640 1 -- 172.21.15.27:0/344324953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ec031640 con 0x7f65fc155800 2024-04-20T10:55:54.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.433+0000 7f65faffd640 1 -- 172.21.15.27:0/344324953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec02d040 con 0x7f65fc155800 2024-04-20T10:55:54.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.434+0000 7f660269a640 1 -- 172.21.15.27:0/344324953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc155800 msgr2=0x7f65fc15bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:54.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.434+0000 7f660269a640 1 --2- 172.21.15.27:0/344324953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc155800 0x7f65fc15bf90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009870 tx=0x7f65ec02b8f0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.434+0000 7f660269a640 1 -- 172.21.15.27:0/344324953 shutdown_connections 2024-04-20T10:55:54.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.434+0000 7f660269a640 1 --2- 172.21.15.27:0/344324953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc0d1260 0x7f65fc0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.434+0000 7f660269a640 1 --2- 172.21.15.27:0/344324953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc155800 0x7f65fc15bf90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.434+0000 7f660269a640 1 -- 172.21.15.27:0/344324953 >> 172.21.15.27:0/344324953 conn(0x7f65fc067ae0 msgr2=0x7f65fc067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:54.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.434+0000 7f660269a640 1 -- 172.21.15.27:0/344324953 shutdown_connections 2024-04-20T10:55:54.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.434+0000 7f660269a640 1 -- 172.21.15.27:0/344324953 wait complete. 2024-04-20T10:55:54.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.435+0000 7f660269a640 1 Processor -- start 2024-04-20T10:55:54.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.435+0000 7f660269a640 1 -- start start 2024-04-20T10:55:54.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f660269a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc0cbf50 0x7f65fc1f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:54.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f660269a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65fc0d1260 0x7f65fc0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:54.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f660269a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc155800 0x7f65fc0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:54.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f660269a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc0cdb30 con 0x7f65fc0cbf50 2024-04-20T10:55:54.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f660269a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc0cd850 con 0x7f65fc0d1260 2024-04-20T10:55:54.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f660269a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f65fc0cde10 con 0x7f65fc155800 2024-04-20T10:55:54.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc0cbf50 0x7f65fc1f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:54.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f65fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc0cbf50 0x7f65fc1f24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48822/0 (socket says 172.21.15.27:48822) 2024-04-20T10:55:54.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f65fbfff640 1 -- 172.21.15.27:0/1063651781 learned_addr learned my addr 172.21.15.27:0/1063651781 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:54.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f65fb7fe640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65fc0d1260 0x7f65fc0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:54.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f6600c37640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc155800 0x7f65fc0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:54.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f65fbfff640 1 -- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc155800 msgr2=0x7f65fc0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:54.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f65fbfff640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc155800 0x7f65fc0b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f65fbfff640 1 -- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65fc0d1260 msgr2=0x7f65fc0b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:54.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f65fbfff640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65fc0d1260 0x7f65fc0b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.436+0000 7f65fbfff640 1 -- 172.21.15.27:0/1063651781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc159f00 con 0x7f65fc0cbf50 2024-04-20T10:55:54.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.437+0000 7f65fbfff640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc0cbf50 0x7f65fc1f24b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f65ec02b930 tx=0x7f65ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:54.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.437+0000 7f65f8ff9640 1 -- 172.21.15.27:0/1063651781 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec040070 con 0x7f65fc0cbf50 2024-04-20T10:55:54.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.437+0000 7f65f8ff9640 1 -- 172.21.15.27:0/1063651781 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ec03e070 con 0x7f65fc0cbf50 2024-04-20T10:55:54.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.437+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65fc15a1e0 con 0x7f65fc0cbf50 2024-04-20T10:55:54.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.437+0000 7f65f8ff9640 1 -- 172.21.15.27:0/1063651781 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec035040 con 0x7f65fc0cbf50 2024-04-20T10:55:54.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.437+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65fc0b3c50 con 0x7f65fc0cbf50 2024-04-20T10:55:54.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.438+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65fc0688e0 con 0x7f65fc0cbf50 2024-04-20T10:55:54.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.438+0000 7f65f8ff9640 1 -- 172.21.15.27:0/1063651781 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65ec038020 con 0x7f65fc0cbf50 2024-04-20T10:55:54.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.439+0000 7f65f8ff9640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f65e806c8f0 0x7f65e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:54.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.439+0000 7f65f8ff9640 1 -- 172.21.15.27:0/1063651781 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f65ec0ad3a0 con 0x7f65fc0cbf50 2024-04-20T10:55:54.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.439+0000 7f65fb7fe640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f65e806c8f0 0x7f65e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:54.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.439+0000 7f65fb7fe640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f65e806c8f0 0x7f65e806edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65fc1e76e0 tx=0x7f65f0007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:54.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.443+0000 7f65f8ff9640 1 -- 172.21.15.27:0/1063651781 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f65ec07b180 con 0x7f65fc0cbf50 2024-04-20T10:55:54.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.672+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f65fc0cb3b0 con 0x7f65fc0cbf50 2024-04-20T10:55:54.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.672+0000 7f65f8ff9640 1 -- 172.21.15.27:0/1063651781 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f65fc0cb3b0 con 0x7f65fc0cbf50 2024-04-20T10:55:54.673 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:54.674 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-20T10:55:54.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f65e806c8f0 msgr2=0x7f65e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:54.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f65e806c8f0 0x7f65e806edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65fc1e76e0 tx=0x7f65f0007e80 comp rx=0 tx=0).stop 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc0cbf50 msgr2=0x7f65fc1f24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc0cbf50 0x7f65fc1f24b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f65ec02b930 tx=0x7f65ec0046c0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 shutdown_connections 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f65e806c8f0 0x7f65e806edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f65fc155800 0x7f65fc0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65fc0d1260 0x7f65fc0b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 --2- 172.21.15.27:0/1063651781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65fc0cbf50 0x7f65fc1f24b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 >> 172.21.15.27:0/1063651781 conn(0x7f65fc067ae0 msgr2=0x7f65fc154010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 shutdown_connections 2024-04-20T10:55:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:54.674+0000 7f660269a640 1 -- 172.21.15.27:0/1063651781 wait complete. 2024-04-20T10:55:56.324 INFO:tasks.ceph.mds.d.smithi137.stderr:2024-04-20T10:55:56.324+0000 7f8f0d139640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:55:56.324 INFO:tasks.ceph.mds.d.smithi137.stderr:2024-04-20T10:55:56.324+0000 7f8f0d139640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-20T10:55:57.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:55:57.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.799+0000 7f7893055640 1 Processor -- start 2024-04-20T10:55:57.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.800+0000 7f7893055640 1 -- start start 2024-04-20T10:55:57.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.800+0000 7f7893055640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f788c0b54f0 0x7f788c0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.800+0000 7f7893055640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c0d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.800+0000 7f7893055640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f788c0b5e70 con 0x7f788c155950 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.800+0000 7f7893055640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f788c15b9c0 con 0x7f788c0b54f0 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.800+0000 7f7893055640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f788c15bca0 con 0x7f788c0d0c90 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f788bfff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f788c0b54f0 0x7f788c0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f78915f2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c0d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f7890df1640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f788c155950 0x7f788c0cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:51502/0 (socket says 172.21.15.27:51502) 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f7890df1640 1 -- 172.21.15.27:0/292838657 learned_addr learned my addr 172.21.15.27:0/292838657 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f788bfff640 1 -- 172.21.15.27:0/292838657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 msgr2=0x7f788c0d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f788bfff640 1 --2- 172.21.15.27:0/292838657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c0d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f788bfff640 1 -- 172.21.15.27:0/292838657 >> v1:172.21.15.137:6789/0 conn(0x7f788c155950 legacy=0x7f788c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f788b7fe640 1 -- 172.21.15.27:0/292838657 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710585068 0 0) 0x7f788c0b5e70 con 0x7f788c155950 2024-04-20T10:55:57.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f788bfff640 1 -- 172.21.15.27:0/292838657 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c15bf80 con 0x7f788c0b54f0 2024-04-20T10:55:57.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.801+0000 7f78915f2640 1 --2- 172.21.15.27:0/292838657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c0d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f788bfff640 1 --2- 172.21.15.27:0/292838657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 0x7f788c0b5930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78780097a0 tx=0x7f787802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=645dca1601875b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f788b7fe640 1 -- 172.21.15.27:0/292838657 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878034070 con 0x7f788c0b54f0 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f788b7fe640 1 -- 172.21.15.27:0/292838657 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7878030610 con 0x7f788c0b54f0 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f788b7fe640 1 -- 172.21.15.27:0/292838657 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78780309e0 con 0x7f788c0b54f0 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f7893055640 1 -- 172.21.15.27:0/292838657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 msgr2=0x7f788c0b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f7893055640 1 --2- 172.21.15.27:0/292838657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 0x7f788c0b5930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78780097a0 tx=0x7f787802b960 comp rx=0 tx=0).stop 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f7893055640 1 -- 172.21.15.27:0/292838657 shutdown_connections 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f7893055640 1 --2- 172.21.15.27:0/292838657 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c0d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f7893055640 1 --2- 172.21.15.27:0/292838657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 0x7f788c0b5930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.802+0000 7f7893055640 1 -- 172.21.15.27:0/292838657 >> 172.21.15.27:0/292838657 conn(0x7f788c0676c0 msgr2=0x7f788c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.803+0000 7f7893055640 1 -- 172.21.15.27:0/292838657 shutdown_connections 2024-04-20T10:55:57.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.803+0000 7f7893055640 1 -- 172.21.15.27:0/292838657 wait complete. 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.804+0000 7f7893055640 1 Processor -- start 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.804+0000 7f7893055640 1 -- start start 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.804+0000 7f7893055640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 0x7f788c15a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.804+0000 7f7893055640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c1f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.804+0000 7f7893055640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f788c155950 0x7f788c1f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7893055640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f788c0d1620 con 0x7f788c155950 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7893055640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f788c15c540 con 0x7f788c0b54f0 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7893055640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f788c15fb90 con 0x7f788c0d0c90 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f78915f2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f788c155950 0x7f788c1f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7890df1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 0x7f788c15a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f788bfff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c1f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f78915f2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f788c155950 0x7f788c1f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36406/0 (socket says 172.21.15.27:36406) 2024-04-20T10:55:57.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f78915f2640 1 -- 172.21.15.27:0/4285559626 learned_addr learned my addr 172.21.15.27:0/4285559626 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7890df1640 1 -- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 msgr2=0x7f788c1f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7890df1640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c1f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7890df1640 1 -- 172.21.15.27:0/4285559626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f788c155950 msgr2=0x7f788c1f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7890df1640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f788c155950 0x7f788c1f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7890df1640 1 -- 172.21.15.27:0/4285559626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c159b80 con 0x7f788c0b54f0 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f788bfff640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c1f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7890df1640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 0x7f788c15a480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f788000c480 tx=0x7f788000c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f78897fa640 1 -- 172.21.15.27:0/4285559626 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880018070 con 0x7f788c0b54f0 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f78897fa640 1 -- 172.21.15.27:0/4285559626 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78800074e0 con 0x7f788c0b54f0 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c159e60 con 0x7f788c0b54f0 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.805+0000 7f78897fa640 1 -- 172.21.15.27:0/4285559626 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880004a20 con 0x7f788c0b54f0 2024-04-20T10:55:57.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.806+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c1f1730 con 0x7f788c0b54f0 2024-04-20T10:55:57.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.806+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788c04fcf0 con 0x7f788c0b54f0 2024-04-20T10:55:57.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.807+0000 7f78897fa640 1 -- 172.21.15.27:0/4285559626 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7880004bc0 con 0x7f788c0b54f0 2024-04-20T10:55:57.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.807+0000 7f78897fa640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f787006c8f0 0x7f787006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:55:57.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.807+0000 7f78897fa640 1 -- 172.21.15.27:0/4285559626 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f788008adc0 con 0x7f788c0b54f0 2024-04-20T10:55:57.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.808+0000 7f788bfff640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f787006c8f0 0x7f787006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:55:57.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.808+0000 7f788bfff640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f787006c8f0 0x7f787006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7878036000 tx=0x7f7878008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:55:57.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:57.812+0000 7f78897fa640 1 -- 172.21.15.27:0/4285559626 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7880058b20 con 0x7f788c0b54f0 2024-04-20T10:55:58.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.035+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f788c0cd0a0 con 0x7f788c0b54f0 2024-04-20T10:55:58.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.036+0000 7f78897fa640 1 -- 172.21.15.27:0/4285559626 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f788005ce20 con 0x7f788c0b54f0 2024-04-20T10:55:58.037 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:55:58.037 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.037+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f787006c8f0 msgr2=0x7f787006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.037+0000 7f7893055640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f787006c8f0 0x7f787006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7878036000 tx=0x7f7878008030 comp rx=0 tx=0).stop 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.037+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 msgr2=0x7f788c15a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.037+0000 7f7893055640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 0x7f788c15a480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f788000c480 tx=0x7f788000c940 comp rx=0 tx=0).stop 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.038+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 shutdown_connections 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.038+0000 7f7893055640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f787006c8f0 0x7f787006edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.038+0000 7f7893055640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f788c155950 0x7f788c1f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.038+0000 7f7893055640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f788c0d0c90 0x7f788c1f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.038+0000 7f7893055640 1 --2- 172.21.15.27:0/4285559626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f788c0b54f0 0x7f788c15a480 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.038+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 >> 172.21.15.27:0/4285559626 conn(0x7f788c0676c0 msgr2=0x7f788c152020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.038+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 shutdown_connections 2024-04-20T10:55:58.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:55:58.038+0000 7f7893055640 1 -- 172.21.15.27:0/4285559626 wait complete. 2024-04-20T10:56:01.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:56:01.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.164+0000 7f374da74640 1 Processor -- start 2024-04-20T10:56:01.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.164+0000 7f374da74640 1 -- start start 2024-04-20T10:56:01.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.164+0000 7f374da74640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3748151f40 0x7f374806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:01.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.164+0000 7f374da74640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3748152320 0x7f374806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:01.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.164+0000 7f374da74640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 0x7f374806ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:01.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.164+0000 7f374da74640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f3748070210 con 0x7f3748151f40 2024-04-20T10:56:01.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f374da74640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f37480c9970 con 0x7f3748152320 2024-04-20T10:56:01.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f374da74640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f374815b750 con 0x7f374815e990 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f37477fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3748151f40 0x7f374806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f37477fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3748151f40 0x7f374806f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36424/0 (socket says 172.21.15.27:36424) 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f37477fe640 1 -- 172.21.15.27:0/1613396317 learned_addr learned my addr 172.21.15.27:0/1613396317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f3747fff640 1 --2- 172.21.15.27:0/1613396317 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 0x7f374806ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f3746ffd640 1 --2- 172.21.15.27:0/1613396317 >> v2:172.21.15.137:3300/0 conn(0x7f3748152320 0x7f374806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f37477fe640 1 -- 172.21.15.27:0/1613396317 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 msgr2=0x7f374806ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f37477fe640 1 --2- 172.21.15.27:0/1613396317 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 0x7f374806ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f37477fe640 1 -- 172.21.15.27:0/1613396317 >> v2:172.21.15.137:3300/0 conn(0x7f3748152320 msgr2=0x7f374806fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f37477fe640 1 --2- 172.21.15.27:0/1613396317 >> v2:172.21.15.137:3300/0 conn(0x7f3748152320 0x7f374806fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.165+0000 7f37477fe640 1 -- 172.21.15.27:0/1613396317 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374815ba30 con 0x7f3748151f40 2024-04-20T10:56:01.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.166+0000 7f37477fe640 1 --2- 172.21.15.27:0/1613396317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 0x7f374806f5c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f37480cc9c0 tx=0x7f373802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ff435e0343e5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.166+0000 7f37467fc640 1 -- 172.21.15.27:0/1613396317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738034070 con 0x7f3748151f40 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.166+0000 7f37467fc640 1 -- 172.21.15.27:0/1613396317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3738030840 con 0x7f3748151f40 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.166+0000 7f37467fc640 1 -- 172.21.15.27:0/1613396317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738030c10 con 0x7f3748151f40 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.167+0000 7f374da74640 1 -- 172.21.15.27:0/1613396317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 msgr2=0x7f374806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.167+0000 7f374da74640 1 --2- 172.21.15.27:0/1613396317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 0x7f374806f5c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f37480cc9c0 tx=0x7f373802b980 comp rx=0 tx=0).stop 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.167+0000 7f374da74640 1 -- 172.21.15.27:0/1613396317 shutdown_connections 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.167+0000 7f374da74640 1 --2- 172.21.15.27:0/1613396317 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 0x7f374806ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.167+0000 7f374da74640 1 --2- 172.21.15.27:0/1613396317 >> v2:172.21.15.137:3300/0 conn(0x7f3748152320 0x7f374806fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.167+0000 7f374da74640 1 --2- 172.21.15.27:0/1613396317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 0x7f374806f5c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.167+0000 7f374da74640 1 -- 172.21.15.27:0/1613396317 >> 172.21.15.27:0/1613396317 conn(0x7f37480cee90 msgr2=0x7f37480cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.167+0000 7f374da74640 1 -- 172.21.15.27:0/1613396317 shutdown_connections 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.167+0000 7f374da74640 1 -- 172.21.15.27:0/1613396317 wait complete. 2024-04-20T10:56:01.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.168+0000 7f374da74640 1 Processor -- start 2024-04-20T10:56:01.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.168+0000 7f374da74640 1 -- start start 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f374da74640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 0x7f374815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f374da74640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3748152320 0x7f37480ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f374da74640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 0x7f37480cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f374da74640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f374815c2d0 con 0x7f3748151f40 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f374da74640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f374815bff0 con 0x7f3748152320 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f374da74640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f374806f0a0 con 0x7f374815e990 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f37477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 0x7f374815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f37477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 0x7f374815d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36432/0 (socket says 172.21.15.27:36432) 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f37477fe640 1 -- 172.21.15.27:0/3534388103 learned_addr learned my addr 172.21.15.27:0/3534388103 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f3746ffd640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3748152320 0x7f37480ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f3747fff640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 0x7f37480cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f37477fe640 1 -- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 msgr2=0x7f37480cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f37477fe640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 0x7f37480cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f37477fe640 1 -- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3748152320 msgr2=0x7f37480ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f37477fe640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3748152320 0x7f37480ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f37477fe640 1 -- 172.21.15.27:0/3534388103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37481e5100 con 0x7f3748151f40 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.169+0000 7f3746ffd640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3748152320 0x7f37480ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.170+0000 7f37477fe640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 0x7f374815d8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3738000c00 tx=0x7f3738004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.170+0000 7f372ffff640 1 -- 172.21.15.27:0/3534388103 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738048070 con 0x7f3748151f40 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.170+0000 7f372ffff640 1 -- 172.21.15.27:0/3534388103 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3738039a90 con 0x7f3748151f40 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.170+0000 7f372ffff640 1 -- 172.21.15.27:0/3534388103 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738034040 con 0x7f3748151f40 2024-04-20T10:56:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.170+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37481e53e0 con 0x7f3748151f40 2024-04-20T10:56:01.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.170+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37481e59a0 con 0x7f3748151f40 2024-04-20T10:56:01.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.171+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374806dbc0 con 0x7f3748151f40 2024-04-20T10:56:01.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.171+0000 7f372ffff640 1 -- 172.21.15.27:0/3534388103 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3738037070 con 0x7f3748151f40 2024-04-20T10:56:01.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.172+0000 7f372ffff640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f373406c8f0 0x7f373406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:01.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.172+0000 7f372ffff640 1 -- 172.21.15.27:0/3534388103 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f37380ad670 con 0x7f3748151f40 2024-04-20T10:56:01.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.172+0000 7f3746ffd640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f373406c8f0 0x7f373406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:01.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.172+0000 7f3746ffd640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f373406c8f0 0x7f373406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f37481e4500 tx=0x7f373c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:01.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.176+0000 7f372ffff640 1 -- 172.21.15.27:0/3534388103 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f374806dbc0 con 0x7f3748151f40 2024-04-20T10:56:01.324 INFO:tasks.ceph.mds.d.smithi137.stderr:2024-04-20T10:56:01.325+0000 7f8f0d139640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-20T10:56:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.400+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f374804fcf0 con 0x7f3748151f40 2024-04-20T10:56:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.400+0000 7f372ffff640 1 -- 172.21.15.27:0/3534388103 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f373807b600 con 0x7f3748151f40 2024-04-20T10:56:01.401 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:01.401 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-20T10:56:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f373406c8f0 msgr2=0x7f373406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f373406c8f0 0x7f373406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f37481e4500 tx=0x7f373c009190 comp rx=0 tx=0).stop 2024-04-20T10:56:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 msgr2=0x7f374815d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 0x7f374815d8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3738000c00 tx=0x7f3738004730 comp rx=0 tx=0).stop 2024-04-20T10:56:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 shutdown_connections 2024-04-20T10:56:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f373406c8f0 0x7f373406edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f374815e990 0x7f37480cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3748152320 0x7f37480ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 --2- 172.21.15.27:0/3534388103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3748151f40 0x7f374815d8d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 >> 172.21.15.27:0/3534388103 conn(0x7f37480cee90 msgr2=0x7f37481e6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 shutdown_connections 2024-04-20T10:56:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:01.402+0000 7f374da74640 1 -- 172.21.15.27:0/3534388103 wait complete. 2024-04-20T10:56:04.414 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:56:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.526+0000 7fc5ceffd640 1 Processor -- start 2024-04-20T10:56:04.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.526+0000 7fc5ceffd640 1 -- start start 2024-04-20T10:56:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.527+0000 7fc5ceffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc5c8155950 0x7fc5c80cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.527+0000 7fc5ceffd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c815b9c0 con 0x7fc5c80cc330 2024-04-20T10:56:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.527+0000 7fc5ceffd640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c815bca0 con 0x7fc5c80b5c00 2024-04-20T10:56:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.527+0000 7fc5ceffd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c815bf80 con 0x7fc5c8155950 2024-04-20T10:56:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.527+0000 7fc5ccd99640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc5c8155950 0x7fc5c80cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.527+0000 7fc5ccd99640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc5c8155950 0x7fc5c80cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36444/0 (socket says 172.21.15.27:36444) 2024-04-20T10:56:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.527+0000 7fc5ccd99640 1 -- 172.21.15.27:0/1392037317 learned_addr learned my addr 172.21.15.27:0/1392037317 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.528+0000 7fc5ccd99640 1 -- 172.21.15.27:0/1392037317 >> v1:172.21.15.137:6789/0 conn(0x7fc5c80b5c00 legacy=0x7fc5c80d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:04.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.528+0000 7fc5ccd99640 1 -- 172.21.15.27:0/1392037317 >> v1:172.21.15.27:6789/0 conn(0x7fc5c80cc330 legacy=0x7fc5c80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.528+0000 7fc5ccd99640 1 -- 172.21.15.27:0/1392037317 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c815c260 con 0x7fc5c8155950 2024-04-20T10:56:04.529 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.528+0000 7fc5ccd99640 1 --2- 172.21.15.27:0/1392037317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c8155950 0x7fc5c80cbdf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009e50 tx=0x7fc5c002c090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb391074f1f73d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:04.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5c77fe640 1 -- 172.21.15.27:0/1392037317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c002cc30 con 0x7fc5c8155950 2024-04-20T10:56:04.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5c77fe640 1 -- 172.21.15.27:0/1392037317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c002cdd0 con 0x7fc5c8155950 2024-04-20T10:56:04.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5c77fe640 1 -- 172.21.15.27:0/1392037317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0034610 con 0x7fc5c8155950 2024-04-20T10:56:04.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5ceffd640 1 -- 172.21.15.27:0/1392037317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c8155950 msgr2=0x7fc5c80cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:04.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5ceffd640 1 --2- 172.21.15.27:0/1392037317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c8155950 0x7fc5c80cbdf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009e50 tx=0x7fc5c002c090 comp rx=0 tx=0).stop 2024-04-20T10:56:04.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5ceffd640 1 -- 172.21.15.27:0/1392037317 shutdown_connections 2024-04-20T10:56:04.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5ceffd640 1 --2- 172.21.15.27:0/1392037317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c8155950 0x7fc5c80cbdf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:04.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5ceffd640 1 -- 172.21.15.27:0/1392037317 >> 172.21.15.27:0/1392037317 conn(0x7fc5c80676c0 msgr2=0x7fc5c8067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:04.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5ceffd640 1 -- 172.21.15.27:0/1392037317 shutdown_connections 2024-04-20T10:56:04.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.529+0000 7fc5ceffd640 1 -- 172.21.15.27:0/1392037317 wait complete. 2024-04-20T10:56:04.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.530+0000 7fc5ceffd640 1 Processor -- start 2024-04-20T10:56:04.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.530+0000 7fc5ceffd640 1 -- start start 2024-04-20T10:56:04.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.531+0000 7fc5ceffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c80b5c00 0x7fc5c81f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:04.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.531+0000 7fc5ceffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5c80cc330 0x7fc5c8151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:04.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.531+0000 7fc5ceffd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5c8155950 0x7fc5c80cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:04.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.531+0000 7fc5ceffd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c815fd10 con 0x7fc5c80b5c00 2024-04-20T10:56:04.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.531+0000 7fc5ceffd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c815fb90 con 0x7fc5c80cc330 2024-04-20T10:56:04.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.531+0000 7fc5ceffd640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c8155d30 con 0x7fc5c8155950 2024-04-20T10:56:04.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.531+0000 7fc5ccd99640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c80b5c00 0x7fc5c81f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.531+0000 7fc5ccd99640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c80b5c00 0x7fc5c81f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36454/0 (socket says 172.21.15.27:36454) 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.531+0000 7fc5ccd99640 1 -- 172.21.15.27:0/4003042250 learned_addr learned my addr 172.21.15.27:0/4003042250 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5c7fff640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5c80cc330 0x7fc5c8151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5cd59a640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5c8155950 0x7fc5c80cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5ccd99640 1 -- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5c8155950 msgr2=0x7fc5c80cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5ccd99640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5c8155950 0x7fc5c80cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5ccd99640 1 -- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5c80cc330 msgr2=0x7fc5c8151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5ccd99640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5c80cc330 0x7fc5c8151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5ccd99640 1 -- 172.21.15.27:0/4003042250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c80c74d0 con 0x7fc5c80b5c00 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5ccd99640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c80b5c00 0x7fc5c81f5390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0036040 tx=0x7fc5c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:04.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5c57fa640 1 -- 172.21.15.27:0/4003042250 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0043020 con 0x7fc5c80b5c00 2024-04-20T10:56:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5c57fa640 1 -- 172.21.15.27:0/4003042250 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c003e450 con 0x7fc5c80b5c00 2024-04-20T10:56:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c81f22e0 con 0x7fc5c80b5c00 2024-04-20T10:56:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5c57fa640 1 -- 172.21.15.27:0/4003042250 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0009510 con 0x7fc5c80b5c00 2024-04-20T10:56:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.532+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c80c7a70 con 0x7fc5c80b5c00 2024-04-20T10:56:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.533+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c804fcf0 con 0x7fc5c80b5c00 2024-04-20T10:56:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.534+0000 7fc5c57fa640 1 -- 172.21.15.27:0/4003042250 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5c0046020 con 0x7fc5c80b5c00 2024-04-20T10:56:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.534+0000 7fc5c57fa640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc5a806c8f0 0x7fc5a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.534+0000 7fc5c57fa640 1 -- 172.21.15.27:0/4003042250 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc5c00ae330 con 0x7fc5c80b5c00 2024-04-20T10:56:04.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.534+0000 7fc5c7fff640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc5a806c8f0 0x7fc5a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:04.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.535+0000 7fc5c7fff640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc5a806c8f0 0x7fc5a806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc5c815ec70 tx=0x7fc5b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:04.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.539+0000 7fc5c57fa640 1 -- 172.21.15.27:0/4003042250 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc5c004d030 con 0x7fc5c80b5c00 2024-04-20T10:56:04.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.763+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc5c80cd0a0 con 0x7fc5c80b5c00 2024-04-20T10:56:04.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.764+0000 7fc5c57fa640 1 -- 172.21.15.27:0/4003042250 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7fc5c007c080 con 0x7fc5c80b5c00 2024-04-20T10:56:04.764 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc5a806c8f0 msgr2=0x7fc5a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc5a806c8f0 0x7fc5a806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc5c815ec70 tx=0x7fc5b0009040 comp rx=0 tx=0).stop 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c80b5c00 msgr2=0x7fc5c81f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c80b5c00 0x7fc5c81f5390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0036040 tx=0x7fc5c0004a40 comp rx=0 tx=0).stop 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 shutdown_connections 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc5a806c8f0 0x7fc5a806edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc5c8155950 0x7fc5c80cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5c80cc330 0x7fc5c8151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 --2- 172.21.15.27:0/4003042250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5c80b5c00 0x7fc5c81f5390 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 >> 172.21.15.27:0/4003042250 conn(0x7fc5c80676c0 msgr2=0x7fc5c806d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 shutdown_connections 2024-04-20T10:56:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:04.766+0000 7fc5ceffd640 1 -- 172.21.15.27:0/4003042250 wait complete. 2024-04-20T10:56:06.325 INFO:tasks.ceph.mds.d.smithi137.stderr:2024-04-20T10:56:06.325+0000 7f8f0d139640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-20T10:56:07.777 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:56:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.891+0000 7f259179d640 1 Processor -- start 2024-04-20T10:56:07.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.891+0000 7f259179d640 1 -- start start 2024-04-20T10:56:07.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.892+0000 7f259179d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 0x7f258c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:07.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.892+0000 7f259179d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f258c0b5c00 0x7f258c0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:07.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.892+0000 7f259179d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258c15b9c0 con 0x7f258c0cc330 2024-04-20T10:56:07.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.892+0000 7f259179d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f258c15bca0 con 0x7f258c0b5c00 2024-04-20T10:56:07.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.892+0000 7f259179d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f258c15bf80 con 0x7f258c155950 2024-04-20T10:56:07.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.892+0000 7f258b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f258c0b5c00 0x7f258c0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:07.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.892+0000 7f258b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f258c0b5c00 0x7f258c0d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59468/0 (socket says 172.21.15.27:59468) 2024-04-20T10:56:07.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.892+0000 7f258b7fe640 1 -- 172.21.15.27:0/1444013436 learned_addr learned my addr 172.21.15.27:0/1444013436 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:07.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.892+0000 7f258affd640 1 --2- 172.21.15.27:0/1444013436 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 0x7f258c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:07.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.893+0000 7f258b7fe640 1 -- 172.21.15.27:0/1444013436 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 msgr2=0x7f258c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:07.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.893+0000 7f258b7fe640 1 --2- 172.21.15.27:0/1444013436 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 0x7f258c0cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:07.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.893+0000 7f258b7fe640 1 -- 172.21.15.27:0/1444013436 >> v1:172.21.15.27:6789/0 conn(0x7f258c0cc330 legacy=0x7f258c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:07.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.893+0000 7f258b7fe640 1 -- 172.21.15.27:0/1444013436 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c15c260 con 0x7f258c0b5c00 2024-04-20T10:56:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.893+0000 7f258b7fe640 1 --2- 172.21.15.27:0/1444013436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0b5c00 0x7f258c0d0c90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f257c009e50 tx=0x7f257c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88c12eb262f1721 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f2589ffb640 1 -- 172.21.15.27:0/1444013436 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c02cb90 con 0x7f258c0b5c00 2024-04-20T10:56:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f2589ffb640 1 -- 172.21.15.27:0/1444013436 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c02cd30 con 0x7f258c0b5c00 2024-04-20T10:56:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f2589ffb640 1 -- 172.21.15.27:0/1444013436 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c034610 con 0x7f258c0b5c00 2024-04-20T10:56:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f259179d640 1 -- 172.21.15.27:0/1444013436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0b5c00 msgr2=0x7f258c0d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f259179d640 1 --2- 172.21.15.27:0/1444013436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0b5c00 0x7f258c0d0c90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f257c009e50 tx=0x7f257c02bff0 comp rx=0 tx=0).stop 2024-04-20T10:56:07.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f259179d640 1 -- 172.21.15.27:0/1444013436 shutdown_connections 2024-04-20T10:56:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f259179d640 1 --2- 172.21.15.27:0/1444013436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0b5c00 0x7f258c0d0c90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f259179d640 1 --2- 172.21.15.27:0/1444013436 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 0x7f258c0cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f259179d640 1 -- 172.21.15.27:0/1444013436 >> 172.21.15.27:0/1444013436 conn(0x7f258c0676c0 msgr2=0x7f258c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f259179d640 1 -- 172.21.15.27:0/1444013436 shutdown_connections 2024-04-20T10:56:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.894+0000 7f259179d640 1 -- 172.21.15.27:0/1444013436 wait complete. 2024-04-20T10:56:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.895+0000 7f259179d640 1 Processor -- start 2024-04-20T10:56:07.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f259179d640 1 -- start start 2024-04-20T10:56:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f259179d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c0b5c00 0x7f258c154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f259179d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0cc330 0x7f258c1e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f259179d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 0x7f258c0c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f259179d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f258c15fd10 con 0x7f258c0cc330 2024-04-20T10:56:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f259179d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f258c15fb90 con 0x7f258c0b5c00 2024-04-20T10:56:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f259179d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f258c155d30 con 0x7f258c155950 2024-04-20T10:56:07.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f258a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0cc330 0x7f258c1e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f258a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0cc330 0x7f258c1e4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59472/0 (socket says 172.21.15.27:59472) 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f258a7fc640 1 -- 172.21.15.27:0/635485573 learned_addr learned my addr 172.21.15.27:0/635485573 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f258b7fe640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 0x7f258c0c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.896+0000 7f258affd640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c0b5c00 0x7f258c154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f258a7fc640 1 -- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 msgr2=0x7f258c0c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f258a7fc640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 0x7f258c0c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f258a7fc640 1 -- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c0b5c00 msgr2=0x7f258c154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f258a7fc640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c0b5c00 0x7f258c154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f258a7fc640 1 -- 172.21.15.27:0/635485573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c1f4b00 con 0x7f258c0cc330 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f258b7fe640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 0x7f258c0c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f258affd640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c0b5c00 0x7f258c154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f258a7fc640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0cc330 0x7f258c1e4c00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f257400c400 tx=0x7f257400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f25837fe640 1 -- 172.21.15.27:0/635485573 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740052d0 con 0x7f258c0cc330 2024-04-20T10:56:07.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f25837fe640 1 -- 172.21.15.27:0/635485573 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2574005490 con 0x7f258c0cc330 2024-04-20T10:56:07.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c15e340 con 0x7f258c0cc330 2024-04-20T10:56:07.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f25837fe640 1 -- 172.21.15.27:0/635485573 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574007260 con 0x7f258c0cc330 2024-04-20T10:56:07.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.897+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c1f5080 con 0x7f258c0cc330 2024-04-20T10:56:07.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.898+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258c04fcf0 con 0x7f258c0cc330 2024-04-20T10:56:07.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.899+0000 7f25837fe640 1 -- 172.21.15.27:0/635485573 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2574007400 con 0x7f258c0cc330 2024-04-20T10:56:07.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.899+0000 7f25837fe640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f256c06c920 0x7f256c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:07.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.899+0000 7f25837fe640 1 -- 172.21.15.27:0/635485573 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f257408afd0 con 0x7f258c0cc330 2024-04-20T10:56:07.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.899+0000 7f258affd640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f256c06c920 0x7f256c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:07.900 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.900+0000 7f258affd640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f256c06c920 0x7f256c06ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2584009e90 tx=0x7f2584009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:07.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:07.904+0000 7f25837fe640 1 -- 172.21.15.27:0/635485573 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2574058db0 con 0x7f258c0cc330 2024-04-20T10:56:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.126+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f258c0b3dd0 con 0x7f258c0cc330 2024-04-20T10:56:08.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.126+0000 7f25837fe640 1 -- 172.21.15.27:0/635485573 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f257405d0b0 con 0x7f258c0cc330 2024-04-20T10:56:08.126 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:08.126 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T10:56:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f256c06c920 msgr2=0x7f256c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f256c06c920 0x7f256c06ede0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2584009e90 tx=0x7f2584009190 comp rx=0 tx=0).stop 2024-04-20T10:56:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0cc330 msgr2=0x7f258c1e4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0cc330 0x7f258c1e4c00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f257400c400 tx=0x7f257400c8c0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 shutdown_connections 2024-04-20T10:56:08.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f256c06c920 0x7f256c06ede0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f258c155950 0x7f258c0c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f258c0cc330 0x7f258c1e4c00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 --2- 172.21.15.27:0/635485573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f258c0b5c00 0x7f258c154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 >> 172.21.15.27:0/635485573 conn(0x7f258c0676c0 msgr2=0x7f258c0d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:08.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 shutdown_connections 2024-04-20T10:56:08.129 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.128+0000 7f259179d640 1 -- 172.21.15.27:0/635485573 wait complete. 2024-04-20T10:56:08.139 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-20T10:56:08.139 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:56:08.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.239+0000 7f9875596640 1 Processor -- start 2024-04-20T10:56:08.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.239+0000 7f9875596640 1 -- start start 2024-04-20T10:56:08.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f9875596640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f987006b930 0x7f98700c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f9875596640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9870159520 con 0x7f98700c77e0 2024-04-20T10:56:08.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f9875596640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9870159800 con 0x7f9870153440 2024-04-20T10:56:08.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f9875596640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f9870153820 con 0x7f987006b930 2024-04-20T10:56:08.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f986e7fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f987006b930 0x7f98700c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f986f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f98700c77e0 0x7f98700c8980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40846/0 (socket says 172.21.15.27:40846) 2024-04-20T10:56:08.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f986f7fe640 1 -- 172.21.15.27:0/1001239758 learned_addr learned my addr 172.21.15.27:0/1001239758 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:08.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f986e7fc640 1 -- 172.21.15.27:0/1001239758 >> v1:172.21.15.137:6789/0 conn(0x7f9870153440 legacy=0x7f987015d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f986e7fc640 1 -- 172.21.15.27:0/1001239758 >> v1:172.21.15.27:6789/0 conn(0x7f98700c77e0 legacy=0x7f98700c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.240+0000 7f986dffb640 1 -- 172.21.15.27:0/1001239758 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214526938 0 0) 0x7f9870159800 con 0x7f9870153440 2024-04-20T10:56:08.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.241+0000 7f986e7fc640 1 -- 172.21.15.27:0/1001239758 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9870159ae0 con 0x7f987006b930 2024-04-20T10:56:08.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.241+0000 7f986e7fc640 1 --2- 172.21.15.27:0/1001239758 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f987006b930 0x7f98700c72a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98580097d0 tx=0x7f985802b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b6303c19595ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:08.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.241+0000 7f986dffb640 1 -- 172.21.15.27:0/1001239758 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858037070 con 0x7f987006b930 2024-04-20T10:56:08.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.241+0000 7f986dffb640 1 -- 172.21.15.27:0/1001239758 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9858030710 con 0x7f987006b930 2024-04-20T10:56:08.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.241+0000 7f986dffb640 1 -- 172.21.15.27:0/1001239758 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858030ae0 con 0x7f987006b930 2024-04-20T10:56:08.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.242+0000 7f9875596640 1 -- 172.21.15.27:0/1001239758 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f987006b930 msgr2=0x7f98700c72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.242+0000 7f9875596640 1 --2- 172.21.15.27:0/1001239758 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f987006b930 0x7f98700c72a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98580097d0 tx=0x7f985802b770 comp rx=0 tx=0).stop 2024-04-20T10:56:08.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.242+0000 7f9875596640 1 -- 172.21.15.27:0/1001239758 shutdown_connections 2024-04-20T10:56:08.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.242+0000 7f9875596640 1 --2- 172.21.15.27:0/1001239758 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f987006b930 0x7f98700c72a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.242+0000 7f9875596640 1 -- 172.21.15.27:0/1001239758 >> 172.21.15.27:0/1001239758 conn(0x7f987006f950 msgr2=0x7f987006fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:08.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.242+0000 7f9875596640 1 -- 172.21.15.27:0/1001239758 shutdown_connections 2024-04-20T10:56:08.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.242+0000 7f9875596640 1 -- 172.21.15.27:0/1001239758 wait complete. 2024-04-20T10:56:08.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.243+0000 7f9875596640 1 Processor -- start 2024-04-20T10:56:08.243 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.243+0000 7f9875596640 1 -- start start 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f9875596640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f987006b930 0x7f9870155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f9875596640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98700c77e0 0x7f98701f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f9875596640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9870153440 0x7f98701f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f9875596640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f98700c8460 con 0x7f9870153440 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f9875596640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f987015a0a0 con 0x7f987006b930 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f9875596640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f98701590e0 con 0x7f98700c77e0 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9870153440 0x7f98701f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9870153440 0x7f98701f49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59478/0 (socket says 172.21.15.27:59478) 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986f7fe640 1 -- 172.21.15.27:0/772219206 learned_addr learned my addr 172.21.15.27:0/772219206 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986effd640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f987006b930 0x7f9870155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986e7fc640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98700c77e0 0x7f98701f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986f7fe640 1 -- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98700c77e0 msgr2=0x7f98701f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986f7fe640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98700c77e0 0x7f98701f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986f7fe640 1 -- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f987006b930 msgr2=0x7f9870155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986f7fe640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f987006b930 0x7f9870155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.244+0000 7f986f7fe640 1 -- 172.21.15.27:0/772219206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9870148f80 con 0x7f9870153440 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.245+0000 7f986f7fe640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9870153440 0x7f98701f49a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f986000c8f0 tx=0x7f986000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.245+0000 7f98677fe640 1 -- 172.21.15.27:0/772219206 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860018070 con 0x7f9870153440 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.245+0000 7f98677fe640 1 -- 172.21.15.27:0/772219206 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986000a5e0 con 0x7f9870153440 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.245+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98701e50a0 con 0x7f9870153440 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.245+0000 7f98677fe640 1 -- 172.21.15.27:0/772219206 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986000aa10 con 0x7f9870153440 2024-04-20T10:56:08.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.245+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98701f5070 con 0x7f9870153440 2024-04-20T10:56:08.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.246+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987004fcf0 con 0x7f9870153440 2024-04-20T10:56:08.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.247+0000 7f98677fe640 1 -- 172.21.15.27:0/772219206 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f986001b070 con 0x7f9870153440 2024-04-20T10:56:08.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.247+0000 7f98677fe640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f985006c8f0 0x7f985006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.247+0000 7f98677fe640 1 -- 172.21.15.27:0/772219206 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f986008b130 con 0x7f9870153440 2024-04-20T10:56:08.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.247+0000 7f986effd640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f985006c8f0 0x7f985006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.247+0000 7f986effd640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f985006c8f0 0x7f985006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9868001120 tx=0x7f9868009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:08.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.251+0000 7f98677fe640 1 -- 172.21.15.27:0/772219206 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9870148f80 con 0x7f9870153440 2024-04-20T10:56:08.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.440+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98700c7c70 con 0x7f9870153440 2024-04-20T10:56:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.441+0000 7f98677fe640 1 -- 172.21.15.27:0/772219206 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16569 (secure 0 0 0) 0x7f9860058f10 con 0x7f9870153440 2024-04-20T10:56:08.441 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:08.441 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:43.787038+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:56:08.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f985006c8f0 msgr2=0x7f985006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f985006c8f0 0x7f985006edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9868001120 tx=0x7f9868009040 comp rx=0 tx=0).stop 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9870153440 msgr2=0x7f98701f49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9870153440 0x7f98701f49a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f986000c8f0 tx=0x7f986000cdb0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 shutdown_connections 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f985006c8f0 0x7f985006edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9870153440 0x7f98701f49a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f98700c77e0 0x7f98701f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 --2- 172.21.15.27:0/772219206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f987006b930 0x7f9870155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 >> 172.21.15.27:0/772219206 conn(0x7f987006f950 msgr2=0x7f987014aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 shutdown_connections 2024-04-20T10:56:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.444+0000 7f9875596640 1 -- 172.21.15.27:0/772219206 wait complete. 2024-04-20T10:56:08.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:55:29.717620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:55:39.832461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:55:41.103975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:56:08.455 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:56:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.559+0000 7fa45bbe6640 1 Processor -- start 2024-04-20T10:56:08.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.559+0000 7fa45bbe6640 1 -- start start 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa45bbe6640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa454155800 0x7fa45415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa45bbe6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa45415c4d0 con 0x7fa4540d1260 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa45bbe6640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4540cd290 con 0x7fa4540cbf50 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa45bbe6640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fa4540ce610 con 0x7fa454155800 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa45a183640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa4540d1260 0x7fa454165e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40848/0 (socket says 172.21.15.27:40848) 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa45a183640 1 -- 172.21.15.27:0/2814296872 learned_addr learned my addr 172.21.15.27:0/2814296872 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa459982640 1 --2- 172.21.15.27:0/2814296872 >> v2:172.21.15.137:3300/0 conn(0x7fa454155800 0x7fa45415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa458980640 1 -- 172.21.15.27:0/2814296872 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1886511168 0 0) 0x7fa44c001490 con 0x7fa4540d1260 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa458980640 1 -- 172.21.15.27:0/2814296872 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa440003520 con 0x7fa4540d1260 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa459982640 1 -- 172.21.15.27:0/2814296872 >> v1:172.21.15.137:6789/0 conn(0x7fa4540cbf50 legacy=0x7fa4540c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa458980640 1 -- 172.21.15.27:0/2814296872 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245825147 0 0) 0x7fa4540cd290 con 0x7fa4540cbf50 2024-04-20T10:56:08.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa459982640 1 -- 172.21.15.27:0/2814296872 >> v1:172.21.15.27:6789/0 conn(0x7fa4540d1260 legacy=0x7fa454165e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.560+0000 7fa459982640 1 -- 172.21.15.27:0/2814296872 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4540cd570 con 0x7fa454155800 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.561+0000 7fa459982640 1 --2- 172.21.15.27:0/2814296872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa454155800 0x7fa45415bf90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa4500097a0 tx=0x7fa45002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89576d7e704fba7d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.561+0000 7fa458980640 1 -- 172.21.15.27:0/2814296872 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450035020 con 0x7fa454155800 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.561+0000 7fa458980640 1 -- 172.21.15.27:0/2814296872 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa450031830 con 0x7fa454155800 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.561+0000 7fa458980640 1 -- 172.21.15.27:0/2814296872 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450031c00 con 0x7fa454155800 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.562+0000 7fa45bbe6640 1 -- 172.21.15.27:0/2814296872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa454155800 msgr2=0x7fa45415bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.562+0000 7fa45bbe6640 1 --2- 172.21.15.27:0/2814296872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa454155800 0x7fa45415bf90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa4500097a0 tx=0x7fa45002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.562+0000 7fa45bbe6640 1 -- 172.21.15.27:0/2814296872 shutdown_connections 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.562+0000 7fa45bbe6640 1 --2- 172.21.15.27:0/2814296872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa454155800 0x7fa45415bf90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.562+0000 7fa45bbe6640 1 -- 172.21.15.27:0/2814296872 >> 172.21.15.27:0/2814296872 conn(0x7fa454067ae0 msgr2=0x7fa454067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.562+0000 7fa45bbe6640 1 -- 172.21.15.27:0/2814296872 shutdown_connections 2024-04-20T10:56:08.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.562+0000 7fa45bbe6640 1 -- 172.21.15.27:0/2814296872 wait complete. 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.563+0000 7fa45bbe6640 1 Processor -- start 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.563+0000 7fa45bbe6640 1 -- start start 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.563+0000 7fa45bbe6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4540cbf50 0x7fa4540b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45bbe6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4540d1260 0x7fa4540b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45bbe6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa454155800 0x7fa454149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45bbe6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4540cdb30 con 0x7fa454155800 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45bbe6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa4540ce180 con 0x7fa4540d1260 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45bbe6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fa4540cde10 con 0x7fa4540cbf50 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45a183640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa454155800 0x7fa454149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45a183640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa454155800 0x7fa454149580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59486/0 (socket says 172.21.15.27:59486) 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45a183640 1 -- 172.21.15.27:0/448445149 learned_addr learned my addr 172.21.15.27:0/448445149 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa459982640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4540cbf50 0x7fa4540b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa459181640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4540d1260 0x7fa4540b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45a183640 1 -- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4540cbf50 msgr2=0x7fa4540b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45a183640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4540cbf50 0x7fa4540b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45a183640 1 -- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4540d1260 msgr2=0x7fa4540b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45a183640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4540d1260 0x7fa4540b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45a183640 1 -- 172.21.15.27:0/448445149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4541583f0 con 0x7fa454155800 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45a183640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa454155800 0x7fa454149580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00cb50 tx=0x7fa44c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa44a7fc640 1 -- 172.21.15.27:0/448445149 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c017070 con 0x7fa454155800 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa44a7fc640 1 -- 172.21.15.27:0/448445149 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa44c00ad40 con 0x7fa454155800 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4541586b0 con 0x7fa454155800 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa44a7fc640 1 -- 172.21.15.27:0/448445149 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c012620 con 0x7fa454155800 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.564+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4540b3d30 con 0x7fa454155800 2024-04-20T10:56:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.565+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4540688e0 con 0x7fa454155800 2024-04-20T10:56:08.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.566+0000 7fa44a7fc640 1 -- 172.21.15.27:0/448445149 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa44c01a070 con 0x7fa454155800 2024-04-20T10:56:08.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.566+0000 7fa44a7fc640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa43806c920 0x7fa43806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.566+0000 7fa44a7fc640 1 -- 172.21.15.27:0/448445149 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa44c08b800 con 0x7fa454155800 2024-04-20T10:56:08.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.566+0000 7fa459982640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa43806c920 0x7fa43806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.567+0000 7fa459982640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa43806c920 0x7fa43806ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa450036000 tx=0x7fa45003c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.570+0000 7fa44a7fc640 1 -- 172.21.15.27:0/448445149 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa45415c4d0 con 0x7fa454155800 2024-04-20T10:56:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.754+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa4540b3f30 con 0x7fa454155800 2024-04-20T10:56:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.755+0000 7fa44a7fc640 1 -- 172.21.15.27:0/448445149 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa4540b3f30 con 0x7fa454155800 2024-04-20T10:56:08.754 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-20T10:56:08.756 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.756+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa43806c920 msgr2=0x7fa43806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.756+0000 7fa45bbe6640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa43806c920 0x7fa43806ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa450036000 tx=0x7fa45003c040 comp rx=0 tx=0).stop 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.756+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa454155800 msgr2=0x7fa454149580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.756+0000 7fa45bbe6640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa454155800 0x7fa454149580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00cb50 tx=0x7fa44c00a430 comp rx=0 tx=0).stop 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.756+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 shutdown_connections 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.756+0000 7fa45bbe6640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa43806c920 0x7fa43806ede0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.756+0000 7fa45bbe6640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa454155800 0x7fa454149580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.756+0000 7fa45bbe6640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4540d1260 0x7fa4540b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.756+0000 7fa45bbe6640 1 --2- 172.21.15.27:0/448445149 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4540cbf50 0x7fa4540b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.757+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 >> 172.21.15.27:0/448445149 conn(0x7fa454067ae0 msgr2=0x7fa45406e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.757+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 shutdown_connections 2024-04-20T10:56:08.757 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.757+0000 7fa45bbe6640 1 -- 172.21.15.27:0/448445149 wait complete. 2024-04-20T10:56:08.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:56:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.871+0000 7fbcca3f4640 1 Processor -- start 2024-04-20T10:56:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.871+0000 7fbcca3f4640 1 -- start start 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcca3f4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc40c9800 0x7fbcc40ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcca3f4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbcc40cf2d0 0x7fbcc40cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcca3f4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcc406cbe0 con 0x7fbcc4155a10 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcca3f4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc40c9c60 con 0x7fbcc40cf2d0 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcca3f4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc406cec0 con 0x7fbcc40c9800 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcc8991640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbcc40cf2d0 0x7fbcc40cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcc8991640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbcc40cf2d0 0x7fbcc40cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59494/0 (socket says 172.21.15.27:59494) 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcc8991640 1 -- 172.21.15.27:0/658126925 learned_addr learned my addr 172.21.15.27:0/658126925 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcc37fe640 1 --2- 172.21.15.27:0/658126925 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc40c9800 0x7fbcc40ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcc8991640 1 -- 172.21.15.27:0/658126925 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc40c9800 msgr2=0x7fbcc40ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcc8991640 1 --2- 172.21.15.27:0/658126925 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc40c9800 0x7fbcc40ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcc8991640 1 -- 172.21.15.27:0/658126925 >> v1:172.21.15.27:6789/0 conn(0x7fbcc4155a10 legacy=0x7fbcc415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.872+0000 7fbcc8991640 1 -- 172.21.15.27:0/658126925 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc406d1a0 con 0x7fbcc40cf2d0 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.873+0000 7fbcc8991640 1 --2- 172.21.15.27:0/658126925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40cf2d0 0x7fbcc40cab80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009e50 tx=0x7fbcbc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236f766fc7d3f260 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.873+0000 7fbcc2ffd640 1 -- 172.21.15.27:0/658126925 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc02cb00 con 0x7fbcc40cf2d0 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.873+0000 7fbcc2ffd640 1 -- 172.21.15.27:0/658126925 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc02cca0 con 0x7fbcc40cf2d0 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.873+0000 7fbcc2ffd640 1 -- 172.21.15.27:0/658126925 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc034610 con 0x7fbcc40cf2d0 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.873+0000 7fbcca3f4640 1 -- 172.21.15.27:0/658126925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40cf2d0 msgr2=0x7fbcc40cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.873+0000 7fbcca3f4640 1 --2- 172.21.15.27:0/658126925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40cf2d0 0x7fbcc40cab80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009e50 tx=0x7fbcbc02bf60 comp rx=0 tx=0).stop 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.873+0000 7fbcca3f4640 1 -- 172.21.15.27:0/658126925 shutdown_connections 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.873+0000 7fbcca3f4640 1 --2- 172.21.15.27:0/658126925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40cf2d0 0x7fbcc40cab80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.873+0000 7fbcca3f4640 1 --2- 172.21.15.27:0/658126925 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc40c9800 0x7fbcc40ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.874+0000 7fbcca3f4640 1 -- 172.21.15.27:0/658126925 >> 172.21.15.27:0/658126925 conn(0x7fbcc40676c0 msgr2=0x7fbcc4067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.874+0000 7fbcca3f4640 1 -- 172.21.15.27:0/658126925 shutdown_connections 2024-04-20T10:56:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.874+0000 7fbcca3f4640 1 -- 172.21.15.27:0/658126925 wait complete. 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.875+0000 7fbcca3f4640 1 Processor -- start 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.875+0000 7fbcca3f4640 1 -- start start 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcca3f4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40c9800 0x7fbcc41f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcca3f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcc40cf2d0 0x7fbcc414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcca3f4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc4155a10 0x7fbcc414fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcca3f4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4155f70 con 0x7fbcc40c9800 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcca3f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4155df0 con 0x7fbcc40cf2d0 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcca3f4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc40718c0 con 0x7fbcc4155a10 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40c9800 0x7fbcc41f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40c9800 0x7fbcc41f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59496/0 (socket says 172.21.15.27:59496) 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc3fff640 1 -- 172.21.15.27:0/3648386620 learned_addr learned my addr 172.21.15.27:0/3648386620 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc8991640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc4155a10 0x7fbcc414fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc37fe640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcc40cf2d0 0x7fbcc414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc3fff640 1 -- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc4155a10 msgr2=0x7fbcc414fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc3fff640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc4155a10 0x7fbcc414fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc3fff640 1 -- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcc40cf2d0 msgr2=0x7fbcc414f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc3fff640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcc40cf2d0 0x7fbcc414f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc3fff640 1 -- 172.21.15.27:0/3648386620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc4152030 con 0x7fbcc40c9800 2024-04-20T10:56:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc3fff640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40c9800 0x7fbcc41f5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00c3c0 tx=0x7fbcac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc0ff9640 1 -- 172.21.15.27:0/3648386620 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0052d0 con 0x7fbcc40c9800 2024-04-20T10:56:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc0ff9640 1 -- 172.21.15.27:0/3648386620 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcac005490 con 0x7fbcc40c9800 2024-04-20T10:56:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc40c66a0 con 0x7fbcc40c9800 2024-04-20T10:56:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcc0ff9640 1 -- 172.21.15.27:0/3648386620 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac007260 con 0x7fbcc40c9800 2024-04-20T10:56:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.876+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc40b4b90 con 0x7fbcc40c9800 2024-04-20T10:56:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.877+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc41f4910 con 0x7fbcc40c9800 2024-04-20T10:56:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.878+0000 7fbcc0ff9640 1 -- 172.21.15.27:0/3648386620 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcac007400 con 0x7fbcc40c9800 2024-04-20T10:56:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.878+0000 7fbcc0ff9640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbca406c8f0 0x7fbca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.878+0000 7fbcc0ff9640 1 -- 172.21.15.27:0/3648386620 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbcac08af40 con 0x7fbcc40c9800 2024-04-20T10:56:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.878+0000 7fbcc37fe640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbca406c8f0 0x7fbca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.879+0000 7fbcc37fe640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbca406c8f0 0x7fbca406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbcc41550f0 tx=0x7fbcb4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:08.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:08.882+0000 7fbcc0ff9640 1 -- 172.21.15.27:0/3648386620 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbcac058d20 con 0x7fbcc40c9800 2024-04-20T10:56:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.068+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbcc404fcf0 con 0x7fbcc40c9800 2024-04-20T10:56:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.069+0000 7fbcc0ff9640 1 -- 172.21.15.27:0/3648386620 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fbcac05f030 con 0x7fbcc40c9800 2024-04-20T10:56:09.069 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbca406c8f0 msgr2=0x7fbca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbca406c8f0 0x7fbca406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbcc41550f0 tx=0x7fbcb4009190 comp rx=0 tx=0).stop 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40c9800 msgr2=0x7fbcc41f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40c9800 0x7fbcc41f5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00c3c0 tx=0x7fbcac00c880 comp rx=0 tx=0).stop 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 shutdown_connections 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbca406c8f0 0x7fbca406edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbcc4155a10 0x7fbcc414fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcc40cf2d0 0x7fbcc414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 --2- 172.21.15.27:0/3648386620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc40c9800 0x7fbcc41f5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 >> 172.21.15.27:0/3648386620 conn(0x7fbcc40676c0 msgr2=0x7fbcc40c3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 shutdown_connections 2024-04-20T10:56:09.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.071+0000 7fbcca3f4640 1 -- 172.21.15.27:0/3648386620 wait complete. 2024-04-20T10:56:09.082 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:56:09.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.185+0000 7ff77cf4a640 1 Processor -- start 2024-04-20T10:56:09.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.185+0000 7ff77cf4a640 1 -- start start 2024-04-20T10:56:09.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff77cf4a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff77815edc0 0x7ff7780c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff77cf4a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7780cc5a0 con 0x7ff7780cd4f0 2024-04-20T10:56:09.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff77cf4a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7780cc880 con 0x7ff77815f8b0 2024-04-20T10:56:09.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff77cf4a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7ff7780cc350 con 0x7ff77815edc0 2024-04-20T10:56:09.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff76ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff7780cd4f0 0x7ff77815f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40854/0 (socket says 172.21.15.27:40854) 2024-04-20T10:56:09.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff76ffff640 1 -- 172.21.15.27:0/4025481180 learned_addr learned my addr 172.21.15.27:0/4025481180 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:09.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff77659c640 1 --2- 172.21.15.27:0/4025481180 >> v2:172.21.15.137:3300/0 conn(0x7ff77815edc0 0x7ff7780c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff775d9b640 1 -- 172.21.15.27:0/4025481180 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1806588481 0 0) 0x7ff7780cc5a0 con 0x7ff7780cd4f0 2024-04-20T10:56:09.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff775d9b640 1 -- 172.21.15.27:0/4025481180 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff764003520 con 0x7ff7780cd4f0 2024-04-20T10:56:09.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff77659c640 1 -- 172.21.15.27:0/4025481180 >> v1:172.21.15.137:6789/0 conn(0x7ff77815f8b0 legacy=0x7ff778155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff77659c640 1 -- 172.21.15.27:0/4025481180 >> v1:172.21.15.27:6789/0 conn(0x7ff7780cd4f0 legacy=0x7ff77815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff775d9b640 1 -- 172.21.15.27:0/4025481180 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279848454 0 0) 0x7ff7780cc880 con 0x7ff77815f8b0 2024-04-20T10:56:09.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.186+0000 7ff77659c640 1 -- 172.21.15.27:0/4025481180 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7780c30b0 con 0x7ff77815edc0 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.187+0000 7ff77659c640 1 --2- 172.21.15.27:0/4025481180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77815edc0 0x7ff7780c9620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7680097a0 tx=0x7ff76802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1717bc70524a635c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.187+0000 7ff775d9b640 1 -- 172.21.15.27:0/4025481180 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76802f810 con 0x7ff77815edc0 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.187+0000 7ff775d9b640 1 -- 172.21.15.27:0/4025481180 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76802f9b0 con 0x7ff77815edc0 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.187+0000 7ff775d9b640 1 -- 172.21.15.27:0/4025481180 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76802fd60 con 0x7ff77815edc0 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.188+0000 7ff77cf4a640 1 -- 172.21.15.27:0/4025481180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77815edc0 msgr2=0x7ff7780c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.188+0000 7ff77cf4a640 1 --2- 172.21.15.27:0/4025481180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77815edc0 0x7ff7780c9620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7680097a0 tx=0x7ff76802b9f0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.188+0000 7ff77cf4a640 1 -- 172.21.15.27:0/4025481180 shutdown_connections 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.188+0000 7ff77cf4a640 1 --2- 172.21.15.27:0/4025481180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff77815edc0 0x7ff7780c9620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.188+0000 7ff77cf4a640 1 -- 172.21.15.27:0/4025481180 >> 172.21.15.27:0/4025481180 conn(0x7ff77806fc90 msgr2=0x7ff778070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:09.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.188+0000 7ff77cf4a640 1 -- 172.21.15.27:0/4025481180 shutdown_connections 2024-04-20T10:56:09.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.188+0000 7ff77cf4a640 1 -- 172.21.15.27:0/4025481180 wait complete. 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.189+0000 7ff77cf4a640 1 Processor -- start 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.189+0000 7ff77cf4a640 1 -- start start 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff77cf4a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7780cd4f0 0x7ff778152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff77cf4a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815edc0 0x7ff778156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff77cf4a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff778156ba0 0x7ff7781502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff77cf4a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780c3670 con 0x7ff77815edc0 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff77cf4a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780cd940 con 0x7ff7780cd4f0 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff77cf4a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff7780c8b70 con 0x7ff778156ba0 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff76ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815edc0 0x7ff778156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff76ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815edc0 0x7ff778156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59512/0 (socket says 172.21.15.27:59512) 2024-04-20T10:56:09.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff76ffff640 1 -- 172.21.15.27:0/3061169308 learned_addr learned my addr 172.21.15.27:0/3061169308 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff77659c640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7780cd4f0 0x7ff778152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff776d9d640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff778156ba0 0x7ff7781502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff76ffff640 1 -- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff778156ba0 msgr2=0x7ff7781502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff76ffff640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff778156ba0 0x7ff7781502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff76ffff640 1 -- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7780cd4f0 msgr2=0x7ff778152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff76ffff640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7780cd4f0 0x7ff778152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff76ffff640 1 -- 172.21.15.27:0/3061169308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7780ca700 con 0x7ff77815edc0 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff77659c640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7780cd4f0 0x7ff778152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.190+0000 7ff76ffff640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815edc0 0x7ff778156660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff76000c430 tx=0x7ff76000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.191+0000 7ff76effd640 1 -- 172.21.15.27:0/3061169308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760017070 con 0x7ff77815edc0 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.191+0000 7ff76effd640 1 -- 172.21.15.27:0/3061169308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff760005ce0 con 0x7ff77815edc0 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.191+0000 7ff76effd640 1 -- 172.21.15.27:0/3061169308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7600072f0 con 0x7ff77815edc0 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.191+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff778155930 con 0x7ff77815edc0 2024-04-20T10:56:09.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.191+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77815a830 con 0x7ff77815edc0 2024-04-20T10:56:09.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.192+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77804fcf0 con 0x7ff77815edc0 2024-04-20T10:56:09.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.192+0000 7ff76effd640 1 -- 172.21.15.27:0/3061169308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff76001a070 con 0x7ff77815edc0 2024-04-20T10:56:09.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.192+0000 7ff76effd640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff75806c8f0 0x7ff75806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.193+0000 7ff76effd640 1 -- 172.21.15.27:0/3061169308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff76008aef0 con 0x7ff77815edc0 2024-04-20T10:56:09.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.193+0000 7ff77659c640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff75806c8f0 0x7ff75806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.193+0000 7ff77659c640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff75806c8f0 0x7ff75806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff768036000 tx=0x7ff768008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.197+0000 7ff76effd640 1 -- 172.21.15.27:0/3061169308 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff760058d60 con 0x7ff77815edc0 2024-04-20T10:56:09.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.385+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff778073d80 con 0x7ff77815edc0 2024-04-20T10:56:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.385+0000 7ff76effd640 1 -- 172.21.15.27:0/3061169308 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff76005d060 con 0x7ff77815edc0 2024-04-20T10:56:09.385 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff75806c8f0 msgr2=0x7ff75806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff75806c8f0 0x7ff75806edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff768036000 tx=0x7ff768008030 comp rx=0 tx=0).stop 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815edc0 msgr2=0x7ff778156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815edc0 0x7ff778156660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff76000c430 tx=0x7ff76000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 shutdown_connections 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff75806c8f0 0x7ff75806edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff778156ba0 0x7ff7781502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815edc0 0x7ff778156660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 --2- 172.21.15.27:0/3061169308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7780cd4f0 0x7ff778152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 >> 172.21.15.27:0/3061169308 conn(0x7ff77806fc90 msgr2=0x7ff778072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 shutdown_connections 2024-04-20T10:56:09.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.387+0000 7ff77cf4a640 1 -- 172.21.15.27:0/3061169308 wait complete. 2024-04-20T10:56:09.397 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:56:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.498+0000 7f48a0c9a640 1 Processor -- start 2024-04-20T10:56:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.498+0000 7f48a0c9a640 1 -- start start 2024-04-20T10:56:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f48a0c9a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f489c15ed10 0x7f489c073970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f48a0c9a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15f0f0 0x7f489c15f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f48a0c9a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f489c073fc0 con 0x7f489c072130 2024-04-20T10:56:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f48a0c9a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f489c0c3bc0 con 0x7f489c15ed10 2024-04-20T10:56:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f48a0c9a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f489c0c4050 con 0x7f489c15f0f0 2024-04-20T10:56:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f489a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f489c15ed10 0x7f489c073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f489a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f489c15ed10 0x7f489c073970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59528/0 (socket says 172.21.15.27:59528) 2024-04-20T10:56:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f489ad9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15f0f0 0x7f489c15f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f489a59c640 1 -- 172.21.15.27:0/3836353330 learned_addr learned my addr 172.21.15.27:0/3836353330 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f489a59c640 1 -- 172.21.15.27:0/3836353330 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15f0f0 msgr2=0x7f489c15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f489a59c640 1 --2- 172.21.15.27:0/3836353330 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15f0f0 0x7f489c15f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f489a59c640 1 -- 172.21.15.27:0/3836353330 >> v1:172.21.15.137:6789/0 conn(0x7f489c072130 legacy=0x7f489c0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f4899d9b640 1 -- 172.21.15.27:0/3836353330 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1794096263 0 0) 0x7f489c073fc0 con 0x7f489c072130 2024-04-20T10:56:09.500 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.499+0000 7f489a59c640 1 -- 172.21.15.27:0/3836353330 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c0c2fb0 con 0x7f489c15ed10 2024-04-20T10:56:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.500+0000 7f489a59c640 1 --2- 172.21.15.27:0/3836353330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15ed10 0x7f489c073970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f488c009870 tx=0x7f488c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366e00758608fd98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.500+0000 7f4899d9b640 1 -- 172.21.15.27:0/3836353330 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c034070 con 0x7f489c15ed10 2024-04-20T10:56:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.500+0000 7f4899d9b640 1 -- 172.21.15.27:0/3836353330 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488c030840 con 0x7f489c15ed10 2024-04-20T10:56:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.500+0000 7f4899d9b640 1 -- 172.21.15.27:0/3836353330 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c030c10 con 0x7f489c15ed10 2024-04-20T10:56:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.501+0000 7f48a0c9a640 1 -- 172.21.15.27:0/3836353330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15ed10 msgr2=0x7f489c073970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.501+0000 7f48a0c9a640 1 --2- 172.21.15.27:0/3836353330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15ed10 0x7f489c073970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f488c009870 tx=0x7f488c02b980 comp rx=0 tx=0).stop 2024-04-20T10:56:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.501+0000 7f48a0c9a640 1 -- 172.21.15.27:0/3836353330 shutdown_connections 2024-04-20T10:56:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.501+0000 7f48a0c9a640 1 --2- 172.21.15.27:0/3836353330 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15f0f0 0x7f489c15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.501+0000 7f48a0c9a640 1 --2- 172.21.15.27:0/3836353330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15ed10 0x7f489c073970 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.501+0000 7f48a0c9a640 1 -- 172.21.15.27:0/3836353330 >> 172.21.15.27:0/3836353330 conn(0x7f489c0ca8f0 msgr2=0x7f489c0cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.501+0000 7f48a0c9a640 1 -- 172.21.15.27:0/3836353330 shutdown_connections 2024-04-20T10:56:09.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.501+0000 7f48a0c9a640 1 -- 172.21.15.27:0/3836353330 wait complete. 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.502+0000 7f48a0c9a640 1 Processor -- start 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.502+0000 7f48a0c9a640 1 -- start start 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f48a0c9a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f489c072130 0x7f489c1e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f48a0c9a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15ed10 0x7f489c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f48a0c9a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15f0f0 0x7f489c15d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f48a0c9a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f489c0c3850 con 0x7f489c15f0f0 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f48a0c9a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f489c0c3570 con 0x7f489c072130 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f48a0c9a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f489c0c8f00 con 0x7f489c15ed10 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15f0f0 0x7f489c15d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15f0f0 0x7f489c15d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59538/0 (socket says 172.21.15.27:59538) 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f489c072130 0x7f489c1e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489ad9d640 1 -- 172.21.15.27:0/1638045262 learned_addr learned my addr 172.21.15.27:0/1638045262 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:09.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f4893fff640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15ed10 0x7f489c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489ad9d640 1 -- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15ed10 msgr2=0x7f489c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489ad9d640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15ed10 0x7f489c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489ad9d640 1 -- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f489c072130 msgr2=0x7f489c1e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489ad9d640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f489c072130 0x7f489c1e5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489ad9d640 1 -- 172.21.15.27:0/1638045262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c1e63f0 con 0x7f489c15f0f0 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489a59c640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f489c072130 0x7f489c1e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f489ad9d640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15f0f0 0x7f489c15d010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f489400efc0 tx=0x7f489400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.503+0000 7f4892ffd640 1 -- 172.21.15.27:0/1638045262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894018020 con 0x7f489c15f0f0 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.504+0000 7f4892ffd640 1 -- 172.21.15.27:0/1638045262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4894014480 con 0x7f489c15f0f0 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.504+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c1e6680 con 0x7f489c15f0f0 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.504+0000 7f4892ffd640 1 -- 172.21.15.27:0/1638045262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894014870 con 0x7f489c15f0f0 2024-04-20T10:56:09.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.504+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c1e6c40 con 0x7f489c15f0f0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.505+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c04fcf0 con 0x7f489c15f0f0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.505+0000 7f4892ffd640 1 -- 172.21.15.27:0/1638045262 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4894005d10 con 0x7f489c15f0f0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.505+0000 7f4892ffd640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f487c06c920 0x7f487c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.506+0000 7f4892ffd640 1 -- 172.21.15.27:0/1638045262 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f489408c550 con 0x7f489c15f0f0 2024-04-20T10:56:09.506 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.506+0000 7f489a59c640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f487c06c920 0x7f487c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.506+0000 7f489a59c640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f487c06c920 0x7f487c06ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f488c036000 tx=0x7f488c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.510+0000 7f4892ffd640 1 -- 172.21.15.27:0/1638045262 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4894022070 con 0x7f489c15f0f0 2024-04-20T10:56:09.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.735+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f489c072b60 con 0x7f489c15f0f0 2024-04-20T10:56:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.736+0000 7f4892ffd640 1 -- 172.21.15.27:0/1638045262 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6927 (secure 0 0 0) 0x7f4894010040 con 0x7f489c15f0f0 2024-04-20T10:56:09.736 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:09.736 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:05.863854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:56:09.736 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2024-04-20T10:56:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f487c06c920 msgr2=0x7f487c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f487c06c920 0x7f487c06ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f488c036000 tx=0x7f488c008030 comp rx=0 tx=0).stop 2024-04-20T10:56:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15f0f0 msgr2=0x7f489c15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15f0f0 0x7f489c15d010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f489400efc0 tx=0x7f489400c490 comp rx=0 tx=0).stop 2024-04-20T10:56:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 shutdown_connections 2024-04-20T10:56:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f487c06c920 0x7f487c06ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f489c15f0f0 0x7f489c15d010 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f489c15ed10 0x7f489c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 --2- 172.21.15.27:0/1638045262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f489c072130 0x7f489c1e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 >> 172.21.15.27:0/1638045262 conn(0x7f489c0ca8f0 msgr2=0x7f489c14eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 shutdown_connections 2024-04-20T10:56:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.738+0000 7f48a0c9a640 1 -- 172.21.15.27:0/1638045262 wait complete. 2024-04-20T10:56:09.749 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:56:09.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.849+0000 7f55ea7b6640 1 Processor -- start 2024-04-20T10:56:09.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.849+0000 7f55ea7b6640 1 -- start start 2024-04-20T10:56:09.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.850+0000 7f55ea7b6640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f55e4155690 0x7f55e415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.850+0000 7f55ea7b6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e40cf5f0 con 0x7f55e4070c00 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.850+0000 7f55ea7b6640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e40cf8d0 con 0x7f55e40cbe80 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.850+0000 7f55ea7b6640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f55e40ca390 con 0x7f55e4155690 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.850+0000 7f55e8d53640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f55e4070c00 0x7f55e4165d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40862/0 (socket says 172.21.15.27:40862) 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.850+0000 7f55e8d53640 1 -- 172.21.15.27:0/3327497094 learned_addr learned my addr 172.21.15.27:0/3327497094 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.850+0000 7f55e3fff640 1 --2- 172.21.15.27:0/3327497094 >> v2:172.21.15.137:3300/0 conn(0x7f55e4155690 0x7f55e415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.851+0000 7f55e2ffd640 1 -- 172.21.15.27:0/3327497094 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487444871 0 0) 0x7f55e40cf5f0 con 0x7f55e4070c00 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.851+0000 7f55e2ffd640 1 -- 172.21.15.27:0/3327497094 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55cc003520 con 0x7f55e4070c00 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.851+0000 7f55e2ffd640 1 -- 172.21.15.27:0/3327497094 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1782762924 0 0) 0x7f55e40cf8d0 con 0x7f55e40cbe80 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.851+0000 7f55e2ffd640 1 -- 172.21.15.27:0/3327497094 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55e40cf5f0 con 0x7f55e40cbe80 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.851+0000 7f55e3fff640 1 -- 172.21.15.27:0/3327497094 >> v1:172.21.15.137:6789/0 conn(0x7f55e40cbe80 legacy=0x7f55e40ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.851+0000 7f55e3fff640 1 -- 172.21.15.27:0/3327497094 >> v1:172.21.15.27:6789/0 conn(0x7f55e4070c00 legacy=0x7f55e4165d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.851+0000 7f55e3fff640 1 -- 172.21.15.27:0/3327497094 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e406ca40 con 0x7f55e4155690 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55e3fff640 1 --2- 172.21.15.27:0/3327497094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e4155690 0x7f55e415b3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55d40097a0 tx=0x7f55d402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1f97ac9fc335d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55e2ffd640 1 -- 172.21.15.27:0/3327497094 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d402f810 con 0x7f55e4155690 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55e2ffd640 1 -- 172.21.15.27:0/3327497094 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d402f9b0 con 0x7f55e4155690 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55e2ffd640 1 -- 172.21.15.27:0/3327497094 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d402fd60 con 0x7f55e4155690 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55ea7b6640 1 -- 172.21.15.27:0/3327497094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e4155690 msgr2=0x7f55e415b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55ea7b6640 1 --2- 172.21.15.27:0/3327497094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e4155690 0x7f55e415b3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55d40097a0 tx=0x7f55d402b9f0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55ea7b6640 1 -- 172.21.15.27:0/3327497094 shutdown_connections 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55ea7b6640 1 --2- 172.21.15.27:0/3327497094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e4155690 0x7f55e415b3f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55ea7b6640 1 -- 172.21.15.27:0/3327497094 >> 172.21.15.27:0/3327497094 conn(0x7f55e406c070 msgr2=0x7f55e40676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55ea7b6640 1 -- 172.21.15.27:0/3327497094 shutdown_connections 2024-04-20T10:56:09.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.852+0000 7f55ea7b6640 1 -- 172.21.15.27:0/3327497094 wait complete. 2024-04-20T10:56:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.853+0000 7f55ea7b6640 1 Processor -- start 2024-04-20T10:56:09.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.854+0000 7f55ea7b6640 1 -- start start 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.854+0000 7f55ea7b6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55e4070c00 0x7f55e414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.854+0000 7f55ea7b6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55e40cbe80 0x7f55e4073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.854+0000 7f55ea7b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e4155690 0x7f55e41531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.854+0000 7f55ea7b6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4155a70 con 0x7f55e40cbe80 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.854+0000 7f55ea7b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55e406d000 con 0x7f55e4155690 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.854+0000 7f55ea7b6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f55e40cf250 con 0x7f55e4070c00 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.854+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55e40cbe80 0x7f55e4073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55e40cbe80 0x7f55e4073260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59542/0 (socket says 172.21.15.27:59542) 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e37fe640 1 -- 172.21.15.27:0/1032567974 learned_addr learned my addr 172.21.15.27:0/1032567974 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e3fff640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55e4070c00 0x7f55e414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e8d53640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e4155690 0x7f55e41531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e37fe640 1 -- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55e4070c00 msgr2=0x7f55e414b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e37fe640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55e4070c00 0x7f55e414b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e37fe640 1 -- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e4155690 msgr2=0x7f55e41531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e37fe640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e4155690 0x7f55e41531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e37fe640 1 -- 172.21.15.27:0/1032567974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e4149b10 con 0x7f55e40cbe80 2024-04-20T10:56:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e37fe640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55e40cbe80 0x7f55e4073260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f55d800c530 tx=0x7f55d800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e0ff9640 1 -- 172.21.15.27:0/1032567974 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d80052d0 con 0x7f55e40cbe80 2024-04-20T10:56:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e0ff9640 1 -- 172.21.15.27:0/1032567974 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d8005490 con 0x7f55e40cbe80 2024-04-20T10:56:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e4149db0 con 0x7f55e40cbe80 2024-04-20T10:56:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55e0ff9640 1 -- 172.21.15.27:0/1032567974 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8012870 con 0x7f55e40cbe80 2024-04-20T10:56:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.855+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e414a330 con 0x7f55e40cbe80 2024-04-20T10:56:09.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.856+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e404fcf0 con 0x7f55e40cbe80 2024-04-20T10:56:09.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.857+0000 7f55e0ff9640 1 -- 172.21.15.27:0/1032567974 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55d8012a10 con 0x7f55e40cbe80 2024-04-20T10:56:09.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.857+0000 7f55e0ff9640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f55d006c8f0 0x7f55d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:09.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.857+0000 7f55e0ff9640 1 -- 172.21.15.27:0/1032567974 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f55d808b5d0 con 0x7f55e40cbe80 2024-04-20T10:56:09.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.857+0000 7f55e3fff640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f55d006c8f0 0x7f55d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:09.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.858+0000 7f55e3fff640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f55d006c8f0 0x7f55d006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f55d4036000 tx=0x7f55d4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:09.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:09.861+0000 7f55e0ff9640 1 -- 172.21.15.27:0/1032567974 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f55d80593c0 con 0x7f55e40cbe80 2024-04-20T10:56:10.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.047+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55e40c5b00 con 0x7f55e40cbe80 2024-04-20T10:56:10.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.048+0000 7f55e0ff9640 1 -- 172.21.15.27:0/1032567974 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16569 (secure 0 0 0) 0x7f55d805d6c0 con 0x7f55e40cbe80 2024-04-20T10:56:10.048 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:10.048 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:43.787038+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:56:10.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f55d006c8f0 msgr2=0x7f55d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f55d006c8f0 0x7f55d006edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f55d4036000 tx=0x7f55d4008030 comp rx=0 tx=0).stop 2024-04-20T10:56:10.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55e40cbe80 msgr2=0x7f55e4073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55e40cbe80 0x7f55e4073260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f55d800c530 tx=0x7f55d800c9f0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 shutdown_connections 2024-04-20T10:56:10.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f55d006c8f0 0x7f55d006edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55e4155690 0x7f55e41531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55e40cbe80 0x7f55e4073260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 --2- 172.21.15.27:0/1032567974 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55e4070c00 0x7f55e414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 >> 172.21.15.27:0/1032567974 conn(0x7f55e406c070 msgr2=0x7f55e40c7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:10.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 shutdown_connections 2024-04-20T10:56:10.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.050+0000 7f55ea7b6640 1 -- 172.21.15.27:0/1032567974 wait complete. 2024-04-20T10:56:10.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:55:29.717620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:55:39.832461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:55:41.103975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:56:10.061 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:56:10.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.165+0000 7f104c987640 1 Processor -- start 2024-04-20T10:56:10.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.165+0000 7f104c987640 1 -- start start 2024-04-20T10:56:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104c987640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f104815ee50 0x7f104815c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104c987640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f10480ccbd0 0x7f1048155410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104c987640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f104815f9a0 con 0x7f104815f230 2024-04-20T10:56:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104c987640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f10480cb870 con 0x7f104815ee50 2024-04-20T10:56:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104c987640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f10480cbb30 con 0x7f10480ccbd0 2024-04-20T10:56:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f104815ee50 0x7f104815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f104815ee50 0x7f104815c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59544/0 (socket says 172.21.15.27:59544) 2024-04-20T10:56:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104659c640 1 -- 172.21.15.27:0/200769127 learned_addr learned my addr 172.21.15.27:0/200769127 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f1046d9d640 1 --2- 172.21.15.27:0/200769127 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f10480ccbd0 0x7f1048155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104659c640 1 -- 172.21.15.27:0/200769127 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f10480ccbd0 msgr2=0x7f1048155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104659c640 1 --2- 172.21.15.27:0/200769127 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f10480ccbd0 0x7f1048155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104659c640 1 -- 172.21.15.27:0/200769127 >> v1:172.21.15.137:6789/0 conn(0x7f104815f230 legacy=0x7f104806cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.166+0000 7f104659c640 1 -- 172.21.15.27:0/200769127 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10480cbe10 con 0x7f104815ee50 2024-04-20T10:56:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.167+0000 7f104659c640 1 --2- 172.21.15.27:0/200769127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 0x7f104815c180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1038009870 tx=0x7f103802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10c882c1751aa1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.167+0000 7f1045d9b640 1 -- 172.21.15.27:0/200769127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103802f840 con 0x7f104815ee50 2024-04-20T10:56:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.167+0000 7f1045d9b640 1 -- 172.21.15.27:0/200769127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f103802f9e0 con 0x7f104815ee50 2024-04-20T10:56:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.167+0000 7f1045d9b640 1 -- 172.21.15.27:0/200769127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103802fdb0 con 0x7f104815ee50 2024-04-20T10:56:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.168+0000 7f104c987640 1 -- 172.21.15.27:0/200769127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 msgr2=0x7f104815c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.168+0000 7f104c987640 1 --2- 172.21.15.27:0/200769127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 0x7f104815c180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1038009870 tx=0x7f103802ba20 comp rx=0 tx=0).stop 2024-04-20T10:56:10.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.168+0000 7f104c987640 1 -- 172.21.15.27:0/200769127 shutdown_connections 2024-04-20T10:56:10.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.168+0000 7f104c987640 1 --2- 172.21.15.27:0/200769127 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f10480ccbd0 0x7f1048155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.168+0000 7f104c987640 1 --2- 172.21.15.27:0/200769127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 0x7f104815c180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.168+0000 7f104c987640 1 -- 172.21.15.27:0/200769127 >> 172.21.15.27:0/200769127 conn(0x7f104806e010 msgr2=0x7f10480d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:10.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.168+0000 7f104c987640 1 -- 172.21.15.27:0/200769127 shutdown_connections 2024-04-20T10:56:10.169 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.168+0000 7f104c987640 1 -- 172.21.15.27:0/200769127 wait complete. 2024-04-20T10:56:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.169+0000 7f104c987640 1 Processor -- start 2024-04-20T10:56:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.169+0000 7f104c987640 1 -- start start 2024-04-20T10:56:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f104c987640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10480ccbd0 0x7f10480c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f104c987640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 0x7f104815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f104c987640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f104815f230 0x7f10481f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f104c987640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f10480cc6b0 con 0x7f104815ee50 2024-04-20T10:56:10.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f104c987640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10480cc3d0 con 0x7f10480ccbd0 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f104c987640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f104815cae0 con 0x7f104815f230 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 0x7f104815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 0x7f104815b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59556/0 (socket says 172.21.15.27:59556) 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103ffff640 1 -- 172.21.15.27:0/200151166 learned_addr learned my addr 172.21.15.27:0/200151166 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f104659c640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10480ccbd0 0x7f10480c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f1046d9d640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f104815f230 0x7f10481f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103ffff640 1 -- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f104815f230 msgr2=0x7f10481f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103ffff640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f104815f230 0x7f10481f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103ffff640 1 -- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10480ccbd0 msgr2=0x7f10480c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103ffff640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10480ccbd0 0x7f10480c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103ffff640 1 -- 172.21.15.27:0/200151166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10481f15c0 con 0x7f104815ee50 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103ffff640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 0x7f104815b170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f103000c430 tx=0x7f103000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103effd640 1 -- 172.21.15.27:0/200151166 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10300052d0 con 0x7f104815ee50 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103effd640 1 -- 172.21.15.27:0/200151166 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1030005490 con 0x7f104815ee50 2024-04-20T10:56:10.172 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f103effd640 1 -- 172.21.15.27:0/200151166 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030007260 con 0x7f104815ee50 2024-04-20T10:56:10.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10481f18a0 con 0x7f104815ee50 2024-04-20T10:56:10.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.170+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10481f1e60 con 0x7f104815ee50 2024-04-20T10:56:10.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.171+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10480688e0 con 0x7f104815ee50 2024-04-20T10:56:10.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.172+0000 7f103effd640 1 -- 172.21.15.27:0/200151166 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1030007400 con 0x7f104815ee50 2024-04-20T10:56:10.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.172+0000 7f103effd640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f102806c920 0x7f102806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.172+0000 7f103effd640 1 -- 172.21.15.27:0/200151166 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f103008afc0 con 0x7f104815ee50 2024-04-20T10:56:10.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.172+0000 7f104659c640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f102806c920 0x7f102806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.173+0000 7f104659c640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f102806c920 0x7f102806ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1038036000 tx=0x7f1038008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:10.176 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.176+0000 7f103effd640 1 -- 172.21.15.27:0/200151166 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f104815f9a0 con 0x7f104815ee50 2024-04-20T10:56:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.359+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1048077070 con 0x7f104815ee50 2024-04-20T10:56:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.359+0000 7f103effd640 1 -- 172.21.15.27:0/200151166 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1048077070 con 0x7f104815ee50 2024-04-20T10:56:10.359 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.361+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f102806c920 msgr2=0x7f102806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.361+0000 7f104c987640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f102806c920 0x7f102806ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1038036000 tx=0x7f1038008030 comp rx=0 tx=0).stop 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 msgr2=0x7f104815b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 0x7f104815b170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f103000c430 tx=0x7f103000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 shutdown_connections 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f102806c920 0x7f102806ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f104815f230 0x7f10481f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f104815ee50 0x7f104815b170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 --2- 172.21.15.27:0/200151166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10480ccbd0 0x7f10480c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 >> 172.21.15.27:0/200151166 conn(0x7f104806e010 msgr2=0x7f10480cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 shutdown_connections 2024-04-20T10:56:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.362+0000 7f104c987640 1 -- 172.21.15.27:0/200151166 wait complete. 2024-04-20T10:56:10.372 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:56:10.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.476+0000 7f8d99184640 1 Processor -- start 2024-04-20T10:56:10.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.476+0000 7f8d99184640 1 -- start start 2024-04-20T10:56:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d99184640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d9415ee50 0x7f8d9415c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d99184640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d9415f9a0 con 0x7f8d9415f230 2024-04-20T10:56:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d99184640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d940cb870 con 0x7f8d940ccbd0 2024-04-20T10:56:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d99184640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8d940cbb30 con 0x7f8d9415ee50 2024-04-20T10:56:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d92d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d9415ee50 0x7f8d9415c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d92d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d9415ee50 0x7f8d9415c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59558/0 (socket says 172.21.15.27:59558) 2024-04-20T10:56:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d92d9d640 1 -- 172.21.15.27:0/185035321 learned_addr learned my addr 172.21.15.27:0/185035321 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d92d9d640 1 -- 172.21.15.27:0/185035321 >> v1:172.21.15.137:6789/0 conn(0x7f8d940ccbd0 legacy=0x7f8d94155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d92d9d640 1 -- 172.21.15.27:0/185035321 >> v1:172.21.15.27:6789/0 conn(0x7f8d9415f230 legacy=0x7f8d9406cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.477+0000 7f8d92d9d640 1 -- 172.21.15.27:0/185035321 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d940cbe10 con 0x7f8d9415ee50 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.478+0000 7f8d92d9d640 1 --2- 172.21.15.27:0/185035321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415ee50 0x7f8d9415c180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009870 tx=0x7f8d8402bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdde43889ed27725 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.478+0000 7f8d9259c640 1 -- 172.21.15.27:0/185035321 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8402d870 con 0x7f8d9415ee50 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.478+0000 7f8d9259c640 1 -- 172.21.15.27:0/185035321 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d8402da10 con 0x7f8d9415ee50 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.478+0000 7f8d9259c640 1 -- 172.21.15.27:0/185035321 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8402dde0 con 0x7f8d9415ee50 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.478+0000 7f8d99184640 1 -- 172.21.15.27:0/185035321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415ee50 msgr2=0x7f8d9415c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.478+0000 7f8d99184640 1 --2- 172.21.15.27:0/185035321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415ee50 0x7f8d9415c180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009870 tx=0x7f8d8402bab0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.479+0000 7f8d99184640 1 -- 172.21.15.27:0/185035321 shutdown_connections 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.479+0000 7f8d99184640 1 --2- 172.21.15.27:0/185035321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415ee50 0x7f8d9415c180 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.479+0000 7f8d99184640 1 -- 172.21.15.27:0/185035321 >> 172.21.15.27:0/185035321 conn(0x7f8d9406e010 msgr2=0x7f8d940d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.479+0000 7f8d99184640 1 -- 172.21.15.27:0/185035321 shutdown_connections 2024-04-20T10:56:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.479+0000 7f8d99184640 1 -- 172.21.15.27:0/185035321 wait complete. 2024-04-20T10:56:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.480+0000 7f8d99184640 1 Processor -- start 2024-04-20T10:56:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.480+0000 7f8d99184640 1 -- start start 2024-04-20T10:56:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d99184640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d940ccbd0 0x7f8d941f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d99184640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d9415ee50 0x7f8d9414c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d99184640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415f230 0x7f8d94151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d99184640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d940cc6b0 con 0x7f8d9415f230 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d99184640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d940cc3d0 con 0x7f8d940ccbd0 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d99184640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9415cae0 con 0x7f8d9415ee50 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d9359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415f230 0x7f8d94151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d92d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d940ccbd0 0x7f8d941f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d9359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415f230 0x7f8d94151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59564/0 (socket says 172.21.15.27:59564) 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d9359e640 1 -- 172.21.15.27:0/3406040234 learned_addr learned my addr 172.21.15.27:0/3406040234 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d8bfff640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d9415ee50 0x7f8d9414c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d9359e640 1 -- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d9415ee50 msgr2=0x7f8d9414c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d9359e640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d9415ee50 0x7f8d9414c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d9359e640 1 -- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d940ccbd0 msgr2=0x7f8d941f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d9359e640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d940ccbd0 0x7f8d941f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d9359e640 1 -- 172.21.15.27:0/3406040234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d941f49b0 con 0x7f8d9415f230 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d92d9d640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d940ccbd0 0x7f8d941f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d9359e640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415f230 0x7f8d94151970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c6b0 tx=0x7f8d8c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d8b7fe640 1 -- 172.21.15.27:0/3406040234 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c003b70 con 0x7f8d9415f230 2024-04-20T10:56:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d8b7fe640 1 -- 172.21.15.27:0/3406040234 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d8c003d30 con 0x7f8d9415f230 2024-04-20T10:56:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d940ca810 con 0x7f8d9415f230 2024-04-20T10:56:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d8b7fe640 1 -- 172.21.15.27:0/3406040234 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c005310 con 0x7f8d9415f230 2024-04-20T10:56:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.481+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d941f1060 con 0x7f8d9415f230 2024-04-20T10:56:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.482+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d940688e0 con 0x7f8d9415f230 2024-04-20T10:56:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.483+0000 7f8d8b7fe640 1 -- 172.21.15.27:0/3406040234 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d8c0054b0 con 0x7f8d9415f230 2024-04-20T10:56:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.483+0000 7f8d8b7fe640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d7406c8f0 0x7f8d7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.483+0000 7f8d8b7fe640 1 -- 172.21.15.27:0/3406040234 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8d8c08b8e0 con 0x7f8d9415f230 2024-04-20T10:56:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.483+0000 7f8d92d9d640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d7406c8f0 0x7f8d7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.484+0000 7f8d92d9d640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d7406c8f0 0x7f8d7406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8d84036000 tx=0x7f8d84008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:10.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.488+0000 7f8d8b7fe640 1 -- 172.21.15.27:0/3406040234 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8d8c059630 con 0x7f8d9415f230 2024-04-20T10:56:10.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.672+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8d9404fcf0 con 0x7f8d9415f230 2024-04-20T10:56:10.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.673+0000 7f8d8b7fe640 1 -- 172.21.15.27:0/3406040234 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f8d8c05d930 con 0x7f8d9415f230 2024-04-20T10:56:10.673 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d7406c8f0 msgr2=0x7f8d7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d7406c8f0 0x7f8d7406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8d84036000 tx=0x7f8d84008030 comp rx=0 tx=0).stop 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415f230 msgr2=0x7f8d94151970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415f230 0x7f8d94151970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00c6b0 tx=0x7f8d8c00cb70 comp rx=0 tx=0).stop 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 shutdown_connections 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d7406c8f0 0x7f8d7406edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d9415f230 0x7f8d94151970 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d9415ee50 0x7f8d9414c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 --2- 172.21.15.27:0/3406040234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d940ccbd0 0x7f8d941f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 >> 172.21.15.27:0/3406040234 conn(0x7f8d9406e010 msgr2=0x7f8d940cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 shutdown_connections 2024-04-20T10:56:10.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.675+0000 7f8d99184640 1 -- 172.21.15.27:0/3406040234 wait complete. 2024-04-20T10:56:10.685 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:56:10.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.789+0000 7f31caf52640 1 Processor -- start 2024-04-20T10:56:10.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.789+0000 7f31caf52640 1 -- start start 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.789+0000 7f31caf52640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f31c4155800 0x7f31c415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.789+0000 7f31caf52640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c40d1260 0x7f31c40d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.789+0000 7f31caf52640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c415c4d0 con 0x7f31c40cbf50 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.789+0000 7f31caf52640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f31c40ce610 con 0x7f31c4155800 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.789+0000 7f31caf52640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f31c40ce180 con 0x7f31c40d1260 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.790+0000 7f31c8cee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f31c4155800 0x7f31c415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.790+0000 7f31c8cee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f31c4155800 0x7f31c415bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59570/0 (socket says 172.21.15.27:59570) 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.790+0000 7f31c8cee640 1 -- 172.21.15.27:0/2054747243 learned_addr learned my addr 172.21.15.27:0/2054747243 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.790+0000 7f31c94ef640 1 --2- 172.21.15.27:0/2054747243 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c40d1260 0x7f31c40d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.790+0000 7f31c8cee640 1 -- 172.21.15.27:0/2054747243 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c40d1260 msgr2=0x7f31c40d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.790+0000 7f31c8cee640 1 --2- 172.21.15.27:0/2054747243 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c40d1260 0x7f31c40d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.790+0000 7f31c8cee640 1 -- 172.21.15.27:0/2054747243 >> v1:172.21.15.137:6789/0 conn(0x7f31c40cbf50 legacy=0x7f31c40c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.790+0000 7f31c8cee640 1 -- 172.21.15.27:0/2054747243 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c40cd290 con 0x7f31c4155800 2024-04-20T10:56:10.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31c8cee640 1 --2- 172.21.15.27:0/2054747243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c4155800 0x7f31c415bf90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009870 tx=0x7f31b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141369fc3cd92ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31c37fe640 1 -- 172.21.15.27:0/2054747243 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8035020 con 0x7f31c4155800 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31c37fe640 1 -- 172.21.15.27:0/2054747243 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b8031640 con 0x7f31c4155800 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31c37fe640 1 -- 172.21.15.27:0/2054747243 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b802d040 con 0x7f31c4155800 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31caf52640 1 -- 172.21.15.27:0/2054747243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c4155800 msgr2=0x7f31c415bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31caf52640 1 --2- 172.21.15.27:0/2054747243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c4155800 0x7f31c415bf90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009870 tx=0x7f31b802b8f0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31caf52640 1 -- 172.21.15.27:0/2054747243 shutdown_connections 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31caf52640 1 --2- 172.21.15.27:0/2054747243 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c40d1260 0x7f31c40d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31caf52640 1 --2- 172.21.15.27:0/2054747243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c4155800 0x7f31c415bf90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31caf52640 1 -- 172.21.15.27:0/2054747243 >> 172.21.15.27:0/2054747243 conn(0x7f31c4067ae0 msgr2=0x7f31c4067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31caf52640 1 -- 172.21.15.27:0/2054747243 shutdown_connections 2024-04-20T10:56:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.791+0000 7f31caf52640 1 -- 172.21.15.27:0/2054747243 wait complete. 2024-04-20T10:56:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.793+0000 7f31caf52640 1 Processor -- start 2024-04-20T10:56:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.793+0000 7f31caf52640 1 -- start start 2024-04-20T10:56:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.793+0000 7f31caf52640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31c40cbf50 0x7f31c4150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.793+0000 7f31caf52640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c40d1260 0x7f31c40b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.793+0000 7f31caf52640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c4155800 0x7f31c40b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31caf52640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f31c40cdb30 con 0x7f31c40d1260 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31caf52640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31c40cd850 con 0x7f31c40cbf50 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31caf52640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f31c40cde10 con 0x7f31c4155800 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c40d1260 0x7f31c40b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c8cee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31c40cbf50 0x7f31c4150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c40d1260 0x7f31c40b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59578/0 (socket says 172.21.15.27:59578) 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c8cee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31c40cbf50 0x7f31c4150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:52986/0 (socket says 172.21.15.27:52986) 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c94ef640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c4155800 0x7f31c40b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c3fff640 1 -- 172.21.15.27:0/2191116251 learned_addr learned my addr 172.21.15.27:0/2191116251 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c3fff640 1 -- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c4155800 msgr2=0x7f31c40b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c3fff640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c4155800 0x7f31c40b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c3fff640 1 -- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31c40cbf50 msgr2=0x7f31c4150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c3fff640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31c40cbf50 0x7f31c4150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c3fff640 1 -- 172.21.15.27:0/2191116251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c40b3c40 con 0x7f31c40d1260 2024-04-20T10:56:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c94ef640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c4155800 0x7f31c40b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c3fff640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c40d1260 0x7f31c40b4b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f31b000c430 tx=0x7f31b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c17fa640 1 -- 172.21.15.27:0/2191116251 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b0017070 con 0x7f31c40d1260 2024-04-20T10:56:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c17fa640 1 -- 172.21.15.27:0/2191116251 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b00052d0 con 0x7f31c40d1260 2024-04-20T10:56:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31c17fa640 1 -- 172.21.15.27:0/2191116251 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b0003830 con 0x7f31c40d1260 2024-04-20T10:56:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c40b6b40 con 0x7f31c40d1260 2024-04-20T10:56:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.794+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c40b3eb0 con 0x7f31c40d1260 2024-04-20T10:56:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.795+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c40688e0 con 0x7f31c40d1260 2024-04-20T10:56:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.796+0000 7f31c17fa640 1 -- 172.21.15.27:0/2191116251 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31b001a070 con 0x7f31c40d1260 2024-04-20T10:56:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.796+0000 7f31c17fa640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f31a806c8f0 0x7f31a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.796+0000 7f31c17fa640 1 -- 172.21.15.27:0/2191116251 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f31b008ada0 con 0x7f31c40d1260 2024-04-20T10:56:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.796+0000 7f31c8cee640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f31a806c8f0 0x7f31a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.797+0000 7f31c8cee640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f31a806c8f0 0x7f31a806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31b8036000 tx=0x7f31b803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:10.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.801+0000 7f31c17fa640 1 -- 172.21.15.27:0/2191116251 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f31c415c4d0 con 0x7f31c40d1260 2024-04-20T10:56:10.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.990+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f31c415bcd0 con 0x7f31c40d1260 2024-04-20T10:56:10.990 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.990+0000 7f31c17fa640 1 -- 172.21.15.27:0/2191116251 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f31c415bcd0 con 0x7f31c40d1260 2024-04-20T10:56:10.990 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-20T10:56:10.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f31a806c8f0 msgr2=0x7f31a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f31a806c8f0 0x7f31a806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f31b8036000 tx=0x7f31b803c040 comp rx=0 tx=0).stop 2024-04-20T10:56:10.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c40d1260 msgr2=0x7f31c40b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:10.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c40d1260 0x7f31c40b4b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f31b000c430 tx=0x7f31b000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 shutdown_connections 2024-04-20T10:56:10.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f31a806c8f0 0x7f31a806edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f31c4155800 0x7f31c40b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31c40d1260 0x7f31c40b4b70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 --2- 172.21.15.27:0/2191116251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31c40cbf50 0x7f31c4150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:10.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 >> 172.21.15.27:0/2191116251 conn(0x7f31c4067ae0 msgr2=0x7f31c40ca900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:10.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 shutdown_connections 2024-04-20T10:56:10.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:10.992+0000 7f31caf52640 1 -- 172.21.15.27:0/2191116251 wait complete. 2024-04-20T10:56:11.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:56:11.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.103+0000 7f7f75e6d640 1 Processor -- start 2024-04-20T10:56:11.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.103+0000 7f7f75e6d640 1 -- start start 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f75e6d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f7015ee50 0x7f7f7015c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f75e6d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f7015f230 0x7f7f7006cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f75e6d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f7015f9a0 con 0x7f7f700ccbd0 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f75e6d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7f700cb870 con 0x7f7f7015f230 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f75e6d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7006d490 con 0x7f7f7015ee50 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f6effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f7015f230 0x7f7f7006cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f6effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7f7015f230 0x7f7f7006cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59594/0 (socket says 172.21.15.27:59594) 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f6effd640 1 -- 172.21.15.27:0/4170599087 learned_addr learned my addr 172.21.15.27:0/4170599087 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f6f7fe640 1 --2- 172.21.15.27:0/4170599087 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f7015ee50 0x7f7f7015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f6effd640 1 -- 172.21.15.27:0/4170599087 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f7015ee50 msgr2=0x7f7f7015c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:11.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f6effd640 1 --2- 172.21.15.27:0/4170599087 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f7015ee50 0x7f7f7015c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f6effd640 1 -- 172.21.15.27:0/4170599087 >> v1:172.21.15.27:6789/0 conn(0x7f7f700ccbd0 legacy=0x7f7f70155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.104+0000 7f7f6effd640 1 -- 172.21.15.27:0/4170599087 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f700cbb30 con 0x7f7f7015f230 2024-04-20T10:56:11.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.105+0000 7f7f6effd640 1 --2- 172.21.15.27:0/4170599087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 0x7f7f7006cf50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009870 tx=0x7f7f6402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1689ffbea1c2c156 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.105+0000 7f7f6e7fc640 1 -- 172.21.15.27:0/4170599087 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6402f840 con 0x7f7f7015f230 2024-04-20T10:56:11.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.105+0000 7f7f6e7fc640 1 -- 172.21.15.27:0/4170599087 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f6402f9e0 con 0x7f7f7015f230 2024-04-20T10:56:11.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.105+0000 7f7f6e7fc640 1 -- 172.21.15.27:0/4170599087 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6402fdb0 con 0x7f7f7015f230 2024-04-20T10:56:11.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.106+0000 7f7f75e6d640 1 -- 172.21.15.27:0/4170599087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 msgr2=0x7f7f7006cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.106+0000 7f7f75e6d640 1 --2- 172.21.15.27:0/4170599087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 0x7f7f7006cf50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009870 tx=0x7f7f6402ba20 comp rx=0 tx=0).stop 2024-04-20T10:56:11.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.106+0000 7f7f75e6d640 1 -- 172.21.15.27:0/4170599087 shutdown_connections 2024-04-20T10:56:11.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.106+0000 7f7f75e6d640 1 --2- 172.21.15.27:0/4170599087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 0x7f7f7006cf50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.106+0000 7f7f75e6d640 1 --2- 172.21.15.27:0/4170599087 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f7015ee50 0x7f7f7015c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.106+0000 7f7f75e6d640 1 -- 172.21.15.27:0/4170599087 >> 172.21.15.27:0/4170599087 conn(0x7f7f7006e010 msgr2=0x7f7f700d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:11.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.106+0000 7f7f75e6d640 1 -- 172.21.15.27:0/4170599087 shutdown_connections 2024-04-20T10:56:11.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.106+0000 7f7f75e6d640 1 -- 172.21.15.27:0/4170599087 wait complete. 2024-04-20T10:56:11.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.107+0000 7f7f75e6d640 1 Processor -- start 2024-04-20T10:56:11.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.107+0000 7f7f75e6d640 1 -- start start 2024-04-20T10:56:11.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f75e6d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f700ccbd0 0x7f7f70158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f75e6d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f7015ee50 0x7f7f700c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f75e6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 0x7f7f700cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f75e6d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700cc3d0 con 0x7f7f7015f230 2024-04-20T10:56:11.110 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f75e6d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700cc0f0 con 0x7f7f7015ee50 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f75e6d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7f700cc6b0 con 0x7f7f700ccbd0 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 0x7f7f700cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 0x7f7f700cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59602/0 (socket says 172.21.15.27:59602) 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6ffff640 1 -- 172.21.15.27:0/1743075011 learned_addr learned my addr 172.21.15.27:0/1743075011 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6effd640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f7015ee50 0x7f7f700c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6f7fe640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f700ccbd0 0x7f7f70158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6ffff640 1 -- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f700ccbd0 msgr2=0x7f7f70158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6ffff640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f700ccbd0 0x7f7f70158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6ffff640 1 -- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f7015ee50 msgr2=0x7f7f700c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6ffff640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f7015ee50 0x7f7f700c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6ffff640 1 -- 172.21.15.27:0/1743075011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f70155fb0 con 0x7f7f7015f230 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6f7fe640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f700ccbd0 0x7f7f70158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6effd640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f7015ee50 0x7f7f700c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.108+0000 7f7f6ffff640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 0x7f7f700cf3e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800c6b0 tx=0x7f7f6800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.109+0000 7f7f57fff640 1 -- 172.21.15.27:0/1743075011 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68003b70 con 0x7f7f7015f230 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.109+0000 7f7f57fff640 1 -- 172.21.15.27:0/1743075011 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f68003d30 con 0x7f7f7015f230 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.109+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f70156be0 con 0x7f7f7015f230 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.109+0000 7f7f57fff640 1 -- 172.21.15.27:0/1743075011 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68005310 con 0x7f7f7015f230 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.109+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f70157810 con 0x7f7f7015f230 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.110+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f700688e0 con 0x7f7f7015f230 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.110+0000 7f7f57fff640 1 -- 172.21.15.27:0/1743075011 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f680054b0 con 0x7f7f7015f230 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.111+0000 7f7f57fff640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.111+0000 7f7f57fff640 1 -- 172.21.15.27:0/1743075011 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7f6808ba40 con 0x7f7f7015f230 2024-04-20T10:56:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.111+0000 7f7f6f7fe640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.111+0000 7f7f6f7fe640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009ac0 tx=0x7f7f60008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.115+0000 7f7f57fff640 1 -- 172.21.15.27:0/1743075011 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7f7015f9a0 con 0x7f7f7015f230 2024-04-20T10:56:11.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.343+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f700ce310 con 0x7f7f7015f230 2024-04-20T10:56:11.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.343+0000 7f7f57fff640 1 -- 172.21.15.27:0/1743075011 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6927 (secure 0 0 0) 0x7f7f6808be20 con 0x7f7f7015f230 2024-04-20T10:56:11.343 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:11.343 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:05.863854+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:56:11.343 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 16 2024-04-20T10:56:11.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7f5c06c8f0 msgr2=0x7f7f5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009ac0 tx=0x7f7f60008040 comp rx=0 tx=0).stop 2024-04-20T10:56:11.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 msgr2=0x7f7f700cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 0x7f7f700cf3e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800c6b0 tx=0x7f7f6800cb70 comp rx=0 tx=0).stop 2024-04-20T10:56:11.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 shutdown_connections 2024-04-20T10:56:11.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7f5c06c8f0 0x7f7f5c06edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f7015f230 0x7f7f700cf3e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f7015ee50 0x7f7f700c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 --2- 172.21.15.27:0/1743075011 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7f700ccbd0 0x7f7f70158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 >> 172.21.15.27:0/1743075011 conn(0x7f7f7006e010 msgr2=0x7f7f70154870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:11.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 shutdown_connections 2024-04-20T10:56:11.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.346+0000 7f7f75e6d640 1 -- 172.21.15.27:0/1743075011 wait complete. 2024-04-20T10:56:11.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:56:11.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.456+0000 7f7d54e8c640 1 Processor -- start 2024-04-20T10:56:11.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.456+0000 7f7d54e8c640 1 -- start start 2024-04-20T10:56:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.457+0000 7f7d54e8c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7d50152070 0x7f7d50158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.457+0000 7f7d54e8c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d5006daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.457+0000 7f7d54e8c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d50072280 con 0x7f7d5006ca40 2024-04-20T10:56:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.457+0000 7f7d54e8c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f7d50072b20 con 0x7f7d50152070 2024-04-20T10:56:11.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.457+0000 7f7d54e8c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7d50072560 con 0x7f7d5006d660 2024-04-20T10:56:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.458+0000 7f7d4dd9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d5006daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.458+0000 7f7d4e59c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7d50152070 0x7f7d50158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.458+0000 7f7d4dd9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d5006daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:46650/0 (socket says 172.21.15.27:46650) 2024-04-20T10:56:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.458+0000 7f7d4dd9b640 1 -- 172.21.15.27:0/3860791442 learned_addr learned my addr 172.21.15.27:0/3860791442 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.458+0000 7f7d4dd9b640 1 -- 172.21.15.27:0/3860791442 >> v2:172.21.15.137:3300/0 conn(0x7f7d50152070 msgr2=0x7f7d50158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.458+0000 7f7d4dd9b640 1 --2- 172.21.15.27:0/3860791442 >> v2:172.21.15.137:3300/0 conn(0x7f7d50152070 0x7f7d50158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.458+0000 7f7d4dd9b640 1 -- 172.21.15.27:0/3860791442 >> v1:172.21.15.137:6789/0 conn(0x7f7d5006ca40 legacy=0x7f7d5015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.458+0000 7f7d4dd9b640 1 -- 172.21.15.27:0/3860791442 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d50072840 con 0x7f7d5006d660 2024-04-20T10:56:11.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.459+0000 7f7d4dd9b640 1 --2- 172.21.15.27:0/3860791442 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d5006daa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009870 tx=0x7f7d3802ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e4db371256e51e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.459+0000 7f7d4d59a640 1 -- 172.21.15.27:0/3860791442 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3802f840 con 0x7f7d5006d660 2024-04-20T10:56:11.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.459+0000 7f7d4d59a640 1 -- 172.21.15.27:0/3860791442 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d3802f9e0 con 0x7f7d5006d660 2024-04-20T10:56:11.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.459+0000 7f7d4d59a640 1 -- 172.21.15.27:0/3860791442 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3802fdb0 con 0x7f7d5006d660 2024-04-20T10:56:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.459+0000 7f7d54e8c640 1 -- 172.21.15.27:0/3860791442 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 msgr2=0x7f7d5006daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.459+0000 7f7d54e8c640 1 --2- 172.21.15.27:0/3860791442 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d5006daa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009870 tx=0x7f7d3802ba20 comp rx=0 tx=0).stop 2024-04-20T10:56:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.460+0000 7f7d54e8c640 1 -- 172.21.15.27:0/3860791442 shutdown_connections 2024-04-20T10:56:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.460+0000 7f7d54e8c640 1 --2- 172.21.15.27:0/3860791442 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d5006daa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.460+0000 7f7d54e8c640 1 --2- 172.21.15.27:0/3860791442 >> v2:172.21.15.137:3300/0 conn(0x7f7d50152070 0x7f7d50158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.460+0000 7f7d54e8c640 1 -- 172.21.15.27:0/3860791442 >> 172.21.15.27:0/3860791442 conn(0x7f7d5006f950 msgr2=0x7f7d5006fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.460+0000 7f7d54e8c640 1 -- 172.21.15.27:0/3860791442 shutdown_connections 2024-04-20T10:56:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.460+0000 7f7d54e8c640 1 -- 172.21.15.27:0/3860791442 wait complete. 2024-04-20T10:56:11.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.461+0000 7f7d54e8c640 1 Processor -- start 2024-04-20T10:56:11.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.461+0000 7f7d54e8c640 1 -- start start 2024-04-20T10:56:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d54e8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d5006ca40 0x7f7d50150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d54e8c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d501e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d54e8c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d50152070 0x7f7d501e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d54e8c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50071f60 con 0x7f7d50152070 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d54e8c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50158520 con 0x7f7d5006ca40 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d54e8c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7d50158f80 con 0x7f7d5006d660 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d50152070 0x7f7d501e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d50152070 0x7f7d501e7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59606/0 (socket says 172.21.15.27:59606) 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4ed9d640 1 -- 172.21.15.27:0/1893570408 learned_addr learned my addr 172.21.15.27:0/1893570408 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4dd9b640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d501e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4e59c640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d5006ca40 0x7f7d50150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4ed9d640 1 -- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 msgr2=0x7f7d501e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4ed9d640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d501e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4ed9d640 1 -- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d5006ca40 msgr2=0x7f7d50150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4ed9d640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d5006ca40 0x7f7d50150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4ed9d640 1 -- 172.21.15.27:0/1893570408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501f1090 con 0x7f7d50152070 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4dd9b640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d501e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d4ed9d640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d50152070 0x7f7d501e7df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000c7a0 tx=0x7f7d4000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.462+0000 7f7d46ffd640 1 -- 172.21.15.27:0/1893570408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40003b70 con 0x7f7d50152070 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.463+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d501f5370 con 0x7f7d50152070 2024-04-20T10:56:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.463+0000 7f7d46ffd640 1 -- 172.21.15.27:0/1893570408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d40003d30 con 0x7f7d50152070 2024-04-20T10:56:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.463+0000 7f7d46ffd640 1 -- 172.21.15.27:0/1893570408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000aa90 con 0x7f7d50152070 2024-04-20T10:56:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.463+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d501f1630 con 0x7f7d50152070 2024-04-20T10:56:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.464+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5004fcf0 con 0x7f7d50152070 2024-04-20T10:56:11.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.464+0000 7f7d46ffd640 1 -- 172.21.15.27:0/1893570408 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d4000ac30 con 0x7f7d50152070 2024-04-20T10:56:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.464+0000 7f7d46ffd640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7d3006c8f0 0x7f7d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.465+0000 7f7d46ffd640 1 -- 172.21.15.27:0/1893570408 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d4008ba10 con 0x7f7d50152070 2024-04-20T10:56:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.465+0000 7f7d4e59c640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7d3006c8f0 0x7f7d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.465+0000 7f7d4e59c640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7d3006c8f0 0x7f7d3006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009aa0 tx=0x7f7d48009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.469+0000 7f7d46ffd640 1 -- 172.21.15.27:0/1893570408 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7d400596e0 con 0x7f7d50152070 2024-04-20T10:56:11.602 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:56:11.604 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:56:11.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.655+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d500c8470 con 0x7f7d50152070 2024-04-20T10:56:11.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.656+0000 7f7d46ffd640 1 -- 172.21.15.27:0/1893570408 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16569 (secure 0 0 0) 0x7f7d4005d9e0 con 0x7f7d50152070 2024-04-20T10:56:11.656 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:11.656 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:43.787038+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:56:11.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7d3006c8f0 msgr2=0x7f7d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7d3006c8f0 0x7f7d3006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009aa0 tx=0x7f7d48009190 comp rx=0 tx=0).stop 2024-04-20T10:56:11.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d50152070 msgr2=0x7f7d501e7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d50152070 0x7f7d501e7df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000c7a0 tx=0x7f7d4000cc60 comp rx=0 tx=0).stop 2024-04-20T10:56:11.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 shutdown_connections 2024-04-20T10:56:11.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7d3006c8f0 0x7f7d3006edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d50152070 0x7f7d501e7df0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7d5006d660 0x7f7d501e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 --2- 172.21.15.27:0/1893570408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d5006ca40 0x7f7d50150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 >> 172.21.15.27:0/1893570408 conn(0x7f7d5006f950 msgr2=0x7f7d500c6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:11.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 shutdown_connections 2024-04-20T10:56:11.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.658+0000 7f7d54e8c640 1 -- 172.21.15.27:0/1893570408 wait complete. 2024-04-20T10:56:11.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:56:11.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.810+0000 7f77c6ff4640 1 Processor -- start 2024-04-20T10:56:11.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.810+0000 7f77c6ff4640 1 -- start start 2024-04-20T10:56:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.811+0000 7f77c6ff4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f77c006d660 0x7f77c006dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.811+0000 7f77c6ff4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c006ca40 0x7f77c006ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.811+0000 7f77c6ff4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c0072280 con 0x7f77c0152070 2024-04-20T10:56:11.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.811+0000 7f77c6ff4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f77c0072b20 con 0x7f77c006d660 2024-04-20T10:56:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.811+0000 7f77c6ff4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f77c0072560 con 0x7f77c006ca40 2024-04-20T10:56:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77bffff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f77c006d660 0x7f77c006dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77c5591640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c006ca40 0x7f77c006ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77c5591640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c006ca40 0x7f77c006ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:46668/0 (socket says 172.21.15.27:46668) 2024-04-20T10:56:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77c5591640 1 -- 172.21.15.27:0/4030035540 learned_addr learned my addr 172.21.15.27:0/4030035540 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77bffff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f77c006d660 0x7f77c006dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:53014/0 (socket says 172.21.15.27:53014) 2024-04-20T10:56:11.812 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77c4d90640 1 --1- 172.21.15.27:0/4030035540 >> v1:172.21.15.137:6789/0 conn(0x7f77c0152070 0x7f77c0158a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:56958/0 (socket says 172.21.15.27:56958) 2024-04-20T10:56:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77c5591640 1 -- 172.21.15.27:0/4030035540 >> v2:172.21.15.137:3300/0 conn(0x7f77c006d660 msgr2=0x7f77c006dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77c5591640 1 --2- 172.21.15.27:0/4030035540 >> v2:172.21.15.137:3300/0 conn(0x7f77c006d660 0x7f77c006dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77c5591640 1 -- 172.21.15.27:0/4030035540 >> v1:172.21.15.137:6789/0 conn(0x7f77c0152070 legacy=0x7f77c0158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77c5591640 1 -- 172.21.15.27:0/4030035540 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c0072840 con 0x7f77c006ca40 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.812+0000 7f77bffff640 1 --2- 172.21.15.27:0/4030035540 >> v2:172.21.15.137:3300/0 conn(0x7f77c006d660 0x7f77c006dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.813+0000 7f77c5591640 1 --2- 172.21.15.27:0/4030035540 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c006ca40 0x7f77c006ced0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009e50 tx=0x7f77b802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86381f49043dbb52 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.813+0000 7f77bf7fe640 1 -- 172.21.15.27:0/4030035540 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b802cb90 con 0x7f77c006ca40 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.813+0000 7f77bf7fe640 1 -- 172.21.15.27:0/4030035540 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b802cd30 con 0x7f77c006ca40 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.813+0000 7f77bf7fe640 1 -- 172.21.15.27:0/4030035540 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8034610 con 0x7f77c006ca40 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.814+0000 7f77c6ff4640 1 -- 172.21.15.27:0/4030035540 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c006ca40 msgr2=0x7f77c006ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.814+0000 7f77c6ff4640 1 --2- 172.21.15.27:0/4030035540 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c006ca40 0x7f77c006ced0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009e50 tx=0x7f77b802bff0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.814+0000 7f77c6ff4640 1 -- 172.21.15.27:0/4030035540 shutdown_connections 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.814+0000 7f77c6ff4640 1 --2- 172.21.15.27:0/4030035540 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c006ca40 0x7f77c006ced0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.814+0000 7f77c6ff4640 1 --2- 172.21.15.27:0/4030035540 >> v2:172.21.15.137:3300/0 conn(0x7f77c006d660 0x7f77c006dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.814+0000 7f77c6ff4640 1 -- 172.21.15.27:0/4030035540 >> 172.21.15.27:0/4030035540 conn(0x7f77c006f950 msgr2=0x7f77c006fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.814+0000 7f77c6ff4640 1 -- 172.21.15.27:0/4030035540 shutdown_connections 2024-04-20T10:56:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.814+0000 7f77c6ff4640 1 -- 172.21.15.27:0/4030035540 wait complete. 2024-04-20T10:56:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.815+0000 7f77c6ff4640 1 Processor -- start 2024-04-20T10:56:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.815+0000 7f77c6ff4640 1 -- start start 2024-04-20T10:56:11.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c6ff4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c006ca40 0x7f77c015b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c6ff4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c006d660 0x7f77c01571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c6ff4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c0152070 0x7f77c0157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c6ff4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f77c0071f60 con 0x7f77c006ca40 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c6ff4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77c0158520 con 0x7f77c006d660 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c6ff4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f77c0152450 con 0x7f77c0152070 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c4d90640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c006ca40 0x7f77c015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c4d90640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c006ca40 0x7f77c015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59610/0 (socket says 172.21.15.27:59610) 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c4d90640 1 -- 172.21.15.27:0/2845533697 learned_addr learned my addr 172.21.15.27:0/2845533697 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c5591640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c0152070 0x7f77c0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77bffff640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c006d660 0x7f77c01571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c4d90640 1 -- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c0152070 msgr2=0x7f77c0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c4d90640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c0152070 0x7f77c0157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c4d90640 1 -- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c006d660 msgr2=0x7f77c01571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:11.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c4d90640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c006d660 0x7f77c01571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c4d90640 1 -- 172.21.15.27:0/2845533697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c01f2040 con 0x7f77c006ca40 2024-04-20T10:56:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.816+0000 7f77c4d90640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c006ca40 0x7f77c015b280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77b40069a0 tx=0x7f77b400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.817+0000 7f77bd7fa640 1 -- 172.21.15.27:0/2845533697 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4017070 con 0x7f77c006ca40 2024-04-20T10:56:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.817+0000 7f77bd7fa640 1 -- 172.21.15.27:0/2845533697 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b40064e0 con 0x7f77c006ca40 2024-04-20T10:56:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.817+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c0155820 con 0x7f77c006ca40 2024-04-20T10:56:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.817+0000 7f77bd7fa640 1 -- 172.21.15.27:0/2845533697 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40049e0 con 0x7f77c006ca40 2024-04-20T10:56:11.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.817+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c01f1560 con 0x7f77c006ca40 2024-04-20T10:56:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.818+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c004fcf0 con 0x7f77c006ca40 2024-04-20T10:56:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.818+0000 7f77bd7fa640 1 -- 172.21.15.27:0/2845533697 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77b401a070 con 0x7f77c006ca40 2024-04-20T10:56:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.819+0000 7f77bd7fa640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f77a406c8f0 0x7f77a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.819+0000 7f77bd7fa640 1 -- 172.21.15.27:0/2845533697 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f77b408aee0 con 0x7f77c006ca40 2024-04-20T10:56:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.819+0000 7f77bffff640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f77a406c8f0 0x7f77a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:11.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.819+0000 7f77bffff640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f77a406c8f0 0x7f77a406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009560 tx=0x7f77ac009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:11.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:11.823+0000 7f77bd7fa640 1 -- 172.21.15.27:0/2845533697 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f77b4058c30 con 0x7f77c006ca40 2024-04-20T10:56:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.049+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77c006f270 con 0x7f77c006ca40 2024-04-20T10:56:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.049+0000 7f77bd7fa640 1 -- 172.21.15.27:0/2845533697 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7f77b405cf30 con 0x7f77c006ca40 2024-04-20T10:56:12.049 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:12.049 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:56:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:56:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f77a406c8f0 msgr2=0x7f77a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f77a406c8f0 0x7f77a406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009560 tx=0x7f77ac009190 comp rx=0 tx=0).stop 2024-04-20T10:56:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c006ca40 msgr2=0x7f77c015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c006ca40 0x7f77c015b280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77b40069a0 tx=0x7f77b400c960 comp rx=0 tx=0).stop 2024-04-20T10:56:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 shutdown_connections 2024-04-20T10:56:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f77a406c8f0 0x7f77a406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77c0152070 0x7f77c0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c006d660 0x7f77c01571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 --2- 172.21.15.27:0/2845533697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77c006ca40 0x7f77c015b280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 >> 172.21.15.27:0/2845533697 conn(0x7f77c006f950 msgr2=0x7f77c00c96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 shutdown_connections 2024-04-20T10:56:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.051+0000 7f77c6ff4640 1 -- 172.21.15.27:0/2845533697 wait complete. 2024-04-20T10:56:12.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:55:42.150319+0000', 'modified': '2024-04-20T10:56:11.871426+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4293, 'mds_1': 4278, 'mds_2': 4275, 'mds_3': 4272, 'mds_4': 4262}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4262': {'gid': 4262, 'name': 'b', 'rank': 4, 'incarnation': 16, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.137:6838/1990627502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 1990627502}, {'type': 'v1', 'addr': '172.21.15.137:6838', 'nonce': 1990627502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4272': {'gid': 4272, 'name': 'c', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.27:6835/859496751', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6832', 'nonce': 859496751}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 859496751}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4275': {'gid': 4275, 'name': 'e', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.27:6837/1680990583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 1680990583}, {'type': 'v1', 'addr': '172.21.15.27:6837', 'nonce': 1680990583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4278': {'gid': 4278, 'name': 'a', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.27:6836/4195543922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 4195543922}, {'type': 'v1', 'addr': '172.21.15.27:6836', 'nonce': 4195543922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4293': {'gid': 4293, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.137:6839/2929802235', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6837', 'nonce': 2929802235}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 2929802235}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4293, 'qdb_cluster': [4262, 4275, 4272, 4278, 4293]} 2024-04-20T10:56:12.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-20T10:56:12.666 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T10:56:12.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.775+0000 7ff5736b3640 1 Processor -- start 2024-04-20T10:56:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.775+0000 7ff5736b3640 1 -- start start 2024-04-20T10:56:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff5736b3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff56c151940 0x7ff56c158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff5736b3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 0x7ff56c0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff5736b3640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff56c071b40 0x7ff56c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff5736b3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff56c071560 con 0x7ff56c151940 2024-04-20T10:56:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff5736b3640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7ff56c072140 con 0x7ff56c071b40 2024-04-20T10:56:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff5736b3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff56c072420 con 0x7ff56c151d20 2024-04-20T10:56:12.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff57144f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff56c151940 0x7ff56c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff57144f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff56c151940 0x7ff56c158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59622/0 (socket says 172.21.15.27:59622) 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff57144f640 1 -- 172.21.15.27:0/1484535060 learned_addr learned my addr 172.21.15.27:0/1484535060 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff571c50640 1 --2- 172.21.15.27:0/1484535060 >> v2:172.21.15.137:3300/0 conn(0x7ff56c071b40 0x7ff56c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff570c4e640 1 --2- 172.21.15.27:0/1484535060 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 0x7ff56c0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff57144f640 1 -- 172.21.15.27:0/1484535060 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 msgr2=0x7ff56c0731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff57144f640 1 --2- 172.21.15.27:0/1484535060 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 0x7ff56c0731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff57144f640 1 -- 172.21.15.27:0/1484535060 >> v2:172.21.15.137:3300/0 conn(0x7ff56c071b40 msgr2=0x7ff56c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff57144f640 1 --2- 172.21.15.27:0/1484535060 >> v2:172.21.15.137:3300/0 conn(0x7ff56c071b40 0x7ff56c06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff57144f640 1 -- 172.21.15.27:0/1484535060 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56c072700 con 0x7ff56c151940 2024-04-20T10:56:12.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.776+0000 7ff570c4e640 1 --2- 172.21.15.27:0/1484535060 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 0x7ff56c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.777+0000 7ff57144f640 1 --2- 172.21.15.27:0/1484535060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c151940 0x7ff56c158240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0720d0 tx=0x7ff56802e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46720134b3cbae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.777+0000 7ff563fff640 1 -- 172.21.15.27:0/1484535060 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56803b070 con 0x7ff56c151940 2024-04-20T10:56:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.777+0000 7ff563fff640 1 -- 172.21.15.27:0/1484535060 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff568036440 con 0x7ff56c151940 2024-04-20T10:56:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.777+0000 7ff563fff640 1 -- 172.21.15.27:0/1484535060 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568036810 con 0x7ff56c151940 2024-04-20T10:56:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.778+0000 7ff5736b3640 1 -- 172.21.15.27:0/1484535060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c151940 msgr2=0x7ff56c158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.778+0000 7ff5736b3640 1 --2- 172.21.15.27:0/1484535060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c151940 0x7ff56c158240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0720d0 tx=0x7ff56802e170 comp rx=0 tx=0).stop 2024-04-20T10:56:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.778+0000 7ff5736b3640 1 -- 172.21.15.27:0/1484535060 shutdown_connections 2024-04-20T10:56:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.778+0000 7ff5736b3640 1 --2- 172.21.15.27:0/1484535060 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 0x7ff56c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.778+0000 7ff5736b3640 1 --2- 172.21.15.27:0/1484535060 >> v2:172.21.15.137:3300/0 conn(0x7ff56c071b40 0x7ff56c06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.778+0000 7ff5736b3640 1 --2- 172.21.15.27:0/1484535060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c151940 0x7ff56c158240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.778+0000 7ff5736b3640 1 -- 172.21.15.27:0/1484535060 >> 172.21.15.27:0/1484535060 conn(0x7ff56c0ccbd0 msgr2=0x7ff56c0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.778+0000 7ff5736b3640 1 -- 172.21.15.27:0/1484535060 shutdown_connections 2024-04-20T10:56:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.778+0000 7ff5736b3640 1 -- 172.21.15.27:0/1484535060 wait complete. 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.779+0000 7ff5736b3640 1 Processor -- start 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.779+0000 7ff5736b3640 1 -- start start 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff5736b3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c071b40 0x7ff56c153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff5736b3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff56c151940 0x7ff56c0c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff5736b3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 0x7ff56c13d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff5736b3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c158780 con 0x7ff56c071b40 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff5736b3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c072cc0 con 0x7ff56c151940 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff5736b3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff56c073720 con 0x7ff56c151d20 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff57144f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c071b40 0x7ff56c153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff57144f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c071b40 0x7ff56c153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59630/0 (socket says 172.21.15.27:59630) 2024-04-20T10:56:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff57144f640 1 -- 172.21.15.27:0/2215957742 learned_addr learned my addr 172.21.15.27:0/2215957742 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff570c4e640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff56c151940 0x7ff56c0c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff571c50640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 0x7ff56c13d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff57144f640 1 -- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 msgr2=0x7ff56c13d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff57144f640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 0x7ff56c13d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff57144f640 1 -- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff56c151940 msgr2=0x7ff56c0c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff57144f640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff56c151940 0x7ff56c0c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff57144f640 1 -- 172.21.15.27:0/2215957742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56c14feb0 con 0x7ff56c071b40 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff570c4e640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff56c151940 0x7ff56c0c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff57144f640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c071b40 0x7ff56c153290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff568000c00 tx=0x7ff5680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.780+0000 7ff561ffb640 1 -- 172.21.15.27:0/2215957742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56803b070 con 0x7ff56c071b40 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.781+0000 7ff561ffb640 1 -- 172.21.15.27:0/2215957742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56803d050 con 0x7ff56c071b40 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.781+0000 7ff5736b3640 1 -- 172.21.15.27:0/2215957742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56c1f10a0 con 0x7ff56c071b40 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.781+0000 7ff561ffb640 1 -- 172.21.15.27:0/2215957742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568034a90 con 0x7ff56c071b40 2024-04-20T10:56:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.781+0000 7ff5736b3640 1 -- 172.21.15.27:0/2215957742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56c1f4970 con 0x7ff56c071b40 2024-04-20T10:56:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.782+0000 7ff561ffb640 1 -- 172.21.15.27:0/2215957742 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff568034c30 con 0x7ff56c071b40 2024-04-20T10:56:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.783+0000 7ff561ffb640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff55006c920 0x7ff55006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.783+0000 7ff561ffb640 1 -- 172.21.15.27:0/2215957742 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5680adad0 con 0x7ff56c071b40 2024-04-20T10:56:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.783+0000 7ff570c4e640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff55006c920 0x7ff55006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.783+0000 7ff570c4e640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff55006c920 0x7ff55006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0cbd60 tx=0x7ff55c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.784+0000 7ff57637d740 1 Processor -- start 2024-04-20T10:56:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.784+0000 7ff57637d740 1 -- start start 2024-04-20T10:56:12.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.784+0000 7ff57637d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f8270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.784+0000 7ff57637d740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f825da0 0x55b91f867cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.784+0000 7ff57637d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f8676e0 0x55b91f8665e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.784+0000 7ff57637d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b91f739290 con 0x55b91f8676e0 2024-04-20T10:56:12.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff57637d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b91f796a40 con 0x55b91f867300 2024-04-20T10:56:12.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff57637d740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x55b91f751a30 con 0x55b91f825da0 2024-04-20T10:56:12.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff571c50640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f8676e0 0x55b91f8665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff57144f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f8270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff571c50640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f8676e0 0x55b91f8665e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59634/0 (socket says 172.21.15.27:59634) 2024-04-20T10:56:12.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff570c4e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f825da0 0x55b91f867cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff571c50640 1 -- 172.21.15.27:0/841615479 learned_addr learned my addr 172.21.15.27:0/841615479 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff57144f640 1 -- 172.21.15.27:0/841615479 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f825da0 msgr2=0x55b91f867cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff57144f640 1 --2- 172.21.15.27:0/841615479 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f825da0 0x55b91f867cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff57144f640 1 -- 172.21.15.27:0/841615479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f8676e0 msgr2=0x55b91f8665e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff57144f640 1 --2- 172.21.15.27:0/841615479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f8676e0 0x55b91f8665e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff57144f640 1 -- 172.21.15.27:0/841615479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b91f744c10 con 0x55b91f867300 2024-04-20T10:56:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff571c50640 1 --2- 172.21.15.27:0/841615479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f8676e0 0x55b91f8665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff57144f640 1 --2- 172.21.15.27:0/841615479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f8270a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff5680ac050 tx=0x7ff56807b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34a9f97233cabad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff556ffd640 1 -- 172.21.15.27:0/841615479 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56807fb20 con 0x55b91f867300 2024-04-20T10:56:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff556ffd640 1 -- 172.21.15.27:0/841615479 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5680843e0 con 0x55b91f867300 2024-04-20T10:56:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.785+0000 7ff556ffd640 1 -- 172.21.15.27:0/841615479 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568077de0 con 0x55b91f867300 2024-04-20T10:56:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 -- 172.21.15.27:0/841615479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 msgr2=0x55b91f8270a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:12.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 --2- 172.21.15.27:0/841615479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f8270a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff5680ac050 tx=0x7ff56807b370 comp rx=0 tx=0).stop 2024-04-20T10:56:12.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 -- 172.21.15.27:0/841615479 shutdown_connections 2024-04-20T10:56:12.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 --2- 172.21.15.27:0/841615479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f8676e0 0x55b91f8665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 --2- 172.21.15.27:0/841615479 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f825da0 0x55b91f867cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 --2- 172.21.15.27:0/841615479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f8270a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 -- 172.21.15.27:0/841615479 >> 172.21.15.27:0/841615479 conn(0x55b91f822f10 msgr2=0x55b91f8266b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:12.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 -- 172.21.15.27:0/841615479 shutdown_connections 2024-04-20T10:56:12.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 -- 172.21.15.27:0/841615479 wait complete. 2024-04-20T10:56:12.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.786+0000 7ff57637d740 1 -- start start 2024-04-20T10:56:12.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.787+0000 7ff57637d740 1 Processor -- start 2024-04-20T10:56:12.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.787+0000 7ff57637d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f867300 0x55b91f87a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.787+0000 7ff57637d740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f861ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.787+0000 7ff57637d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f845d90 0x55b91f862220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.787+0000 7ff57637d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b91f790140 con 0x55b91f867300 2024-04-20T10:56:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.787+0000 7ff57637d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b91f78ae20 con 0x55b91f845d90 2024-04-20T10:56:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.787+0000 7ff57637d740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x55b91f798d20 con 0x55b91f8676e0 2024-04-20T10:56:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57637d740 10 client.0 _ll_register_callbacks cb 0x55b91f7815c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57144f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f867300 0x55b91f87a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57144f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f867300 0x55b91f87a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59642/0 (socket says 172.21.15.27:59642) 2024-04-20T10:56:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57144f640 1 -- 172.21.15.27:0/1279143727 learned_addr learned my addr 172.21.15.27:0/1279143727 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff571c50640 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f845d90 0x55b91f862220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff570c4e640 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f861ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57144f640 1 -- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 msgr2=0x55b91f861ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57144f640 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f861ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57144f640 1 -- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f845d90 msgr2=0x55b91f862220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57144f640 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f845d90 0x55b91f862220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57144f640 1 -- 172.21.15.27:0/1279143727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b91f49e2d0 con 0x55b91f867300 2024-04-20T10:56:12.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff571c50640 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f845d90 0x55b91f862220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57144f640 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f867300 0x55b91f87a490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff568045040 tx=0x7ff56800b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e793161734ee43f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff54bfff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff54bfff640 1 -- 172.21.15.27:0/1279143727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56807bbd0 con 0x55b91f867300 2024-04-20T10:56:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff54bfff640 1 -- 172.21.15.27:0/1279143727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff568083f50 con 0x55b91f867300 2024-04-20T10:56:12.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57637d740 1 -- 172.21.15.27:0/1279143727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5680843e0 con 0x55b91f867300 2024-04-20T10:56:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff54bfff640 1 -- 172.21.15.27:0/1279143727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff5680a5050 con 0x55b91f867300 2024-04-20T10:56:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57637d740 10 client.4647 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff57637d740 1 -- 172.21.15.27:0/1279143727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b91f5686b0 con 0x55b91f867300 2024-04-20T10:56:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.788+0000 7ff54bfff640 1 -- 172.21.15.27:0/1279143727 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5680843e0 con 0x55b91f867300 2024-04-20T10:56:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff54bfff640 1 -- 172.21.15.27:0/1279143727 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7ff56800e040 con 0x55b91f867300 2024-04-20T10:56:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff57637d740 10 client.4647 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff57637d740 10 client.4647 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:56:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff57637d740 20 client.4647 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff57637d740 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] conn(0x55b91f86af90 0x55b91f8858e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff57637d740 4 client.4647 mds_command: new command op to 4262 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:56:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff57637d740 1 -- 172.21.15.27:0/1279143727 --> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b91f712510 con 0x55b91f86af90 2024-04-20T10:56:12.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff570c4e640 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] conn(0x55b91f86af90 0x55b91f8858e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:12.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff570c4e640 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] conn(0x55b91f86af90 0x55b91f8858e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=285bbcb29dc5c server_cookie=e261bcb173d39d8 in_seq=0 out_seq=0 2024-04-20T10:56:12.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:12.789+0000 7ff54bfff640 10 client.4647 ms_handle_connect on v2:172.21.15.137:6836/1990627502 2024-04-20T10:56:16.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff54bfff640 1 -- 172.21.15.27:0/1279143727 <== mds.4 v2:172.21.15.137:6836/1990627502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b91f712510 con 0x55b91f86af90 2024-04-20T10:56:16.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff54bfff640 10 client.4647 handle_command_reply: tid=0 2024-04-20T10:56:16.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 client.4647 shutdown 2024-04-20T10:56:16.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 -- 172.21.15.27:0/1279143727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f867300 msgr2=0x55b91f87a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:16.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f867300 0x55b91f87a490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff568045040 tx=0x7ff56800b450 comp rx=0 tx=0).stop 2024-04-20T10:56:16.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 -- 172.21.15.27:0/1279143727 shutdown_connections 2024-04-20T10:56:16.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] conn(0x55b91f86af90 0x55b91f8858e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f845d90 0x55b91f862220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f861ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 --2- 172.21.15.27:0/1279143727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f867300 0x55b91f87a490 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff54bfff640 0 client.4647 ms_handle_reset on v2:172.21.15.137:6836/1990627502 2024-04-20T10:56:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 -- 172.21.15.27:0/1279143727 >> 172.21.15.27:0/1279143727 conn(0x55b91f825da0 msgr2=0x55b91f84b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 -- 172.21.15.27:0/1279143727 shutdown_connections 2024-04-20T10:56:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 1 -- 172.21.15.27:0/1279143727 wait complete. 2024-04-20T10:56:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.882+0000 7ff57637d740 20 client.4647 trim_cache size 0 max 16384 2024-04-20T10:56:16.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.902+0000 7ff57637d740 1 Processor -- start 2024-04-20T10:56:16.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.902+0000 7ff57637d740 1 -- start start 2024-04-20T10:56:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.903+0000 7ff57637d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f861dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.903+0000 7ff57637d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f88ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.903+0000 7ff57637d740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f88b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.903+0000 7ff57637d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c157130 con 0x55b91f845d90 2024-04-20T10:56:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.903+0000 7ff57637d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b91f748980 con 0x55b91f867300 2024-04-20T10:56:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.903+0000 7ff57637d740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x55b91f790140 con 0x55b91f8676e0 2024-04-20T10:56:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.903+0000 7ff57144f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f861dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.903+0000 7ff57144f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f861dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35186/0 (socket says 172.21.15.27:35186) 2024-04-20T10:56:16.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.903+0000 7ff57144f640 1 -- 172.21.15.27:0/896654750 learned_addr learned my addr 172.21.15.27:0/896654750 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff571c50640 1 --2- 172.21.15.27:0/896654750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f88b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff570c4e640 1 --2- 172.21.15.27:0/896654750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f88ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff57144f640 1 -- 172.21.15.27:0/896654750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 msgr2=0x55b91f88b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff57144f640 1 --2- 172.21.15.27:0/896654750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f88b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff57144f640 1 -- 172.21.15.27:0/896654750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 msgr2=0x55b91f88ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff57144f640 1 --2- 172.21.15.27:0/896654750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f88ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff57144f640 1 -- 172.21.15.27:0/896654750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b91f81c1e0 con 0x55b91f845d90 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff57144f640 1 --2- 172.21.15.27:0/896654750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f861dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff56802e1b0 tx=0x7ff5680aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306e9fd89bc6b30f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff5567fc640 1 -- 172.21.15.27:0/896654750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568097070 con 0x55b91f845d90 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff5567fc640 1 -- 172.21.15.27:0/896654750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56809d070 con 0x55b91f845d90 2024-04-20T10:56:16.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.904+0000 7ff5567fc640 1 -- 172.21.15.27:0/896654750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56807b770 con 0x55b91f845d90 2024-04-20T10:56:16.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.905+0000 7ff57637d740 1 -- 172.21.15.27:0/896654750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 msgr2=0x55b91f861dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.905+0000 7ff57637d740 1 --2- 172.21.15.27:0/896654750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f861dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff56802e1b0 tx=0x7ff5680aa000 comp rx=0 tx=0).stop 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.905+0000 7ff57637d740 1 -- 172.21.15.27:0/896654750 shutdown_connections 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.905+0000 7ff57637d740 1 --2- 172.21.15.27:0/896654750 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f88b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.905+0000 7ff57637d740 1 --2- 172.21.15.27:0/896654750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f88ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.905+0000 7ff57637d740 1 --2- 172.21.15.27:0/896654750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f861dc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.905+0000 7ff57637d740 1 -- 172.21.15.27:0/896654750 >> 172.21.15.27:0/896654750 conn(0x55b91f825da0 msgr2=0x55b91f8251e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.905+0000 7ff57637d740 1 -- 172.21.15.27:0/896654750 shutdown_connections 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.905+0000 7ff57637d740 1 -- 172.21.15.27:0/896654750 wait complete. 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.906+0000 7ff57637d740 1 -- start start 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.906+0000 7ff57637d740 1 Processor -- start 2024-04-20T10:56:16.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57637d740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f879600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57637d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f8dae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57637d740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f84e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57637d740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b91f84fc20 con 0x55b91f845d90 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57637d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b91f868cb0 con 0x55b91f867300 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57637d740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x55b91f846360 con 0x55b91f8676e0 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57637d740 10 client.0 _ll_register_callbacks cb 0x7ff564009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57144f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f879600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57144f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f879600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35198/0 (socket says 172.21.15.27:35198) 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57144f640 1 -- 172.21.15.27:0/3158967937 learned_addr learned my addr 172.21.15.27:0/3158967937 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff571c50640 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f84e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff570c4e640 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f8dae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57144f640 1 -- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 msgr2=0x55b91f84e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57144f640 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f84e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57144f640 1 -- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 msgr2=0x55b91f8dae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:16.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57144f640 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f8dae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.907+0000 7ff57144f640 1 -- 172.21.15.27:0/3158967937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b91f8e4a00 con 0x55b91f845d90 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff57144f640 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f879600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff568077bd0 tx=0x7ff568036f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3fe3d670110d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff54bfff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff54bfff640 1 -- 172.21.15.27:0/3158967937 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56804f030 con 0x55b91f845d90 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff54bfff640 1 -- 172.21.15.27:0/3158967937 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff56807b9d0 con 0x55b91f845d90 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff57637d740 1 -- 172.21.15.27:0/3158967937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff56809d070 con 0x55b91f845d90 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff54bfff640 1 -- 172.21.15.27:0/3158967937 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff56807b540 con 0x55b91f845d90 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff57637d740 10 client.4653 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff57637d740 1 -- 172.21.15.27:0/3158967937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b91f8e4a00 con 0x55b91f845d90 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff54bfff640 1 -- 172.21.15.27:0/3158967937 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff56809d070 con 0x55b91f845d90 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff54bfff640 1 -- 172.21.15.27:0/3158967937 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7ff56807abc0 con 0x55b91f845d90 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff57637d740 10 client.4653 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff57637d740 10 client.4653 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.908+0000 7ff57637d740 20 client.4653 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff57637d740 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] conn(0x55b91f8e11c0 0x55b91f8e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:16.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff57637d740 4 client.4653 mds_command: new command op to 4262 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff57637d740 1 -- 172.21.15.27:0/3158967937 --> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] -- command(tid 0: {"prefix": "status"}) -- 0x55b91f799530 con 0x55b91f8e11c0 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff570c4e640 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] conn(0x55b91f8e11c0 0x55b91f8e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff54bfff640 10 client.4653 ms_handle_connect on v2:172.21.15.137:6836/1990627502 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff570c4e640 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] conn(0x55b91f8e11c0 0x55b91f8e15a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=90ac3ad473e61ede server_cookie=d223abe87f4b12bf in_seq=0 out_seq=0 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff54bfff640 1 -- 172.21.15.27:0/3158967937 <== mds.4 v2:172.21.15.137:6836/1990627502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b91f799530 con 0x55b91f8e11c0 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff54bfff640 10 client.4653 handle_command_reply: tid=0 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff57637d740 1 client.4653 shutdown 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff57637d740 1 -- 172.21.15.27:0/3158967937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 msgr2=0x55b91f879600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.909+0000 7ff57637d740 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f879600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff568077bd0 tx=0x7ff568036f90 comp rx=0 tx=0).stop 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff57637d740 1 -- 172.21.15.27:0/3158967937 shutdown_connections 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff57637d740 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:6836/1990627502,v1:172.21.15.137:6838/1990627502] conn(0x55b91f8e11c0 0x55b91f8e15a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff57637d740 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55b91f8676e0 0x55b91f84e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff57637d740 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b91f867300 0x55b91f8dae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff57637d740 1 --2- 172.21.15.27:0/3158967937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b91f845d90 0x55b91f879600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff54bfff640 0 client.4653 ms_handle_reset on v2:172.21.15.137:6836/1990627502 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff57637d740 1 -- 172.21.15.27:0/3158967937 >> 172.21.15.27:0/3158967937 conn(0x55b91f825da0 msgr2=0x55b91f844de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff57637d740 1 -- 172.21.15.27:0/3158967937 shutdown_connections 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff57637d740 1 -- 172.21.15.27:0/3158967937 wait complete. 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff57637d740 20 client.4653 trim_cache size 0 max 16384 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "056b9857-269e-4bc1-b62f-8882e32cd3e3", 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 4, 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4262, 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-20T10:56:16.915 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 11.044520488, 2024-04-20T10:56:16.916 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 17, 2024-04-20T10:56:16.916 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 20, 2024-04-20T10:56:16.916 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 20, 2024-04-20T10:56:16.916 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 41.581853756999998 2024-04-20T10:56:16.916 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff5736b3640 1 -- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff55006c920 msgr2=0x7ff55006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff5736b3640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff55006c920 0x7ff55006ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0cbd60 tx=0x7ff55c008040 comp rx=0 tx=0).stop 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff5736b3640 1 -- 172.21.15.27:0/2215957742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c071b40 msgr2=0x7ff56c153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.910+0000 7ff5736b3640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c071b40 0x7ff56c153290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff568000c00 tx=0x7ff5680046c0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.911+0000 7ff5736b3640 1 -- 172.21.15.27:0/2215957742 shutdown_connections 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.911+0000 7ff5736b3640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff55006c920 0x7ff55006ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.911+0000 7ff5736b3640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff56c151d20 0x7ff56c13d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.911+0000 7ff5736b3640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff56c151940 0x7ff56c0c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.911+0000 7ff5736b3640 1 --2- 172.21.15.27:0/2215957742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff56c071b40 0x7ff56c153290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.911+0000 7ff5736b3640 1 -- 172.21.15.27:0/2215957742 >> 172.21.15.27:0/2215957742 conn(0x7ff56c0ccbd0 msgr2=0x7ff56c147b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.911+0000 7ff5736b3640 1 -- 172.21.15.27:0/2215957742 shutdown_connections 2024-04-20T10:56:16.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:16.911+0000 7ff5736b3640 1 -- 172.21.15.27:0/2215957742 wait complete. 2024-04-20T10:56:16.924 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T10:56:17.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.032+0000 7fceb73e6640 1 Processor -- start 2024-04-20T10:56:17.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.033+0000 7fceb73e6640 1 -- start start 2024-04-20T10:56:17.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.033+0000 7fceb73e6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 0x7fceb015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fceb73e6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb015ec70 con 0x7fceb015f710 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fceb73e6640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb006e510 con 0x7fceb01556f0 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fceb73e6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fceb006ead0 con 0x7fceb006ca40 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fceb4981640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fceb015f710 0x7fceb0163e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37338/0 (socket says 172.21.15.27:37338) 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fceb4981640 1 -- 172.21.15.27:0/2294737446 learned_addr learned my addr 172.21.15.27:0/2294737446 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fceb5983640 1 --2- 172.21.15.27:0/2294737446 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 0x7fceb015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fcea7fff640 1 -- 172.21.15.27:0/2294737446 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 621887907 0 0) 0x7fceb015ec70 con 0x7fceb015f710 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fcea7fff640 1 -- 172.21.15.27:0/2294737446 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce9c003520 con 0x7fceb015f710 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fcea7fff640 1 -- 172.21.15.27:0/2294737446 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3028305515 0 0) 0x7fceb006e510 con 0x7fceb01556f0 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fcea7fff640 1 -- 172.21.15.27:0/2294737446 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb015ec70 con 0x7fceb01556f0 2024-04-20T10:56:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fcea7fff640 1 -- 172.21.15.27:0/2294737446 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3051911490 0 0) 0x7fce9c003520 con 0x7fceb015f710 2024-04-20T10:56:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fceb5983640 1 -- 172.21.15.27:0/2294737446 >> v1:172.21.15.137:6789/0 conn(0x7fceb01556f0 legacy=0x7fceb00cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fceb5983640 1 -- 172.21.15.27:0/2294737446 >> v1:172.21.15.27:6789/0 conn(0x7fceb015f710 legacy=0x7fceb0163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.034+0000 7fceb5983640 1 -- 172.21.15.27:0/2294737446 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb006e7f0 con 0x7fceb006ca40 2024-04-20T10:56:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.035+0000 7fceb5983640 1 --2- 172.21.15.27:0/2294737446 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 0x7fceb015c9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009dc0 tx=0x7fcea802be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc969aa414aeb648 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.035+0000 7fcea7fff640 1 -- 172.21.15.27:0/2294737446 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8039070 con 0x7fceb006ca40 2024-04-20T10:56:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.035+0000 7fcea7fff640 1 -- 172.21.15.27:0/2294737446 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea802c9a0 con 0x7fceb006ca40 2024-04-20T10:56:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.035+0000 7fcea7fff640 1 -- 172.21.15.27:0/2294737446 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0001670 con 0x7fceb006ca40 2024-04-20T10:56:17.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.036+0000 7fceb73e6640 1 -- 172.21.15.27:0/2294737446 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 msgr2=0x7fceb015c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.036+0000 7fceb73e6640 1 --2- 172.21.15.27:0/2294737446 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 0x7fceb015c9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009dc0 tx=0x7fcea802be70 comp rx=0 tx=0).stop 2024-04-20T10:56:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.036+0000 7fceb73e6640 1 -- 172.21.15.27:0/2294737446 shutdown_connections 2024-04-20T10:56:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.036+0000 7fceb73e6640 1 --2- 172.21.15.27:0/2294737446 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 0x7fceb015c9d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.036+0000 7fceb73e6640 1 -- 172.21.15.27:0/2294737446 >> 172.21.15.27:0/2294737446 conn(0x7fceb0067ae0 msgr2=0x7fceb0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.036+0000 7fceb73e6640 1 -- 172.21.15.27:0/2294737446 shutdown_connections 2024-04-20T10:56:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.036+0000 7fceb73e6640 1 -- 172.21.15.27:0/2294737446 wait complete. 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.037+0000 7fceb73e6640 1 Processor -- start 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.037+0000 7fceb73e6640 1 -- start start 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb73e6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 0x7fceb01e5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb73e6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fceb01556f0 0x7fceb01506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb73e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb015f710 0x7fceb00c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb73e6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00c9800 con 0x7fceb01556f0 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb73e6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fceb015f1f0 con 0x7fceb015f710 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb73e6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fceb00c9980 con 0x7fceb006ca40 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb4981640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fceb01556f0 0x7fceb01506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb5182640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 0x7fceb01e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb5983640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb015f710 0x7fceb00c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb4981640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fceb01556f0 0x7fceb01506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35212/0 (socket says 172.21.15.27:35212) 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb4981640 1 -- 172.21.15.27:0/2929177311 learned_addr learned my addr 172.21.15.27:0/2929177311 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb4981640 1 -- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 msgr2=0x7fceb01e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb4981640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 0x7fceb01e5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb4981640 1 -- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb015f710 msgr2=0x7fceb00c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb4981640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb015f710 0x7fceb00c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb4981640 1 -- 172.21.15.27:0/2929177311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb00c8500 con 0x7fceb01556f0 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb5983640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb015f710 0x7fceb00c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb4981640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fceb01556f0 0x7fceb01506a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c550 tx=0x7fcea000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fcea5ffb640 1 -- 172.21.15.27:0/2929177311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0017070 con 0x7fceb01556f0 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fcea5ffb640 1 -- 172.21.15.27:0/2929177311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea0005ce0 con 0x7fceb01556f0 2024-04-20T10:56:17.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fceb73e6640 1 -- 172.21.15.27:0/2929177311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb01e69a0 con 0x7fceb01556f0 2024-04-20T10:56:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.038+0000 7fcea5ffb640 1 -- 172.21.15.27:0/2929177311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00072b0 con 0x7fceb01556f0 2024-04-20T10:56:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.039+0000 7fceb73e6640 1 -- 172.21.15.27:0/2929177311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb00c7380 con 0x7fceb01556f0 2024-04-20T10:56:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.040+0000 7fcea5ffb640 1 -- 172.21.15.27:0/2929177311 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcea001a070 con 0x7fceb01556f0 2024-04-20T10:56:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.040+0000 7fcea5ffb640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fce9406c8f0 0x7fce9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.040+0000 7fcea5ffb640 1 -- 172.21.15.27:0/2929177311 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcea008af40 con 0x7fceb01556f0 2024-04-20T10:56:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.040+0000 7fceb5182640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fce9406c8f0 0x7fce9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.041+0000 7fceb5182640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fce9406c8f0 0x7fce9406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fceac0011e0 tx=0x7fceac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.041+0000 7fceb9fd5740 1 Processor -- start 2024-04-20T10:56:17.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb9fd5740 1 -- start start 2024-04-20T10:56:17.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c1ada0 0x560df7c5ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c5c6e0 0x560df7c5b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560df7b2e290 con 0x560df7c5c300 2024-04-20T10:56:17.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560df7b8ba40 con 0x560df7c1ada0 2024-04-20T10:56:17.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x560df7b46a30 con 0x560df7c5c6e0 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb5182640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb5182640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35220/0 (socket says 172.21.15.27:35220) 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb5182640 1 -- 172.21.15.27:0/4147997851 learned_addr learned my addr 172.21.15.27:0/4147997851 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb4981640 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c1ada0 0x560df7c5ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.042+0000 7fceb5983640 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c5c6e0 0x560df7c5b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb5182640 1 -- 172.21.15.27:0/4147997851 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c5c6e0 msgr2=0x560df7c5b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb5182640 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c5c6e0 0x560df7c5b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb5182640 1 -- 172.21.15.27:0/4147997851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c1ada0 msgr2=0x560df7c5ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb5182640 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c1ada0 0x560df7c5ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb5182640 1 -- 172.21.15.27:0/4147997851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df7b39c10 con 0x560df7c5c300 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb4981640 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c1ada0 0x560df7c5ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb5182640 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1c0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fceac00fbd0 tx=0x7fceac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2928b091b848ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fce9affd640 1 -- 172.21.15.27:0/4147997851 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac025070 con 0x560df7c5c300 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fce9affd640 1 -- 172.21.15.27:0/4147997851 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceac01b050 con 0x560df7c5c300 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fce9affd640 1 -- 172.21.15.27:0/4147997851 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac020410 con 0x560df7c5c300 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb9fd5740 1 -- 172.21.15.27:0/4147997851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 msgr2=0x560df7c1c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1c0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fceac00fbd0 tx=0x7fceac012000 comp rx=0 tx=0).stop 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb9fd5740 1 -- 172.21.15.27:0/4147997851 shutdown_connections 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c5c6e0 0x560df7c5b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.043+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c1ada0 0x560df7c5ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.044+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/4147997851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1c0a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.044+0000 7fceb9fd5740 1 -- 172.21.15.27:0/4147997851 >> 172.21.15.27:0/4147997851 conn(0x560df7c17f10 msgr2=0x560df7c1b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.044+0000 7fceb9fd5740 1 -- 172.21.15.27:0/4147997851 shutdown_connections 2024-04-20T10:56:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.044+0000 7fceb9fd5740 1 -- 172.21.15.27:0/4147997851 wait complete. 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.044+0000 7fceb9fd5740 1 -- start start 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.044+0000 7fceb9fd5740 1 Processor -- start 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c300 0x560df7c60850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c6e0 0x560df7c19740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 0x560df7c6c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560df7b85140 con 0x560df7c5c6e0 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560df7b7fe20 con 0x560df7c5c300 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x560df7b8dd20 con 0x560df7c59080 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb9fd5740 10 client.0 _ll_register_callbacks cb 0x560df7b765c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb4981640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c6e0 0x560df7c19740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb4981640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c6e0 0x560df7c19740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35232/0 (socket says 172.21.15.27:35232) 2024-04-20T10:56:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb4981640 1 -- 172.21.15.27:0/740042043 learned_addr learned my addr 172.21.15.27:0/740042043 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb5182640 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c300 0x560df7c60850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb5983640 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 0x560df7c6c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb4981640 1 -- 172.21.15.27:0/740042043 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 msgr2=0x560df7c6c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb4981640 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 0x560df7c6c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb4981640 1 -- 172.21.15.27:0/740042043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c300 msgr2=0x560df7c60850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb4981640 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c300 0x560df7c60850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb4981640 1 -- 172.21.15.27:0/740042043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df78932d0 con 0x560df7c5c6e0 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.045+0000 7fceb5182640 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c300 0x560df7c60850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb4981640 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c6e0 0x560df7c19740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c520 tx=0x7fcea005d170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da99c4ede03fbea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fce8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fce8ffff640 1 -- 172.21.15.27:0/740042043 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0071170 con 0x560df7c5c6e0 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fce8ffff640 1 -- 172.21.15.27:0/740042043 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcea00780b0 con 0x560df7c5c6e0 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb9fd5740 1 -- 172.21.15.27:0/740042043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceac01b050 con 0x560df7c5c6e0 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fce8ffff640 1 -- 172.21.15.27:0/740042043 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fcea0082e60 con 0x560df7c5c6e0 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb9fd5740 10 client.4665 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb9fd5740 1 -- 172.21.15.27:0/740042043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560df795d6b0 con 0x560df7c5c6e0 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fce8ffff640 1 -- 172.21.15.27:0/740042043 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceac01b050 con 0x560df7c5c6e0 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fce8ffff640 1 -- 172.21.15.27:0/740042043 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7fcea0007450 con 0x560df7c5c6e0 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb9fd5740 10 client.4665 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb9fd5740 10 client.4665 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb9fd5740 20 client.4665 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] conn(0x560df7c791d0 0x560df7c7b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb9fd5740 4 client.4665 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb9fd5740 1 -- 172.21.15.27:0/740042043 --> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df7b07510 con 0x560df7c791d0 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.046+0000 7fceb5182640 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] conn(0x560df7c791d0 0x560df7c7b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.047+0000 7fceb5182640 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] conn(0x560df7c791d0 0x560df7c7b5c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e5fb5332963ca223 server_cookie=a4c0b6cfbd43bcaa in_seq=0 out_seq=0 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.047+0000 7fce8ffff640 10 client.4665 ms_handle_connect on v2:172.21.15.27:6832/859496751 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.047+0000 7fce8ffff640 1 -- 172.21.15.27:0/740042043 <== mds.3 v2:172.21.15.27:6832/859496751 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560df7b07510 con 0x560df7c791d0 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.047+0000 7fce8ffff640 10 client.4665 handle_command_reply: tid=0 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.047+0000 7fceb9fd5740 1 client.4665 shutdown 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.047+0000 7fceb9fd5740 1 -- 172.21.15.27:0/740042043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c6e0 msgr2=0x560df7c19740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.047+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c6e0 0x560df7c19740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c520 tx=0x7fcea005d170 comp rx=0 tx=0).stop 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fceb9fd5740 1 -- 172.21.15.27:0/740042043 shutdown_connections 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] conn(0x560df7c791d0 0x560df7c7b5c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 0x560df7c6c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c6e0 0x560df7c19740 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fce8ffff640 0 client.4665 ms_handle_reset on v2:172.21.15.27:6832/859496751 2024-04-20T10:56:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/740042043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c300 0x560df7c60850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fceb9fd5740 1 -- 172.21.15.27:0/740042043 >> 172.21.15.27:0/740042043 conn(0x560df7c1ada0 msgr2=0x560df7c44d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fceb9fd5740 1 -- 172.21.15.27:0/740042043 shutdown_connections 2024-04-20T10:56:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fceb9fd5740 1 -- 172.21.15.27:0/740042043 wait complete. 2024-04-20T10:56:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.048+0000 7fceb9fd5740 20 client.4665 trim_cache size 0 max 16384 2024-04-20T10:56:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.066+0000 7fceb9fd5740 1 Processor -- start 2024-04-20T10:56:17.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.066+0000 7fceb9fd5740 1 -- start start 2024-04-20T10:56:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 0x560df7c6c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c6e0 0x560df7c1bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00caf70 con 0x560df7c5c300 2024-04-20T10:56:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560df7b3d980 con 0x560df7c5c6e0 2024-04-20T10:56:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x560df7c704d0 con 0x560df7c59080 2024-04-20T10:56:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb4981640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb4981640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35242/0 (socket says 172.21.15.27:35242) 2024-04-20T10:56:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb4981640 1 -- 172.21.15.27:0/1615838142 learned_addr learned my addr 172.21.15.27:0/1615838142 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb5182640 1 --2- 172.21.15.27:0/1615838142 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 0x560df7c6c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb5983640 1 --2- 172.21.15.27:0/1615838142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c6e0 0x560df7c1bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb4981640 1 -- 172.21.15.27:0/1615838142 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 msgr2=0x560df7c6c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb4981640 1 --2- 172.21.15.27:0/1615838142 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 0x560df7c6c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb4981640 1 -- 172.21.15.27:0/1615838142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c6e0 msgr2=0x560df7c1bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb4981640 1 --2- 172.21.15.27:0/1615838142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c6e0 0x560df7c1bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb4981640 1 -- 172.21.15.27:0/1615838142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df7c1b180 con 0x560df7c5c300 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fceb4981640 1 --2- 172.21.15.27:0/1615838142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1b5d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcea0058800 tx=0x7fcea005a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf981ec179f6a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fce9a7fc640 1 -- 172.21.15.27:0/1615838142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00615e0 con 0x560df7c5c300 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fce9a7fc640 1 -- 172.21.15.27:0/1615838142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea0056dd0 con 0x560df7c5c300 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.067+0000 7fce9a7fc640 1 -- 172.21.15.27:0/1615838142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00780b0 con 0x560df7c5c300 2024-04-20T10:56:17.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.068+0000 7fceb9fd5740 1 -- 172.21.15.27:0/1615838142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 msgr2=0x560df7c1b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.068+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/1615838142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1b5d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcea0058800 tx=0x7fcea005a000 comp rx=0 tx=0).stop 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.068+0000 7fceb9fd5740 1 -- 172.21.15.27:0/1615838142 shutdown_connections 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.068+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/1615838142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c5c6e0 0x560df7c1bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.068+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/1615838142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c1b5d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.068+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/1615838142 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c59080 0x560df7c6c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.068+0000 7fceb9fd5740 1 -- 172.21.15.27:0/1615838142 >> 172.21.15.27:0/1615838142 conn(0x560df7c1ada0 msgr2=0x560df7c11070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.068+0000 7fceb9fd5740 1 -- 172.21.15.27:0/1615838142 shutdown_connections 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.068+0000 7fceb9fd5740 1 -- 172.21.15.27:0/1615838142 wait complete. 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.069+0000 7fceb9fd5740 1 -- start start 2024-04-20T10:56:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.069+0000 7fceb9fd5740 1 Processor -- start 2024-04-20T10:56:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.069+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c59080 0x560df7c5f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.069+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c154f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.069+0000 7fceb9fd5740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c5c6e0 0x560df7cd0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560df7c57a20 con 0x560df7c5c300 2024-04-20T10:56:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560df7c70160 con 0x560df7c59080 2024-04-20T10:56:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb9fd5740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x560df7c56e00 con 0x560df7c5c6e0 2024-04-20T10:56:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb9fd5740 10 client.0 _ll_register_callbacks cb 0x7fcea0058b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb4981640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb4981640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c154f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35258/0 (socket says 172.21.15.27:35258) 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb4981640 1 -- 172.21.15.27:0/3509241437 learned_addr learned my addr 172.21.15.27:0/3509241437 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb5182640 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c59080 0x560df7c5f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb4981640 1 -- 172.21.15.27:0/3509241437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c5c6e0 msgr2=0x560df7cd0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb4981640 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c5c6e0 0x560df7cd0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb4981640 1 -- 172.21.15.27:0/3509241437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c59080 msgr2=0x560df7c5f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb4981640 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c59080 0x560df7c5f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb4981640 1 -- 172.21.15.27:0/3509241437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df7c3fb50 con 0x560df7c5c300 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb5182640 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c59080 0x560df7c5f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb4981640 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c154f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcea0058cc0 tx=0x7fcea0001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eddc748691615c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fce8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fce8ffff640 1 -- 172.21.15.27:0/3509241437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea00615e0 con 0x560df7c5c300 2024-04-20T10:56:17.076 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fce8ffff640 1 -- 172.21.15.27:0/3509241437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcea0089380 con 0x560df7c5c300 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb9fd5740 1 -- 172.21.15.27:0/3509241437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea0056dd0 con 0x560df7c5c300 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fce8ffff640 1 -- 172.21.15.27:0/3509241437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fcea0058d30 con 0x560df7c5c300 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb9fd5740 10 client.4671 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fceb9fd5740 1 -- 172.21.15.27:0/3509241437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560df7c3f8b0 con 0x560df7c5c300 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.070+0000 7fce8ffff640 1 -- 172.21.15.27:0/3509241437 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea0056dd0 con 0x560df7c5c300 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fce8ffff640 1 -- 172.21.15.27:0/3509241437 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7fcea0007450 con 0x560df7c5c300 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 10 client.4671 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 10 client.4671 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 20 client.4671 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] conn(0x560df7c449f0 0x560df7ce3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 4 client.4671 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 -- 172.21.15.27:0/3509241437 --> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] -- command(tid 0: {"prefix": "status"}) -- 0x560df7b8e530 con 0x560df7c449f0 2024-04-20T10:56:17.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb5182640 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] conn(0x560df7c449f0 0x560df7ce3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb5182640 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] conn(0x560df7c449f0 0x560df7ce3ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b8c1acbe83643545 server_cookie=9fde866195196ddc in_seq=0 out_seq=0 2024-04-20T10:56:17.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fce8ffff640 10 client.4671 ms_handle_connect on v2:172.21.15.27:6832/859496751 2024-04-20T10:56:17.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fce8ffff640 1 -- 172.21.15.27:0/3509241437 <== mds.3 v2:172.21.15.27:6832/859496751 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560df7b8e530 con 0x560df7c449f0 2024-04-20T10:56:17.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fce8ffff640 10 client.4671 handle_command_reply: tid=0 2024-04-20T10:56:17.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 client.4671 shutdown 2024-04-20T10:56:17.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 -- 172.21.15.27:0/3509241437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 msgr2=0x560df7c154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c154f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcea0058cc0 tx=0x7fcea0001150 comp rx=0 tx=0).stop 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 -- 172.21.15.27:0/3509241437 shutdown_connections 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.27:6832/859496751,v1:172.21.15.27:6835/859496751] conn(0x560df7c449f0 0x560df7ce3ca0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560df7c5c6e0 0x560df7cd0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560df7c5c300 0x560df7c154f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fce8ffff640 0 client.4671 ms_handle_reset on v2:172.21.15.27:6832/859496751 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 --2- 172.21.15.27:0/3509241437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560df7c59080 0x560df7c5f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 -- 172.21.15.27:0/3509241437 >> 172.21.15.27:0/3509241437 conn(0x560df7c1ada0 msgr2=0x560df7c588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.071+0000 7fceb9fd5740 1 -- 172.21.15.27:0/3509241437 shutdown_connections 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb9fd5740 1 -- 172.21.15.27:0/3509241437 wait complete. 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb9fd5740 20 client.4671 trim_cache size 0 max 16384 2024-04-20T10:56:17.079 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "056b9857-269e-4bc1-b62f-8882e32cd3e3", 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 3, 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4272, 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 17.217376601000002, 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 17, 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 20, 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 20, 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 41.761021003000003 2024-04-20T10:56:17.080 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T10:56:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 -- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fce9406c8f0 msgr2=0x7fce9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fce9406c8f0 0x7fce9406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fceac0011e0 tx=0x7fceac009040 comp rx=0 tx=0).stop 2024-04-20T10:56:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 -- 172.21.15.27:0/2929177311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fceb01556f0 msgr2=0x7fceb01506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fceb01556f0 0x7fceb01506a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c550 tx=0x7fcea000ca10 comp rx=0 tx=0).stop 2024-04-20T10:56:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 -- 172.21.15.27:0/2929177311 shutdown_connections 2024-04-20T10:56:17.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fce9406c8f0 0x7fce9406edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fceb015f710 0x7fceb00c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fceb01556f0 0x7fceb01506a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 --2- 172.21.15.27:0/2929177311 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fceb006ca40 0x7fceb01e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 -- 172.21.15.27:0/2929177311 >> 172.21.15.27:0/2929177311 conn(0x7fceb0067ae0 msgr2=0x7fceb00c4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 -- 172.21.15.27:0/2929177311 shutdown_connections 2024-04-20T10:56:17.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.072+0000 7fceb73e6640 1 -- 172.21.15.27:0/2929177311 wait complete. 2024-04-20T10:56:17.083 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-20T10:56:17.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.184+0000 7fd4a8d98640 1 Processor -- start 2024-04-20T10:56:17.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.184+0000 7fd4a8d98640 1 -- start start 2024-04-20T10:56:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.185+0000 7fd4a8d98640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 0x7fd4a40b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.185+0000 7fd4a8d98640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a40b5e70 con 0x7fd4a4155950 2024-04-20T10:56:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.185+0000 7fd4a8d98640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a415b9c0 con 0x7fd4a40d0c90 2024-04-20T10:56:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.185+0000 7fd4a8d98640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a415bca0 con 0x7fd4a40b54f0 2024-04-20T10:56:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.185+0000 7fd4a259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd4a4155950 0x7fd4a40cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37350/0 (socket says 172.21.15.27:37350) 2024-04-20T10:56:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.185+0000 7fd4a259c640 1 -- 172.21.15.27:0/1795367578 learned_addr learned my addr 172.21.15.27:0/1795367578 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:17.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288471555 0 0) 0x7fd4a40b5e70 con 0x7fd4a4155950 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd490003520 con 0x7fd4a4155950 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 0x7fd4a40b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2760833503 0 0) 0x7fd490003520 con 0x7fd4a4155950 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 msgr2=0x7fd4a40b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 0x7fd4a40b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 >> v1:172.21.15.137:6789/0 conn(0x7fd4a40d0c90 legacy=0x7fd4a40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a415bf80 con 0x7fd4a4155950 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7fd49c001670 con 0x7fd4a4155950 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 0x7fd4a40b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 >> v1:172.21.15.27:6789/0 conn(0x7fd4a4155950 legacy=0x7fd4a40cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 0x7fd4900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd490004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd490004d40 0x7fd4900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a415bca0 con 0x7fd490003fd0 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd49000bbb0 con 0x7fd490004d40 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd49000bdb0 con 0x7fd4a40b54f0 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd490004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a259c640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 0x7fd4900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a2d9d640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd490004d40 0x7fd4900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 msgr2=0x7fd4900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 0x7fd4900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd490004d40 msgr2=0x7fd4900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd490004d40 0x7fd4900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/1795367578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd490003520 con 0x7fd490003fd0 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.186+0000 7fd4a259c640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a40b54f0 0x7fd4900039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.187+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd490004800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0098a0 tx=0x7fd490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c6d80e540d5ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.187+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00f040 con 0x7fd490003fd0 2024-04-20T10:56:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.187+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c004230 con 0x7fd490003fd0 2024-04-20T10:56:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.187+0000 7fd4a159a640 1 -- 172.21.15.27:0/1795367578 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c02d040 con 0x7fd490003fd0 2024-04-20T10:56:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.188+0000 7fd4a8d98640 1 -- 172.21.15.27:0/1795367578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 msgr2=0x7fd490004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.188+0000 7fd4a8d98640 1 --2- 172.21.15.27:0/1795367578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd490004800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0098a0 tx=0x7fd490005830 comp rx=0 tx=0).stop 2024-04-20T10:56:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.188+0000 7fd4a2d9d640 1 -- 172.21.15.27:0/1795367578 reap_dead start 2024-04-20T10:56:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.188+0000 7fd4a8d98640 1 -- 172.21.15.27:0/1795367578 shutdown_connections 2024-04-20T10:56:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.188+0000 7fd4a8d98640 1 -- 172.21.15.27:0/1795367578 >> 172.21.15.27:0/1795367578 conn(0x7fd4a40676c0 msgr2=0x7fd4a4067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.188+0000 7fd4a8d98640 1 -- 172.21.15.27:0/1795367578 shutdown_connections 2024-04-20T10:56:17.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.188+0000 7fd4a8d98640 1 -- 172.21.15.27:0/1795367578 wait complete. 2024-04-20T10:56:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.189+0000 7fd4a8d98640 1 Processor -- start 2024-04-20T10:56:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.189+0000 7fd4a8d98640 1 -- start start 2024-04-20T10:56:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a8d98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd4a4150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a8d98640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a41f5170 0x7fd4a40d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a8d98640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a41f55f0 0x7fd4a40d15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a8d98640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a415fb90 con 0x7fd490003fd0 2024-04-20T10:56:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a8d98640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a415c540 con 0x7fd4a41f5170 2024-04-20T10:56:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a8d98640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a415fd10 con 0x7fd4a41f55f0 2024-04-20T10:56:17.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd4a4150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd4a4150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35276/0 (socket says 172.21.15.27:35276) 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a259c640 1 -- 172.21.15.27:0/4126492717 learned_addr learned my addr 172.21.15.27:0/4126492717 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a41f5170 0x7fd4a40d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a2d9d640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a41f55f0 0x7fd4a40d15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a259c640 1 -- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a41f55f0 msgr2=0x7fd4a40d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a259c640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a41f55f0 0x7fd4a40d15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a259c640 1 -- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a41f5170 msgr2=0x7fd4a40d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a259c640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a41f5170 0x7fd4a40d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.190+0000 7fd4a259c640 1 -- 172.21.15.27:0/4126492717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a41f49b0 con 0x7fd490003fd0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.191+0000 7fd4a259c640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd4a4150ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd49c007b60 tx=0x7fd49c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.191+0000 7fd49affd640 1 -- 172.21.15.27:0/4126492717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00fa60 con 0x7fd490003fd0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.191+0000 7fd49affd640 1 -- 172.21.15.27:0/4126492717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd49c018070 con 0x7fd490003fd0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.191+0000 7fd4a8d98640 1 -- 172.21.15.27:0/4126492717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a415ac90 con 0x7fd490003fd0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.191+0000 7fd49affd640 1 -- 172.21.15.27:0/4126492717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c00fe00 con 0x7fd490003fd0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.191+0000 7fd4a8d98640 1 -- 172.21.15.27:0/4126492717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a415b250 con 0x7fd490003fd0 2024-04-20T10:56:17.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.192+0000 7fd49affd640 1 -- 172.21.15.27:0/4126492717 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd49c01d460 con 0x7fd490003fd0 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.193+0000 7fd49affd640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c8f0 0x7fd48406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.193+0000 7fd49affd640 1 -- 172.21.15.27:0/4126492717 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd49c08bf10 con 0x7fd490003fd0 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.193+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c8f0 0x7fd48406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.193+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c8f0 0x7fd48406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd48c035000 tx=0x7fd48c031000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.194+0000 7fd4ab982740 1 Processor -- start 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.194+0000 7fd4ab982740 1 -- start start 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.194+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf3360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.194+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf334da0 0x560fbf376cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.194+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 0x560fbf3755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.194+0000 7fd4ab982740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560fbf248290 con 0x560fbf334da0 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.194+0000 7fd4ab982740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560fbf2a5a40 con 0x560fbf376300 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.194+0000 7fd4ab982740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x560fbf260a30 con 0x560fbf3766e0 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf334da0 0x560fbf376cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf3360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf3360a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:60548/0 (socket says 172.21.15.27:60548) 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf334da0 0x560fbf376cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35290/0 (socket says 172.21.15.27:35290) 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a259c640 1 -- 172.21.15.27:0/4251392614 learned_addr learned my addr 172.21.15.27:0/4251392614 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a2d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 0x560fbf3755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a259c640 1 -- 172.21.15.27:0/4251392614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 msgr2=0x560fbf3755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a259c640 1 --2- 172.21.15.27:0/4251392614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 0x560fbf3755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a259c640 1 -- 172.21.15.27:0/4251392614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf334da0 msgr2=0x560fbf376cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a259c640 1 --2- 172.21.15.27:0/4251392614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf334da0 0x560fbf376cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a259c640 1 -- 172.21.15.27:0/4251392614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbf253c10 con 0x560fbf376300 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd4a259c640 1 --2- 172.21.15.27:0/4251392614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf3360a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0622f0 tx=0x7fd49c05a2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2381746281fab3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd483fff640 1 -- 172.21.15.27:0/4251392614 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c062820 con 0x560fbf376300 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd483fff640 1 -- 172.21.15.27:0/4251392614 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd49c057eb0 con 0x560fbf376300 2024-04-20T10:56:17.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.195+0000 7fd483fff640 1 -- 172.21.15.27:0/4251392614 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c056220 con 0x560fbf376300 2024-04-20T10:56:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 -- 172.21.15.27:0/4251392614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 msgr2=0x560fbf3360a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 --2- 172.21.15.27:0/4251392614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf3360a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0622f0 tx=0x7fd49c05a2b0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 -- 172.21.15.27:0/4251392614 shutdown_connections 2024-04-20T10:56:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 --2- 172.21.15.27:0/4251392614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 0x560fbf3755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 --2- 172.21.15.27:0/4251392614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf334da0 0x560fbf376cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 --2- 172.21.15.27:0/4251392614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf3360a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 -- 172.21.15.27:0/4251392614 >> 172.21.15.27:0/4251392614 conn(0x560fbf331f10 msgr2=0x560fbf3356b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 -- 172.21.15.27:0/4251392614 shutdown_connections 2024-04-20T10:56:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 -- 172.21.15.27:0/4251392614 wait complete. 2024-04-20T10:56:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.196+0000 7fd4ab982740 1 -- start start 2024-04-20T10:56:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.197+0000 7fd4ab982740 1 Processor -- start 2024-04-20T10:56:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.197+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 0x560fbf386a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.197+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf372a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.197+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 0x560fbf37fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.197+0000 7fd4ab982740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560fbf29f140 con 0x560fbf3766e0 2024-04-20T10:56:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.197+0000 7fd4ab982740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560fbf299e20 con 0x560fbf37a2b0 2024-04-20T10:56:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.197+0000 7fd4ab982740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x560fbf2a7d20 con 0x560fbf376300 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.197+0000 7fd4ab982740 10 client.0 _ll_register_callbacks cb 0x560fbf2905c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf372a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf372a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35304/0 (socket says 172.21.15.27:35304) 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/3414106605 learned_addr learned my addr 172.21.15.27:0/3414106605 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a2d9d640 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 0x560fbf37fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a259c640 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 0x560fbf386a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/3414106605 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 msgr2=0x560fbf386a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 0x560fbf386a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/3414106605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 msgr2=0x560fbf37fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.205 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 0x560fbf37fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/3414106605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbefad2d0 con 0x560fbf3766e0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf372a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd48c02bf00 tx=0x7fd48c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7066092ab45ec6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4817fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4817fa640 1 -- 172.21.15.27:0/3414106605 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c003ce0 con 0x560fbf3766e0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4817fa640 1 -- 172.21.15.27:0/3414106605 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd48c055ae0 con 0x560fbf3766e0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4ab982740 1 -- 172.21.15.27:0/3414106605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd49c057eb0 con 0x560fbf3766e0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4817fa640 1 -- 172.21.15.27:0/3414106605 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd48c055ce0 con 0x560fbf3766e0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4ab982740 10 client.4683 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4ab982740 1 -- 172.21.15.27:0/3414106605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560fbf0776b0 con 0x560fbf3766e0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.198+0000 7fd4817fa640 1 -- 172.21.15.27:0/3414106605 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd49c057eb0 con 0x560fbf3766e0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4817fa640 1 -- 172.21.15.27:0/3414106605 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7fd48c05e740 con 0x560fbf3766e0 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4ab982740 10 client.4683 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4ab982740 10 client.4683 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4ab982740 20 client.4683 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:17.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] conn(0x560fbf3803f0 0x560fbf395160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4ab982740 4 client.4683 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4ab982740 1 -- 172.21.15.27:0/3414106605 --> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fbf221510 con 0x560fbf3803f0 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4a259c640 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] conn(0x560fbf3803f0 0x560fbf395160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4817fa640 10 client.4683 ms_handle_connect on v2:172.21.15.27:6834/1680990583 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.199+0000 7fd4a259c640 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] conn(0x560fbf3803f0 0x560fbf395160 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30e30ca7511c145d server_cookie=68181c416d6f6274 in_seq=0 out_seq=0 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4817fa640 1 -- 172.21.15.27:0/3414106605 <== mds.2 v2:172.21.15.27:6834/1680990583 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fbf221510 con 0x560fbf3803f0 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4817fa640 10 client.4683 handle_command_reply: tid=0 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 client.4683 shutdown 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 -- 172.21.15.27:0/3414106605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 msgr2=0x560fbf372a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf372a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd48c02bf00 tx=0x7fd48c02e000 comp rx=0 tx=0).stop 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 -- 172.21.15.27:0/3414106605 shutdown_connections 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] conn(0x560fbf3803f0 0x560fbf395160 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 0x560fbf37fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf372a20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3414106605 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 0x560fbf386a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4817fa640 0 client.4683 ms_handle_reset on v2:172.21.15.27:6834/1680990583 2024-04-20T10:56:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 -- 172.21.15.27:0/3414106605 >> 172.21.15.27:0/3414106605 conn(0x560fbf334da0 msgr2=0x560fbf335590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 -- 172.21.15.27:0/3414106605 shutdown_connections 2024-04-20T10:56:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 1 -- 172.21.15.27:0/3414106605 wait complete. 2024-04-20T10:56:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.200+0000 7fd4ab982740 20 client.4683 trim_cache size 0 max 16384 2024-04-20T10:56:17.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.218+0000 7fd4ab982740 1 Processor -- start 2024-04-20T10:56:17.218 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.218+0000 7fd4ab982740 1 -- start start 2024-04-20T10:56:17.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf373770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 0x560fbf335350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf37a2b0 0x560fbf335890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4ab982740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a40b64c0 con 0x560fbf37a2b0 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4ab982740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560fbf257980 con 0x560fbf376300 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4ab982740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x560fbf37a960 con 0x560fbf3766e0 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf37a2b0 0x560fbf335890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf37a2b0 0x560fbf335890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35310/0 (socket says 172.21.15.27:35310) 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a2d9d640 1 -- 172.21.15.27:0/4273446907 learned_addr learned my addr 172.21.15.27:0/4273446907 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a2d9d640 1 -- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 msgr2=0x560fbf335350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 0x560fbf335350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a259c640 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf373770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a2d9d640 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 0x560fbf335350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a2d9d640 1 -- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 msgr2=0x560fbf373770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a2d9d640 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf373770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a2d9d640 1 -- 172.21.15.27:0/4273446907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbf37f9b0 con 0x560fbf37a2b0 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 0x560fbf335350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a259c640 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf373770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:17.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.219+0000 7fd4a2d9d640 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf37a2b0 0x560fbf335890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd494001d80 tx=0x7fd494001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae466402659f3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4837fe640 1 -- 172.21.15.27:0/4273446907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49c007d10 con 0x560fbf37a2b0 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4837fe640 1 -- 172.21.15.27:0/4273446907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd494017d80 con 0x560fbf37a2b0 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4837fe640 1 -- 172.21.15.27:0/4273446907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494005350 con 0x560fbf37a2b0 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4ab982740 1 -- 172.21.15.27:0/4273446907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf37a2b0 msgr2=0x560fbf335890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4ab982740 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf37a2b0 0x560fbf335890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd494001d80 tx=0x7fd494001610 comp rx=0 tx=0).stop 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4ab982740 1 -- 172.21.15.27:0/4273446907 shutdown_connections 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4ab982740 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf37a2b0 0x560fbf335890 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4ab982740 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf3766e0 0x560fbf335350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4ab982740 1 --2- 172.21.15.27:0/4273446907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf376300 0x560fbf373770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4ab982740 1 -- 172.21.15.27:0/4273446907 >> 172.21.15.27:0/4273446907 conn(0x560fbf334da0 msgr2=0x560fbf37ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4ab982740 1 -- 172.21.15.27:0/4273446907 shutdown_connections 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.220+0000 7fd4ab982740 1 -- 172.21.15.27:0/4273446907 wait complete. 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.221+0000 7fd4ab982740 1 -- start start 2024-04-20T10:56:17.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.221+0000 7fd4ab982740 1 Processor -- start 2024-04-20T10:56:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 0x560fbf371e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf3f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4ab982740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 0x560fbf3eb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4ab982740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560fbf3808e0 con 0x560fbf3766e0 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4ab982740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560fbf380270 con 0x560fbf37a2b0 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4ab982740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x560fbf35d0a0 con 0x560fbf376300 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4ab982740 10 client.0 _ll_register_callbacks cb 0x7fd490003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf3f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a1d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf3f4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35324/0 (socket says 172.21.15.27:35324) 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/3726839020 learned_addr learned my addr 172.21.15.27:0/3726839020 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a2d9d640 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 0x560fbf3eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a259c640 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 0x560fbf371e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/3726839020 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 msgr2=0x560fbf371e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 0x560fbf371e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/3726839020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 msgr2=0x560fbf3eb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 0x560fbf3eb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a1d9b640 1 -- 172.21.15.27:0/3726839020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbf334990 con 0x560fbf3766e0 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4a1d9b640 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf3f4d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x560fbf391f80 tx=0x7fd48c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0bed63b3f05099 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4817fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4817fa640 1 -- 172.21.15.27:0/3726839020 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c003ef0 con 0x560fbf3766e0 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4817fa640 1 -- 172.21.15.27:0/3726839020 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd48c05e070 con 0x560fbf3766e0 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.222+0000 7fd4ab982740 1 -- 172.21.15.27:0/3726839020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd494017d80 con 0x560fbf3766e0 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4817fa640 1 -- 172.21.15.27:0/3726839020 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd48c06a050 con 0x560fbf3766e0 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4ab982740 10 client.4689 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4ab982740 1 -- 172.21.15.27:0/3726839020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560fbf39aae0 con 0x560fbf3766e0 2024-04-20T10:56:17.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4817fa640 1 -- 172.21.15.27:0/3726839020 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd494017d80 con 0x560fbf3766e0 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4817fa640 1 -- 172.21.15.27:0/3726839020 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7fd48c06d020 con 0x560fbf3766e0 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4ab982740 10 client.4689 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4ab982740 10 client.4689 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4ab982740 20 client.4689 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] conn(0x560fbf3333c0 0x560fbf3fdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4ab982740 4 client.4689 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4ab982740 1 -- 172.21.15.27:0/3726839020 --> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] -- command(tid 0: {"prefix": "status"}) -- 0x560fbf374700 con 0x560fbf3333c0 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4a259c640 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] conn(0x560fbf3333c0 0x560fbf3fdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4817fa640 10 client.4689 ms_handle_connect on v2:172.21.15.27:6834/1680990583 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4a259c640 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] conn(0x560fbf3333c0 0x560fbf3fdf10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ced0a9960f1c2bfd server_cookie=b2b119db476f906f in_seq=0 out_seq=0 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4817fa640 1 -- 172.21.15.27:0/3726839020 <== mds.2 v2:172.21.15.27:6834/1680990583 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x560fbf374700 con 0x560fbf3333c0 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4817fa640 10 client.4689 handle_command_reply: tid=0 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.223+0000 7fd4ab982740 1 client.4689 shutdown 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 -- 172.21.15.27:0/3726839020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 msgr2=0x560fbf3f4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf3f4d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x560fbf391f80 tx=0x7fd48c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 -- 172.21.15.27:0/3726839020 shutdown_connections 2024-04-20T10:56:17.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.27:6834/1680990583,v1:172.21.15.27:6837/1680990583] conn(0x560fbf3333c0 0x560fbf3fdf10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf37a2b0 0x560fbf3eb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560fbf3766e0 0x560fbf3f4d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 --2- 172.21.15.27:0/3726839020 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x560fbf376300 0x560fbf371e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4817fa640 0 client.4689 ms_handle_reset on v2:172.21.15.27:6834/1680990583 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 -- 172.21.15.27:0/3726839020 >> 172.21.15.27:0/3726839020 conn(0x560fbf334da0 msgr2=0x560fbf37eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 -- 172.21.15.27:0/3726839020 shutdown_connections 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 1 -- 172.21.15.27:0/3726839020 wait complete. 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4ab982740 20 client.4689 trim_cache size 0 max 16384 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "056b9857-269e-4bc1-b62f-8882e32cd3e3", 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 2, 2024-04-20T10:56:17.232 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4275, 2024-04-20T10:56:17.233 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-20T10:56:17.233 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-20T10:56:17.233 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-20T10:56:17.233 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 23.386671608, 2024-04-20T10:56:17.233 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 17, 2024-04-20T10:56:17.233 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 20, 2024-04-20T10:56:17.233 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 20, 2024-04-20T10:56:17.233 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 41.912856572000003 2024-04-20T10:56:17.233 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 -- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c8f0 msgr2=0x7fd48406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c8f0 0x7fd48406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd48c035000 tx=0x7fd48c031000 comp rx=0 tx=0).stop 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 -- 172.21.15.27:0/4126492717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 msgr2=0x7fd4a4150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd4a4150ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd49c007b60 tx=0x7fd49c0077b0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 -- 172.21.15.27:0/4126492717 shutdown_connections 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c8f0 0x7fd48406edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd4a41f55f0 0x7fd4a40d15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4a41f5170 0x7fd4a40d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 --2- 172.21.15.27:0/4126492717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd490003fd0 0x7fd4a4150ec0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 -- 172.21.15.27:0/4126492717 >> 172.21.15.27:0/4126492717 conn(0x7fd4a40676c0 msgr2=0x7fd4a40d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.224+0000 7fd4a8d98640 1 -- 172.21.15.27:0/4126492717 shutdown_connections 2024-04-20T10:56:17.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.225+0000 7fd4a8d98640 1 -- 172.21.15.27:0/4126492717 wait complete. 2024-04-20T10:56:17.236 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T10:56:17.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.337+0000 7f15b84d2640 1 Processor -- start 2024-04-20T10:56:17.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.337+0000 7f15b84d2640 1 -- start start 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.338+0000 7f15b84d2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b014ff80 0x7f15b015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.338+0000 7f15b84d2640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f15b00c5c20 0x7f15b0163f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.338+0000 7f15b84d2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b015a480 con 0x7f15b015d0e0 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.338+0000 7f15b84d2640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f15b00c7440 con 0x7f15b00c5c20 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.338+0000 7f15b84d2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f15b00c76e0 con 0x7f15b014ff80 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b6a6f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f15b00c5c20 0x7f15b0163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b626e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b014ff80 0x7f15b015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b5a6d640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f15b015d0e0 0x7f15b00cd4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:32956/0 (socket says 172.21.15.27:32956) 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b5a6d640 1 -- 172.21.15.27:0/4265758887 learned_addr learned my addr 172.21.15.27:0/4265758887 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b6a6f640 1 -- 172.21.15.27:0/4265758887 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b014ff80 msgr2=0x7f15b015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/4265758887 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b014ff80 0x7f15b015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b6a6f640 1 -- 172.21.15.27:0/4265758887 >> v1:172.21.15.137:6789/0 conn(0x7f15b015d0e0 legacy=0x7f15b00cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b6a6f640 1 -- 172.21.15.27:0/4265758887 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b00c79a0 con 0x7f15b00c5c20 2024-04-20T10:56:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b526c640 1 -- 172.21.15.27:0/4265758887 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4222577630 0 0) 0x7f15b015a480 con 0x7f15b015d0e0 2024-04-20T10:56:17.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.339+0000 7f15b626e640 1 --2- 172.21.15.27:0/4265758887 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b014ff80 0x7f15b015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.340+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/4265758887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b00c5c20 0x7f15b0163f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009d90 tx=0x7f15a802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40832a07446d03cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.340+0000 7f15b526c640 1 -- 172.21.15.27:0/4265758887 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8039070 con 0x7f15b00c5c20 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.340+0000 7f15b526c640 1 -- 172.21.15.27:0/4265758887 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a802cc50 con 0x7f15b00c5c20 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.340+0000 7f15b526c640 1 -- 172.21.15.27:0/4265758887 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8034440 con 0x7f15b00c5c20 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.340+0000 7f15b84d2640 1 -- 172.21.15.27:0/4265758887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b00c5c20 msgr2=0x7f15b0163f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.340+0000 7f15b84d2640 1 --2- 172.21.15.27:0/4265758887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b00c5c20 0x7f15b0163f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009d90 tx=0x7f15a802bea0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.341+0000 7f15b84d2640 1 -- 172.21.15.27:0/4265758887 shutdown_connections 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.341+0000 7f15b84d2640 1 --2- 172.21.15.27:0/4265758887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b00c5c20 0x7f15b0163f30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.341+0000 7f15b84d2640 1 --2- 172.21.15.27:0/4265758887 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b014ff80 0x7f15b015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.341+0000 7f15b84d2640 1 -- 172.21.15.27:0/4265758887 >> 172.21.15.27:0/4265758887 conn(0x7f15b006d5b0 msgr2=0x7f15b006d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.341+0000 7f15b84d2640 1 -- 172.21.15.27:0/4265758887 shutdown_connections 2024-04-20T10:56:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.341+0000 7f15b84d2640 1 -- 172.21.15.27:0/4265758887 wait complete. 2024-04-20T10:56:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.342+0000 7f15b84d2640 1 Processor -- start 2024-04-20T10:56:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.342+0000 7f15b84d2640 1 -- start start 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.342+0000 7f15b84d2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b00c5c20 0x7f15b014de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.342+0000 7f15b84d2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b014ff80 0x7f15b0154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.342+0000 7f15b84d2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b015d0e0 0x7f15b01553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.342+0000 7f15b84d2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0159c80 con 0x7f15b014ff80 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.342+0000 7f15b84d2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01599a0 con 0x7f15b00c5c20 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.342+0000 7f15b84d2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f15b0159f60 con 0x7f15b015d0e0 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b5a6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b014ff80 0x7f15b0154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b626e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b00c5c20 0x7f15b014de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b5a6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b014ff80 0x7f15b0154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35332/0 (socket says 172.21.15.27:35332) 2024-04-20T10:56:17.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b5a6d640 1 -- 172.21.15.27:0/4141903435 learned_addr learned my addr 172.21.15.27:0/4141903435 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b015d0e0 0x7f15b01553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b5a6d640 1 -- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b015d0e0 msgr2=0x7f15b01553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b5a6d640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b015d0e0 0x7f15b01553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b5a6d640 1 -- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b00c5c20 msgr2=0x7f15b014de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b5a6d640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b00c5c20 0x7f15b014de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b5a6d640 1 -- 172.21.15.27:0/4141903435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b0153dd0 con 0x7f15b014ff80 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b5a6d640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b014ff80 0x7f15b0154e70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00c440 tx=0x7f15ac00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15a2ffd640 1 -- 172.21.15.27:0/4141903435 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac018070 con 0x7f15b014ff80 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15a2ffd640 1 -- 172.21.15.27:0/4141903435 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac0074e0 con 0x7f15b014ff80 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b84d2640 1 -- 172.21.15.27:0/4141903435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b01540b0 con 0x7f15b014ff80 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15a2ffd640 1 -- 172.21.15.27:0/4141903435 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac004a20 con 0x7f15b014ff80 2024-04-20T10:56:17.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.343+0000 7f15b84d2640 1 -- 172.21.15.27:0/4141903435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b0154670 con 0x7f15b014ff80 2024-04-20T10:56:17.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.345+0000 7f15a2ffd640 1 -- 172.21.15.27:0/4141903435 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15ac004bc0 con 0x7f15b014ff80 2024-04-20T10:56:17.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.345+0000 7f15a2ffd640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f159406c8f0 0x7f159406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.345+0000 7f15a2ffd640 1 -- 172.21.15.27:0/4141903435 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f15ac08aed0 con 0x7f15b014ff80 2024-04-20T10:56:17.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.346+0000 7f15b626e640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f159406c8f0 0x7f159406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.346+0000 7f15b626e640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f159406c8f0 0x7f159406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009560 tx=0x7f15a4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.346+0000 7f15bb1a5740 1 Processor -- start 2024-04-20T10:56:17.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15bb1a5740 1 -- start start 2024-04-20T10:56:17.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c45d9da0 0x55c0c461bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b6e0 0x55c0c461a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c0c44ed290 con 0x55c0c461b6e0 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0c454aa40 con 0x55c0c45d9da0 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x55c0c4505a30 con 0x55c0c461b300 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15b6a6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b6e0 0x55c0c461a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15b6a6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b6e0 0x55c0c461a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35338/0 (socket says 172.21.15.27:35338) 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15b6a6f640 1 -- 172.21.15.27:0/4006745916 learned_addr learned my addr 172.21.15.27:0/4006745916 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15b5a6d640 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c45d9da0 0x55c0c461bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.347+0000 7f15b626e640 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f15b6a6f640 1 -- 172.21.15.27:0/4006745916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 msgr2=0x55c0c45db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f15b6a6f640 1 -- 172.21.15.27:0/4006745916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c45d9da0 msgr2=0x55c0c461bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c45d9da0 0x55c0c461bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f15b6a6f640 1 -- 172.21.15.27:0/4006745916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c44f8c10 con 0x55c0c461b6e0 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f15b5a6d640 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c45d9da0 0x55c0c461bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b6e0 0x55c0c461a5e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f15a8037040 tx=0x7f15a802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6958665cb3665f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f1593fff640 1 -- 172.21.15.27:0/4006745916 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8045070 con 0x55c0c461b6e0 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f1593fff640 1 -- 172.21.15.27:0/4006745916 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a803a040 con 0x55c0c461b6e0 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.348+0000 7f1593fff640 1 -- 172.21.15.27:0/4006745916 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8039040 con 0x55c0c461b6e0 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 -- 172.21.15.27:0/4006745916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b6e0 msgr2=0x55c0c461a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b6e0 0x55c0c461a5e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f15a8037040 tx=0x7f15a802e000 comp rx=0 tx=0).stop 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 -- 172.21.15.27:0/4006745916 shutdown_connections 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b6e0 0x55c0c461a5e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c45d9da0 0x55c0c461bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/4006745916 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 -- 172.21.15.27:0/4006745916 >> 172.21.15.27:0/4006745916 conn(0x55c0c45d6f10 msgr2=0x55c0c45da6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 -- 172.21.15.27:0/4006745916 shutdown_connections 2024-04-20T10:56:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 -- 172.21.15.27:0/4006745916 wait complete. 2024-04-20T10:56:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 -- start start 2024-04-20T10:56:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.349+0000 7f15bb1a5740 1 Processor -- start 2024-04-20T10:56:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 0x55c0c4617300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c45f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c0c4544140 con 0x55c0c461cce0 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0c453ee20 con 0x55c0c461b6e0 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x55c0c454cd20 con 0x55c0c461b300 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15bb1a5740 10 client.0 _ll_register_callbacks cb 0x55c0c45355c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15b6a6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c45f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15b6a6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c45f4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35354/0 (socket says 172.21.15.27:35354) 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.350+0000 7f15b6a6f640 1 -- 172.21.15.27:0/1649205082 learned_addr learned my addr 172.21.15.27:0/1649205082 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15b626e640 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15b5a6d640 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 0x55c0c4617300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15b6a6f640 1 -- 172.21.15.27:0/1649205082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 msgr2=0x55c0c45d5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45d5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15b6a6f640 1 -- 172.21.15.27:0/1649205082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 msgr2=0x55c0c4617300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 0x55c0c4617300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15b6a6f640 1 -- 172.21.15.27:0/1649205082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c42522d0 con 0x55c0c461cce0 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c45f4ed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009eb0 tx=0x7f15a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187c5c76b7443be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15917fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15917fa640 1 -- 172.21.15.27:0/1649205082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8039070 con 0x55c0c461cce0 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15bb1a5740 1 -- 172.21.15.27:0/1649205082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a803a040 con 0x55c0c461cce0 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15917fa640 1 -- 172.21.15.27:0/1649205082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f15a8003ba0 con 0x55c0c461cce0 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15917fa640 1 -- 172.21.15.27:0/1649205082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f15a8056050 con 0x55c0c461cce0 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15bb1a5740 10 client.4698 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15bb1a5740 1 -- 172.21.15.27:0/1649205082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c0c431c6b0 con 0x55c0c461cce0 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.351+0000 7f15917fa640 1 -- 172.21.15.27:0/1649205082 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a803a040 con 0x55c0c461cce0 2024-04-20T10:56:17.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15917fa640 1 -- 172.21.15.27:0/1649205082 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7f15a8059020 con 0x55c0c461cce0 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15bb1a5740 10 client.4698 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15bb1a5740 10 client.4698 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15bb1a5740 20 client.4698 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] conn(0x55c0c45f5a60 0x55c0c463e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15bb1a5740 4 client.4698 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15bb1a5740 1 -- 172.21.15.27:0/1649205082 --> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0c44c6510 con 0x55c0c45f5a60 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15b626e640 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] conn(0x55c0c45f5a60 0x55c0c463e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15917fa640 10 client.4698 ms_handle_connect on v2:172.21.15.27:6833/4195543922 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.352+0000 7f15b626e640 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] conn(0x55c0c45f5a60 0x55c0c463e860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e14f83c2d188090 server_cookie=1ba6ef456ab2189b in_seq=0 out_seq=0 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15917fa640 1 -- 172.21.15.27:0/1649205082 <== mds.1 v2:172.21.15.27:6833/4195543922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0c44c6510 con 0x55c0c45f5a60 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15917fa640 10 client.4698 handle_command_reply: tid=0 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 client.4698 shutdown 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 -- 172.21.15.27:0/1649205082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 msgr2=0x55c0c45f4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c45f4ed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009eb0 tx=0x7f15a8004ab0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 -- 172.21.15.27:0/1649205082 shutdown_connections 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] conn(0x55c0c45f5a60 0x55c0c463e860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c45f4ed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 0x55c0c4617300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15917fa640 0 client.4698 ms_handle_reset on v2:172.21.15.27:6833/4195543922 2024-04-20T10:56:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/1649205082 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45d5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 -- 172.21.15.27:0/1649205082 >> 172.21.15.27:0/1649205082 conn(0x55c0c45d9da0 msgr2=0x55c0c45d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 -- 172.21.15.27:0/1649205082 shutdown_connections 2024-04-20T10:56:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 1 -- 172.21.15.27:0/1649205082 wait complete. 2024-04-20T10:56:17.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.353+0000 7f15bb1a5740 20 client.4698 trim_cache size 0 max 16384 2024-04-20T10:56:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.371+0000 7f15bb1a5740 1 Processor -- start 2024-04-20T10:56:17.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.371+0000 7f15bb1a5740 1 -- start start 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b300 0x55c0c45d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b6e0 0x55c0c45d8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461cce0 0x55c0c4621b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01f2320 con 0x55c0c461b300 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0c44fc980 con 0x55c0c461cce0 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x55c0c461d5d0 con 0x55c0c461b6e0 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b626e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b300 0x55c0c45d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b626e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b300 0x55c0c45d7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35362/0 (socket says 172.21.15.27:35362) 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b626e640 1 -- 172.21.15.27:0/854087612 learned_addr learned my addr 172.21.15.27:0/854087612 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b5a6d640 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b6e0 0x55c0c45d8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461cce0 0x55c0c4621b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b626e640 1 -- 172.21.15.27:0/854087612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b6e0 msgr2=0x55c0c45d8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b626e640 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b6e0 0x55c0c45d8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b626e640 1 -- 172.21.15.27:0/854087612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461cce0 msgr2=0x55c0c4621b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b626e640 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461cce0 0x55c0c4621b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b626e640 1 -- 172.21.15.27:0/854087612 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c461f790 con 0x55c0c461b300 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.372+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461cce0 0x55c0c4621b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15b626e640 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b300 0x55c0c45d7e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55c0c461d780 tx=0x7f15a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57b37a3e31976b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15937fe640 1 -- 172.21.15.27:0/854087612 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4032070 con 0x55c0c461b300 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15937fe640 1 -- 172.21.15.27:0/854087612 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a402d3b0 con 0x55c0c461b300 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15937fe640 1 -- 172.21.15.27:0/854087612 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a402d760 con 0x55c0c461b300 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15bb1a5740 1 -- 172.21.15.27:0/854087612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b300 msgr2=0x55c0c45d7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b300 0x55c0c45d7e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55c0c461d780 tx=0x7f15a4019000 comp rx=0 tx=0).stop 2024-04-20T10:56:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15bb1a5740 1 -- 172.21.15.27:0/854087612 shutdown_connections 2024-04-20T10:56:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461cce0 0x55c0c4621b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b6e0 0x55c0c45d8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/854087612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461b300 0x55c0c45d7e40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15bb1a5740 1 -- 172.21.15.27:0/854087612 >> 172.21.15.27:0/854087612 conn(0x55c0c45d9da0 msgr2=0x55c0c4623a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15bb1a5740 1 -- 172.21.15.27:0/854087612 shutdown_connections 2024-04-20T10:56:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.373+0000 7f15bb1a5740 1 -- 172.21.15.27:0/854087612 wait complete. 2024-04-20T10:56:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.374+0000 7f15bb1a5740 1 -- start start 2024-04-20T10:56:17.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.374+0000 7f15bb1a5740 1 Processor -- start 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45d3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 0x55c0c469d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15bb1a5740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c46936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c0c45f58e0 con 0x55c0c461cce0 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0c4617b70 con 0x55c0c461b6e0 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15bb1a5740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x55c0c4633300 con 0x55c0c461b300 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15bb1a5740 10 client.0 _ll_register_callbacks cb 0x7f15a8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b6a6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c46936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b6a6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c46936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35372/0 (socket says 172.21.15.27:35372) 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b6a6f640 1 -- 172.21.15.27:0/3633527966 learned_addr learned my addr 172.21.15.27:0/3633527966 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b626e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45d3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b5a6d640 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 0x55c0c469d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b6a6f640 1 -- 172.21.15.27:0/3633527966 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 msgr2=0x55c0c45d3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45d3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b6a6f640 1 -- 172.21.15.27:0/3633527966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 msgr2=0x55c0c469d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 0x55c0c469d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b6a6f640 1 -- 172.21.15.27:0/3633527966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c46154f0 con 0x55c0c461cce0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15b6a6f640 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c46936d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15a802dd10 tx=0x7f15a8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5812f14327232dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.375+0000 7f15917fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15917fa640 1 -- 172.21.15.27:0/3633527966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8039070 con 0x55c0c461cce0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15917fa640 1 -- 172.21.15.27:0/3633527966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f15a80042a0 con 0x55c0c461cce0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15bb1a5740 1 -- 172.21.15.27:0/3633527966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a402d3b0 con 0x55c0c461cce0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15917fa640 1 -- 172.21.15.27:0/3633527966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f15a8043070 con 0x55c0c461cce0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15bb1a5740 10 client.4704 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15bb1a5740 1 -- 172.21.15.27:0/3633527966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55c0c469d550 con 0x55c0c461cce0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15917fa640 1 -- 172.21.15.27:0/3633527966 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a402d3b0 con 0x55c0c461cce0 2024-04-20T10:56:17.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15917fa640 1 -- 172.21.15.27:0/3633527966 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7f15a8059070 con 0x55c0c461cce0 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15bb1a5740 10 client.4704 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15bb1a5740 10 client.4704 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15bb1a5740 20 client.4704 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] conn(0x55c0c46a4240 0x55c0c46a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15bb1a5740 4 client.4704 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15bb1a5740 1 -- 172.21.15.27:0/3633527966 --> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] -- command(tid 0: {"prefix": "status"}) -- 0x55c0c454d530 con 0x55c0c46a4240 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.376+0000 7f15b626e640 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] conn(0x55c0c46a4240 0x55c0c46a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15917fa640 10 client.4704 ms_handle_connect on v2:172.21.15.27:6833/4195543922 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15b626e640 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] conn(0x55c0c46a4240 0x55c0c46a6630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9d0eea22084ee44 server_cookie=3932bbca7ec3a601 in_seq=0 out_seq=0 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15917fa640 1 -- 172.21.15.27:0/3633527966 <== mds.1 v2:172.21.15.27:6833/4195543922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55c0c454d530 con 0x55c0c46a4240 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15917fa640 10 client.4704 handle_command_reply: tid=0 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 client.4704 shutdown 2024-04-20T10:56:17.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 -- 172.21.15.27:0/3633527966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 msgr2=0x55c0c46936d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c46936d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15a802dd10 tx=0x7f15a8030000 comp rx=0 tx=0).stop 2024-04-20T10:56:17.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 -- 172.21.15.27:0/3633527966 shutdown_connections 2024-04-20T10:56:17.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.27:6833/4195543922,v1:172.21.15.27:6836/4195543922] conn(0x55c0c46a4240 0x55c0c46a6630 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c0c461cce0 0x55c0c46936d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0c461b6e0 0x55c0c469d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 --2- 172.21.15.27:0/3633527966 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55c0c461b300 0x55c0c45d3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15917fa640 0 client.4704 ms_handle_reset on v2:172.21.15.27:6833/4195543922 2024-04-20T10:56:17.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 -- 172.21.15.27:0/3633527966 >> 172.21.15.27:0/3633527966 conn(0x55c0c45d9da0 msgr2=0x55c0c4617230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 -- 172.21.15.27:0/3633527966 shutdown_connections 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 1 -- 172.21.15.27:0/3633527966 wait complete. 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.377+0000 7f15bb1a5740 20 client.4704 trim_cache size 0 max 16384 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "056b9857-269e-4bc1-b62f-8882e32cd3e3", 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 1, 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4278, 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 29.561093040999999, 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 17, 2024-04-20T10:56:17.384 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 20, 2024-04-20T10:56:17.385 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 20, 2024-04-20T10:56:17.385 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 42.06639045 2024-04-20T10:56:17.385 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T10:56:17.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 -- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f159406c8f0 msgr2=0x7f159406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f159406c8f0 0x7f159406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009560 tx=0x7f15a4009190 comp rx=0 tx=0).stop 2024-04-20T10:56:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 -- 172.21.15.27:0/4141903435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b014ff80 msgr2=0x7f15b0154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b014ff80 0x7f15b0154e70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00c440 tx=0x7f15ac00c900 comp rx=0 tx=0).stop 2024-04-20T10:56:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 -- 172.21.15.27:0/4141903435 shutdown_connections 2024-04-20T10:56:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f159406c8f0 0x7f159406edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f15b015d0e0 0x7f15b01553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15b014ff80 0x7f15b0154e70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 --2- 172.21.15.27:0/4141903435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15b00c5c20 0x7f15b014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 -- 172.21.15.27:0/4141903435 >> 172.21.15.27:0/4141903435 conn(0x7f15b006d5b0 msgr2=0x7f15b00c8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 -- 172.21.15.27:0/4141903435 shutdown_connections 2024-04-20T10:56:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.378+0000 7f15b84d2640 1 -- 172.21.15.27:0/4141903435 wait complete. 2024-04-20T10:56:17.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-20T10:56:17.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.490+0000 7fbe6c323640 1 Processor -- start 2024-04-20T10:56:17.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.490+0000 7fbe6c323640 1 -- start start 2024-04-20T10:56:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6c323640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbe6415ee50 0x7fbe6415c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6c323640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 0x7fbe6406cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6c323640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe6415f9a0 con 0x7fbe640ccbd0 2024-04-20T10:56:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6c323640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbe640cb870 con 0x7fbe6415ee50 2024-04-20T10:56:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6c323640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6406d490 con 0x7fbe6415f230 2024-04-20T10:56:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6a0bf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbe6415ee50 0x7fbe6415c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6a0bf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbe6415ee50 0x7fbe6415c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35376/0 (socket says 172.21.15.27:35376) 2024-04-20T10:56:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2092141260 learned_addr learned my addr 172.21.15.27:0/2092141260 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe698be640 1 --2- 172.21.15.27:0/2092141260 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 0x7fbe6406cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2092141260 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 msgr2=0x7fbe6406cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/2092141260 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 0x7fbe6406cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2092141260 >> v1:172.21.15.137:6789/0 conn(0x7fbe640ccbd0 legacy=0x7fbe64155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.491+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2092141260 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe640cbb30 con 0x7fbe6415ee50 2024-04-20T10:56:17.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.492+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/2092141260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe6415ee50 0x7fbe6415c180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe6002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724226b660f051b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.492+0000 7fbe690bd640 1 -- 172.21.15.27:0/2092141260 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6002f840 con 0x7fbe6415ee50 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.492+0000 7fbe690bd640 1 -- 172.21.15.27:0/2092141260 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe6002f9e0 con 0x7fbe6415ee50 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.492+0000 7fbe690bd640 1 -- 172.21.15.27:0/2092141260 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6002fdb0 con 0x7fbe6415ee50 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.492+0000 7fbe6c323640 1 -- 172.21.15.27:0/2092141260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe6415ee50 msgr2=0x7fbe6415c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.492+0000 7fbe6c323640 1 --2- 172.21.15.27:0/2092141260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe6415ee50 0x7fbe6415c180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe6002ba20 comp rx=0 tx=0).stop 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.493+0000 7fbe6c323640 1 -- 172.21.15.27:0/2092141260 shutdown_connections 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.493+0000 7fbe6c323640 1 --2- 172.21.15.27:0/2092141260 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 0x7fbe6406cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.493+0000 7fbe6c323640 1 --2- 172.21.15.27:0/2092141260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe6415ee50 0x7fbe6415c180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.493+0000 7fbe6c323640 1 -- 172.21.15.27:0/2092141260 >> 172.21.15.27:0/2092141260 conn(0x7fbe6406e010 msgr2=0x7fbe640d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.493+0000 7fbe6c323640 1 -- 172.21.15.27:0/2092141260 shutdown_connections 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.493+0000 7fbe6c323640 1 -- 172.21.15.27:0/2092141260 wait complete. 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.494+0000 7fbe6c323640 1 Processor -- start 2024-04-20T10:56:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.494+0000 7fbe6c323640 1 -- start start 2024-04-20T10:56:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.494+0000 7fbe6c323640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe640ccbd0 0x7fbe640c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.494+0000 7fbe6c323640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe6415ee50 0x7fbe6415b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.494+0000 7fbe6c323640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 0x7fbe640c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.494+0000 7fbe6c323640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe640cc3d0 con 0x7fbe640ccbd0 2024-04-20T10:56:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.494+0000 7fbe6c323640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe640cc0f0 con 0x7fbe6415ee50 2024-04-20T10:56:17.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.494+0000 7fbe6c323640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbe640cc6b0 con 0x7fbe6415f230 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.494+0000 7fbe6a0bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe640ccbd0 0x7fbe640c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6a0bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe640ccbd0 0x7fbe640c55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35378/0 (socket says 172.21.15.27:35378) 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/705689201 learned_addr learned my addr 172.21.15.27:0/705689201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe698be640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe6415ee50 0x7fbe6415b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 0x7fbe640c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 msgr2=0x7fbe640c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 0x7fbe640c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe6415ee50 msgr2=0x7fbe6415b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe6415ee50 0x7fbe6415b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/705689201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe641f5690 con 0x7fbe640ccbd0 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe640ccbd0 0x7fbe640c55b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe60000c00 tx=0x7fbe60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe5affd640 1 -- 172.21.15.27:0/705689201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60037bc0 con 0x7fbe640ccbd0 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe5affd640 1 -- 172.21.15.27:0/705689201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe60046070 con 0x7fbe640ccbd0 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6c323640 1 -- 172.21.15.27:0/705689201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe640c7a00 con 0x7fbe640ccbd0 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe5affd640 1 -- 172.21.15.27:0/705689201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60041400 con 0x7fbe640ccbd0 2024-04-20T10:56:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.495+0000 7fbe6c323640 1 -- 172.21.15.27:0/705689201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe641f4bb0 con 0x7fbe640ccbd0 2024-04-20T10:56:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.497+0000 7fbe5affd640 1 -- 172.21.15.27:0/705689201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe60040070 con 0x7fbe640ccbd0 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.497+0000 7fbe5affd640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe4806c8f0 0x7fbe4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.497+0000 7fbe5affd640 1 -- 172.21.15.27:0/705689201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe600ae200 con 0x7fbe640ccbd0 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.497+0000 7fbe698be640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe4806c8f0 0x7fbe4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.498+0000 7fbe698be640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe4806c8f0 0x7fbe4806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbe640ca650 tx=0x7fbe54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.498+0000 7fbe6efef740 1 Processor -- start 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.498+0000 7fbe6efef740 1 -- start start 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 0x556f8e6f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e6f6da0 0x556f8e738cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e7386e0 0x556f8e7375e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6efef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556f8e60a290 con 0x556f8e7386e0 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6efef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556f8e667a40 con 0x556f8e6f6da0 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6efef740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x556f8e622a30 con 0x556f8e738300 2024-04-20T10:56:17.501 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6a8c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e7386e0 0x556f8e7375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6a8c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e7386e0 0x556f8e7375e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35382/0 (socket says 172.21.15.27:35382) 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6a8c0640 1 -- 172.21.15.27:0/1087899428 learned_addr learned my addr 172.21.15.27:0/1087899428 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/1087899428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 0x556f8e6f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe698be640 1 --2- 172.21.15.27:0/1087899428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e6f6da0 0x556f8e738cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6a8c0640 1 -- 172.21.15.27:0/1087899428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 msgr2=0x556f8e6f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/1087899428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 0x556f8e6f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6a8c0640 1 -- 172.21.15.27:0/1087899428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e6f6da0 msgr2=0x556f8e738cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/1087899428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e6f6da0 0x556f8e738cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.499+0000 7fbe6a8c0640 1 -- 172.21.15.27:0/1087899428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8e615c10 con 0x556f8e7386e0 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/1087899428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e7386e0 0x556f8e7375e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c001c00 tx=0x7fbe5c0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee07a24bb645529e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe47fff640 1 -- 172.21.15.27:0/1087899428 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c019070 con 0x556f8e7386e0 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe47fff640 1 -- 172.21.15.27:0/1087899428 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5c003b80 con 0x556f8e7386e0 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe47fff640 1 -- 172.21.15.27:0/1087899428 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c00e800 con 0x556f8e7386e0 2024-04-20T10:56:17.502 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe6efef740 1 -- 172.21.15.27:0/1087899428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e7386e0 msgr2=0x556f8e7375e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe6efef740 1 --2- 172.21.15.27:0/1087899428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e7386e0 0x556f8e7375e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c001c00 tx=0x7fbe5c0020c0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe6efef740 1 -- 172.21.15.27:0/1087899428 shutdown_connections 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe6efef740 1 --2- 172.21.15.27:0/1087899428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e7386e0 0x556f8e7375e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe6efef740 1 --2- 172.21.15.27:0/1087899428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e6f6da0 0x556f8e738cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe6efef740 1 --2- 172.21.15.27:0/1087899428 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 0x556f8e6f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.500+0000 7fbe6efef740 1 -- 172.21.15.27:0/1087899428 >> 172.21.15.27:0/1087899428 conn(0x556f8e6f3f10 msgr2=0x556f8e6f76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.501+0000 7fbe6efef740 1 -- 172.21.15.27:0/1087899428 shutdown_connections 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.501+0000 7fbe6efef740 1 -- 172.21.15.27:0/1087899428 wait complete. 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.501+0000 7fbe6efef740 1 -- start start 2024-04-20T10:56:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.501+0000 7fbe6efef740 1 Processor -- start 2024-04-20T10:56:17.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 0x556f8e720080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e7386e0 0x556f8e6f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e73c850 0x556f8e74c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6efef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556f8e661140 con 0x556f8e73c850 2024-04-20T10:56:17.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6efef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556f8e65be20 con 0x556f8e7386e0 2024-04-20T10:56:17.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6efef740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x556f8e669d20 con 0x556f8e738300 2024-04-20T10:56:17.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6efef740 10 client.0 _ll_register_callbacks cb 0x556f8e6525c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:17.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6a8c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e73c850 0x556f8e74c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.508 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6a8c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e73c850 0x556f8e74c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35394/0 (socket says 172.21.15.27:35394) 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6a8c0640 1 -- 172.21.15.27:0/641070495 learned_addr learned my addr 172.21.15.27:0/641070495 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 0x556f8e720080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe698be640 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e7386e0 0x556f8e6f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6a8c0640 1 -- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 msgr2=0x556f8e720080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 0x556f8e720080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6a8c0640 1 -- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e7386e0 msgr2=0x556f8e6f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.502+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e7386e0 0x556f8e6f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6a8c0640 1 -- 172.21.15.27:0/641070495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8e36f2d0 con 0x556f8e73c850 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e73c850 0x556f8e74c280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c017040 tx=0x7fbe5c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43433f80db257d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe457fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe457fa640 1 -- 172.21.15.27:0/641070495 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c023060 con 0x556f8e73c850 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe457fa640 1 -- 172.21.15.27:0/641070495 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe5c00ea10 con 0x556f8e73c850 2024-04-20T10:56:17.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6efef740 1 -- 172.21.15.27:0/641070495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c003b80 con 0x556f8e73c850 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe457fa640 1 -- 172.21.15.27:0/641070495 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbe5c029050 con 0x556f8e73c850 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6efef740 10 client.4716 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6efef740 1 -- 172.21.15.27:0/641070495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x556f8e4396b0 con 0x556f8e73c850 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe457fa640 1 -- 172.21.15.27:0/641070495 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5c003b80 con 0x556f8e73c850 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe457fa640 1 -- 172.21.15.27:0/641070495 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7fbe5c02c020 con 0x556f8e73c850 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6efef740 10 client.4716 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6efef740 10 client.4716 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6efef740 20 client.4716 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6efef740 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] conn(0x556f8e734530 0x556f8e757140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6efef740 4 client.4716 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.503+0000 7fbe6efef740 1 -- 172.21.15.27:0/641070495 --> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f8e5e3510 con 0x556f8e734530 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.504+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] conn(0x556f8e734530 0x556f8e757140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.504+0000 7fbe457fa640 10 client.4716 ms_handle_connect on v2:172.21.15.137:6837/2929802235 2024-04-20T10:56:17.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.504+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] conn(0x556f8e734530 0x556f8e757140 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a49f35370e67a8 server_cookie=b10f4ead83b24976 in_seq=0 out_seq=0 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe457fa640 1 -- 172.21.15.27:0/641070495 <== mds.0 v2:172.21.15.137:6837/2929802235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f8e5e3510 con 0x556f8e734530 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe457fa640 10 client.4716 handle_command_reply: tid=0 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 client.4716 shutdown 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 -- 172.21.15.27:0/641070495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e73c850 msgr2=0x556f8e74c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e73c850 0x556f8e74c280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c017040 tx=0x7fbe5c001250 comp rx=0 tx=0).stop 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 -- 172.21.15.27:0/641070495 shutdown_connections 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e73c850 0x556f8e74c280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] conn(0x556f8e734530 0x556f8e757140 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e7386e0 0x556f8e6f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 --2- 172.21.15.27:0/641070495 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e738300 0x556f8e720080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe457fa640 0 client.4716 ms_handle_reset on v2:172.21.15.137:6837/2929802235 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 -- 172.21.15.27:0/641070495 >> 172.21.15.27:0/641070495 conn(0x556f8e6f6da0 msgr2=0x556f8e6f0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 -- 172.21.15.27:0/641070495 shutdown_connections 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 1 -- 172.21.15.27:0/641070495 wait complete. 2024-04-20T10:56:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.505+0000 7fbe6efef740 20 client.4716 trim_cache size 0 max 16384 2024-04-20T10:56:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.523+0000 7fbe6efef740 1 Processor -- start 2024-04-20T10:56:17.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.523+0000 7fbe6efef740 1 -- start start 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e6f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 0x556f8e6f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 0x556f8e74b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6efef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64155930 con 0x556f8e738300 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6efef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556f8e619980 con 0x556f8e73c850 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6efef740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x556f8e7488b0 con 0x556f8e7386e0 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a0bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e6f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a0bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e6f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35400/0 (socket says 172.21.15.27:35400) 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2413248794 learned_addr learned my addr 172.21.15.27:0/2413248794 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.526 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe698be640 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 0x556f8e6f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 0x556f8e74b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2413248794 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 msgr2=0x556f8e6f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 0x556f8e6f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2413248794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 msgr2=0x556f8e74b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 0x556f8e74b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2413248794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f8e721200 con 0x556f8e738300 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 0x556f8e74b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.524+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e6f72b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe600ae040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe7e42fc2cdde70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe477fe640 1 -- 172.21.15.27:0/2413248794 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60099070 con 0x556f8e738300 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe477fe640 1 -- 172.21.15.27:0/2413248794 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe6007bea0 con 0x556f8e738300 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe477fe640 1 -- 172.21.15.27:0/2413248794 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60080250 con 0x556f8e738300 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe6efef740 1 -- 172.21.15.27:0/2413248794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 msgr2=0x556f8e6f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e6f72b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe600ae040 comp rx=0 tx=0).stop 2024-04-20T10:56:17.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe6efef740 1 -- 172.21.15.27:0/2413248794 shutdown_connections 2024-04-20T10:56:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 0x556f8e74b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 0x556f8e6f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2413248794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e6f72b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe6efef740 1 -- 172.21.15.27:0/2413248794 >> 172.21.15.27:0/2413248794 conn(0x556f8e6f6da0 msgr2=0x556f8e6f0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe6efef740 1 -- 172.21.15.27:0/2413248794 shutdown_connections 2024-04-20T10:56:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.525+0000 7fbe6efef740 1 -- 172.21.15.27:0/2413248794 wait complete. 2024-04-20T10:56:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.526+0000 7fbe6efef740 1 -- start start 2024-04-20T10:56:17.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.526+0000 7fbe6efef740 1 Processor -- start 2024-04-20T10:56:17.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e7b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 0x556f8e75b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 0x556f8e75b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556f8e71d460 con 0x556f8e738300 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556f8e7207e0 con 0x556f8e73c850 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x556f8e7349c0 con 0x556f8e7386e0 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 10 client.0 _ll_register_callbacks cb 0x7fbe5c017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a0bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e7b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a0bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e7b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35406/0 (socket says 172.21.15.27:35406) 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2133553857 learned_addr learned my addr 172.21.15.27:0/2133553857 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe698be640 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 0x556f8e75b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a8c0640 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 0x556f8e75b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 msgr2=0x556f8e75b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 0x556f8e75b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 msgr2=0x556f8e75b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 0x556f8e75b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a0bf640 1 -- 172.21.15.27:0/2133553857 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f8e73f4a0 con 0x556f8e738300 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe698be640 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 0x556f8e75b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6a0bf640 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e7b60a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbe6007be30 tx=0x7fbe60092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d212f4026c5b3832 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe457fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe457fa640 1 -- 172.21.15.27:0/2133553857 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60099070 con 0x556f8e738300 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe457fa640 1 -- 172.21.15.27:0/2133553857 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe60079f40 con 0x556f8e738300 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 1 -- 172.21.15.27:0/2133553857 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe6007bea0 con 0x556f8e738300 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe457fa640 1 -- 172.21.15.27:0/2133553857 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fbe60084b10 con 0x556f8e738300 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 10 client.4722 fetch_fsmap learned FSMap version 17 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.527+0000 7fbe6efef740 1 -- 172.21.15.27:0/2133553857 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x556f8e7b3490 con 0x556f8e738300 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe457fa640 1 -- 172.21.15.27:0/2133553857 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe6007bea0 con 0x556f8e738300 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe457fa640 1 -- 172.21.15.27:0/2133553857 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 4785+0+0 (secure 0 0 0) 0x7fbe60078510 con 0x556f8e738300 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe6efef740 10 client.4722 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe6efef740 10 client.4722 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe6efef740 20 client.4722 populate_metadata read hostname 'smithi027' 2024-04-20T10:56:17.533 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] conn(0x556f8e6f2850 0x556f8e7bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe6efef740 4 client.4722 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe6efef740 1 -- 172.21.15.27:0/2133553857 --> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] -- command(tid 0: {"prefix": "status"}) -- 0x556f8e6f4670 con 0x556f8e6f2850 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe698be640 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] conn(0x556f8e6f2850 0x556f8e7bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe698be640 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] conn(0x556f8e6f2850 0x556f8e7bf220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f277491720ff13b server_cookie=b585047189bc24b1 in_seq=0 out_seq=0 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe457fa640 10 client.4722 ms_handle_connect on v2:172.21.15.137:6837/2929802235 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe457fa640 1 -- 172.21.15.27:0/2133553857 <== mds.0 v2:172.21.15.137:6837/2929802235 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x556f8e6f4670 con 0x556f8e6f2850 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe457fa640 10 client.4722 handle_command_reply: tid=0 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.528+0000 7fbe6efef740 1 client.4722 shutdown 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 -- 172.21.15.27:0/2133553857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 msgr2=0x556f8e7b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e7b60a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbe6007be30 tx=0x7fbe60092040 comp rx=0 tx=0).stop 2024-04-20T10:56:17.534 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 -- 172.21.15.27:0/2133553857 shutdown_connections 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:6837/2929802235,v1:172.21.15.137:6839/2929802235] conn(0x556f8e6f2850 0x556f8e7bf220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f8e73c850 0x556f8e75b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x556f8e7386e0 0x556f8e75b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe457fa640 0 client.4722 ms_handle_reset on v2:172.21.15.137:6837/2929802235 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 --2- 172.21.15.27:0/2133553857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556f8e738300 0x556f8e7b60a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 -- 172.21.15.27:0/2133553857 >> 172.21.15.27:0/2133553857 conn(0x556f8e6f6da0 msgr2=0x556f8e6f60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 -- 172.21.15.27:0/2133553857 shutdown_connections 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 1 -- 172.21.15.27:0/2133553857 wait complete. 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6efef740 20 client.4722 trim_cache size 0 max 16384 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T10:56:17.535 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "056b9857-269e-4bc1-b62f-8882e32cd3e3", 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4293, 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 34.750885480999997, 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 17, 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 20, 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 20, 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 42.200016302000002 2024-04-20T10:56:17.536 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 -- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe4806c8f0 msgr2=0x7fbe4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe4806c8f0 0x7fbe4806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbe640ca650 tx=0x7fbe54008040 comp rx=0 tx=0).stop 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 -- 172.21.15.27:0/705689201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe640ccbd0 msgr2=0x7fbe640c55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe640ccbd0 0x7fbe640c55b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe60000c00 tx=0x7fbe60004730 comp rx=0 tx=0).stop 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 -- 172.21.15.27:0/705689201 shutdown_connections 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbe4806c8f0 0x7fbe4806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbe6415f230 0x7fbe640c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe6415ee50 0x7fbe6415b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 --2- 172.21.15.27:0/705689201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe640ccbd0 0x7fbe640c55b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.529+0000 7fbe6c323640 1 -- 172.21.15.27:0/705689201 >> 172.21.15.27:0/705689201 conn(0x7fbe6406e010 msgr2=0x7fbe640cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.530+0000 7fbe6c323640 1 -- 172.21.15.27:0/705689201 shutdown_connections 2024-04-20T10:56:17.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.530+0000 7fbe6c323640 1 -- 172.21.15.27:0/705689201 wait complete. 2024-04-20T10:56:17.540 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T10:56:17.550 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T10:56:17.550 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T10:56:17.551 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T10:56:17.551 INFO:teuthology.run_tasks:Running task kclient... 2024-04-20T10:56:17.572 INFO:tasks.kclient:Mounting kernel clients... 2024-04-20T10:56:17.572 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-20T10:56:17.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:56:17.572 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-20T10:56:17.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:56:17.588 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-04-20T10:56:17.655 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2024-04-20T10:56:17.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:56:17.656 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-20T10:56:17.656 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:56:17.657 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:56:17.657 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:56:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.797+0000 7f777679d640 1 Processor -- start 2024-04-20T10:56:17.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.797+0000 7f777679d640 1 -- start start 2024-04-20T10:56:17.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.798+0000 7f777679d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7770152260 0x7f77700719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.798+0000 7f777679d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7770152640 0x7f77700c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.798+0000 7f777679d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f777015ba80 con 0x7f77700cd6f0 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.798+0000 7f777679d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f777015bd60 con 0x7f7770152640 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.798+0000 7f777679d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f777015c040 con 0x7f7770152260 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.798+0000 7f776f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7770152640 0x7f77700c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.798+0000 7f776f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7770152640 0x7f77700c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35418/0 (socket says 172.21.15.27:35418) 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.798+0000 7f776f7fe640 1 -- 172.21.15.27:0/2651111776 learned_addr learned my addr 172.21.15.27:0/2651111776 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.798+0000 7f776ffff640 1 --2- 172.21.15.27:0/2651111776 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7770152260 0x7f77700719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.799+0000 7f776f7fe640 1 -- 172.21.15.27:0/2651111776 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7770152260 msgr2=0x7f77700719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.799+0000 7f776f7fe640 1 --2- 172.21.15.27:0/2651111776 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7770152260 0x7f77700719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.799+0000 7f776f7fe640 1 -- 172.21.15.27:0/2651111776 >> v1:172.21.15.137:6789/0 conn(0x7f77700cd6f0 legacy=0x7f7770158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.799+0000 7f776f7fe640 1 -- 172.21.15.27:0/2651111776 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777015c320 con 0x7f7770152640 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.799+0000 7f776f7fe640 1 --2- 172.21.15.27:0/2651111776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 0x7f77700c86e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7760009870 tx=0x7f776002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a68c764e0540797 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.799+0000 7f776effd640 1 -- 172.21.15.27:0/2651111776 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776002f840 con 0x7f7770152640 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.799+0000 7f776effd640 1 -- 172.21.15.27:0/2651111776 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776002f9e0 con 0x7f7770152640 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.799+0000 7f776effd640 1 -- 172.21.15.27:0/2651111776 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776002fdb0 con 0x7f7770152640 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.800+0000 7f777679d640 1 -- 172.21.15.27:0/2651111776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 msgr2=0x7f77700c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.800+0000 7f777679d640 1 --2- 172.21.15.27:0/2651111776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 0x7f77700c86e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7760009870 tx=0x7f776002ba20 comp rx=0 tx=0).stop 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.800+0000 7f777679d640 1 -- 172.21.15.27:0/2651111776 shutdown_connections 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.800+0000 7f777679d640 1 --2- 172.21.15.27:0/2651111776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 0x7f77700c86e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.800+0000 7f777679d640 1 --2- 172.21.15.27:0/2651111776 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7770152260 0x7f77700719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.800+0000 7f777679d640 1 -- 172.21.15.27:0/2651111776 >> 172.21.15.27:0/2651111776 conn(0x7f77700ccbd0 msgr2=0x7f77700ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:17.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.800+0000 7f777679d640 1 -- 172.21.15.27:0/2651111776 shutdown_connections 2024-04-20T10:56:17.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.800+0000 7f777679d640 1 -- 172.21.15.27:0/2651111776 wait complete. 2024-04-20T10:56:17.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.801+0000 7f777679d640 1 Processor -- start 2024-04-20T10:56:17.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f777679d640 1 -- start start 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f777679d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77700cd6f0 0x7f77701589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f777679d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7770152260 0x7f7770148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f777679d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 0x7f77701494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f777679d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f777015c600 con 0x7f7770152640 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f777679d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7770074260 con 0x7f7770152260 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f777679d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f777015b720 con 0x7f77700cd6f0 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f7774d3a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 0x7f77701494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f7774d3a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 0x7f77701494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35426/0 (socket says 172.21.15.27:35426) 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f776ffff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77700cd6f0 0x7f77701589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f7774d3a640 1 -- 172.21.15.27:0/3558606909 learned_addr learned my addr 172.21.15.27:0/3558606909 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:17.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.802+0000 7f776f7fe640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7770152260 0x7f7770148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f7774d3a640 1 -- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77700cd6f0 msgr2=0x7f77701589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f7774d3a640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77700cd6f0 0x7f77701589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f7774d3a640 1 -- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7770152260 msgr2=0x7f7770148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f7774d3a640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7770152260 0x7f7770148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f7774d3a640 1 -- 172.21.15.27:0/3558606909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777015b300 con 0x7f7770152640 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f7774d3a640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 0x7f77701494c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f776800c6e0 tx=0x7f776800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f776cff9640 1 -- 172.21.15.27:0/3558606909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768003b70 con 0x7f7770152640 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f776cff9640 1 -- 172.21.15.27:0/3558606909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7768003d30 con 0x7f7770152640 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77701f2890 con 0x7f7770152640 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f776cff9640 1 -- 172.21.15.27:0/3558606909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768005310 con 0x7f7770152640 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.803+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77701f2e50 con 0x7f7770152640 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.804+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777004fcf0 con 0x7f7770152640 2024-04-20T10:56:17.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.804+0000 7f776cff9640 1 -- 172.21.15.27:0/3558606909 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77680054b0 con 0x7f7770152640 2024-04-20T10:56:17.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.805+0000 7f776cff9640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f775006c8f0 0x7f775006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:17.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.805+0000 7f776cff9640 1 -- 172.21.15.27:0/3558606909 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f776808b7d0 con 0x7f7770152640 2024-04-20T10:56:17.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.805+0000 7f776ffff640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f775006c8f0 0x7f775006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:17.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.806+0000 7f776ffff640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f775006c8f0 0x7f775006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7758009aa0 tx=0x7f7758009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:17.810+0000 7f776cff9640 1 -- 172.21.15.27:0/3558606909 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7768059640 con 0x7f7770152640 2024-04-20T10:56:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.038+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f77700c8fe0 con 0x7f7770152640 2024-04-20T10:56:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.038+0000 7f776cff9640 1 -- 172.21.15.27:0/3558606909 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f776805d940 con 0x7f7770152640 2024-04-20T10:56:18.038 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T10:56:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f775006c8f0 msgr2=0x7f775006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f775006c8f0 0x7f775006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7758009aa0 tx=0x7f7758009190 comp rx=0 tx=0).stop 2024-04-20T10:56:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 msgr2=0x7f77701494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 0x7f77701494c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f776800c6e0 tx=0x7f776800cba0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 shutdown_connections 2024-04-20T10:56:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f775006c8f0 0x7f775006edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7770152640 0x7f77701494c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7770152260 0x7f7770148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 --2- 172.21.15.27:0/3558606909 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f77700cd6f0 0x7f77701589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 >> 172.21.15.27:0/3558606909 conn(0x7f77700ccbd0 msgr2=0x7f77700cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 shutdown_connections 2024-04-20T10:56:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18.040+0000 7f777679d640 1 -- 172.21.15.27:0/3558606909 wait complete. 2024-04-20T10:56:18.050 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:56:18.051 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:56:18.051 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2024-04-20T10:56:18.051 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T10:56:18.051 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:56:18.051 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:56:18.051 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:56:18.051 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:56:18.051 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:56:18.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:56:18.051 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2024-04-20T10:56:18.066 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:56:18.066 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2024-04-20T10:56:18.067 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-20T10:56:18.068 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42805sec preferred_lft 42805sec 2024-04-20T10:56:18.068 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2024-04-20T10:56:18.068 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:56:18.068 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:56:18.068 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2024-04-20T10:56:18.068 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2024-04-20T10:56:18.068 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T10:56:18.068 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:56:18.068 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-20T10:56:18.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2024-04-20T10:56:18.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2024-04-20T10:56:18.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2024-04-20T10:56:18.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T10:56:18.069 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-20T10:56:18.155 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:56:18.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:56:18.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:56:18.247 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:56:18.271 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-04-20T10:56:18.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:56:18.273 DEBUG:teuthology.orchestra.run.smithi027:> route 2024-04-20T10:56:18.344 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2024-04-20T10:56:18.344 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:56:18.344 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T10:56:18.344 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T10:56:18.344 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:56:18.344 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:56:18.344 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-20T10:56:18.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T10:56:18.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T10:56:18.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T10:56:18.345 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-20T10:56:18.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:56:18.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:56:18.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:56:18.631 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-20T10:56:18.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:56:18.646 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2024-04-20T10:56:18.702 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:56:18.702 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-20T10:56:18.702 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:56:18.702 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T10:56:18.702 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-20T10:56:18.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:56:18.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:56:18.819 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T10:56:18.820 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:56:18.820 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-20T10:56:18.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T10:56:18.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T10:56:18.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T10:56:18.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T10:56:18.820 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T10:56:18.820 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-20T10:56:18.870 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:56:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:56:18.949 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:56:18.949 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2024-04-20T10:56:18.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2024-04-20T10:56:18.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T10:56:18.949 DEBUG:teuthology.orchestra.run.smithi027:> ') 2024-04-20T10:56:18.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:56:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:56:19.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:56:19.027 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T10:56:19.041 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:56:19.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:56:19.042 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:56:19.093 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-20T10:56:19.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:56:19.093 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-20T10:56:19.184 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-20T10:56:19.184 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-20T10:56:19.293 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-20T10:56:19.294 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-20T10:56:19.294 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: resolved to: "172.21.15.27:3300,172.21.15.137:3300,172.21.15.137:3301" 2024-04-20T10:56:19.294 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: trying mount with new device syntax: 0@056b9857-269e-4bc1-b62f-8882e32cd3e3.cephfs=/ 2024-04-20T10:56:19.294 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.27:3300/172.21.15.137:3300/172.21.15.137:3301" will pass to kernel 2024-04-20T10:56:19.294 INFO:teuthology.orchestra.run.smithi027.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-20T10:56:19.295 INFO:teuthology.orchestra.run.smithi027.stdout: You just mounted a file system that supports labels which does not 2024-04-20T10:56:19.295 INFO:teuthology.orchestra.run.smithi027.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-20T10:56:19.295 INFO:teuthology.orchestra.run.smithi027.stdout: applications will generate AVC messages and not be allowed access to 2024-04-20T10:56:19.295 INFO:teuthology.orchestra.run.smithi027.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-20T10:56:19.297 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-20T10:56:19.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:56:19.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:56:19.325 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T10:56:19.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T10:56:19.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:56:19.429 INFO:teuthology.orchestra.run.smithi027.stdout:client4731 2024-04-20T10:56:19.430 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:56:19.430 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:56:19.446 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T10:56:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T10:56:19.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:56:19.550 INFO:teuthology.orchestra.run.smithi027.stdout:client4731 2024-04-20T10:56:19.550 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T10:56:19.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T10:56:19.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:56:19.613 INFO:teuthology.orchestra.run.smithi027.stdout:client4731 2024-04-20T10:56:19.613 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/056b9857-269e-4bc1-b62f-8882e32cd3e3.client4731/status') 2024-04-20T10:56:19.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:56:19.677 INFO:teuthology.orchestra.run.smithi027.stdout:instance: client.4731 (3)192.168.0.1:0/1057522381 2024-04-20T10:56:19.677 INFO:teuthology.orchestra.run.smithi027.stdout:blocklisted: false 2024-04-20T10:56:19.678 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records in 2024-04-20T10:56:19.678 INFO:teuthology.orchestra.run.smithi027.stderr:0+1 records out 2024-04-20T10:56:19.678 INFO:teuthology.orchestra.run.smithi027.stderr:69 bytes copied, 2.3933e-05 s, 2.9 MB/s 2024-04-20T10:56:19.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:56:19.685 INFO:teuthology.run_tasks:Running task mds_thrash... 2024-04-20T10:56:19.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:56:19.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.798+0000 7f9a0f025640 1 Processor -- start 2024-04-20T10:56:19.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.798+0000 7f9a0f025640 1 -- start start 2024-04-20T10:56:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0f025640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080b54f0 0x7f9a080b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0f025640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9a080d0c90 0x7f9a080d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0f025640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a080b5e70 con 0x7f9a08155950 2024-04-20T10:56:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0f025640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f9a0815b9c0 con 0x7f9a080d0c90 2024-04-20T10:56:19.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0f025640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0815bca0 con 0x7f9a080b54f0 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0d5c2640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9a080d0c90 0x7f9a080d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f99fffff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080b54f0 0x7f9a080b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0d5c2640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9a080d0c90 0x7f9a080d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:60692/0 (socket says 172.21.15.27:60692) 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0d5c2640 1 -- 172.21.15.27:0/2525244890 learned_addr learned my addr 172.21.15.27:0/2525244890 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0cdc1640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f9a08155950 0x7f9a080cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:32990/0 (socket says 172.21.15.27:32990) 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0d5c2640 1 -- 172.21.15.27:0/2525244890 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080b54f0 msgr2=0x7f9a080b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0d5c2640 1 --2- 172.21.15.27:0/2525244890 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080b54f0 0x7f9a080b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0d5c2640 1 -- 172.21.15.27:0/2525244890 >> v1:172.21.15.137:6789/0 conn(0x7f9a08155950 legacy=0x7f9a080cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f99ff7fe640 1 -- 172.21.15.27:0/2525244890 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367798168 0 0) 0x7f9a080b5e70 con 0x7f9a08155950 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f9a0d5c2640 1 -- 172.21.15.27:0/2525244890 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0815bf80 con 0x7f9a080d0c90 2024-04-20T10:56:19.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.799+0000 7f99fffff640 1 --2- 172.21.15.27:0/2525244890 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080b54f0 0x7f9a080b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:56:19.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.800+0000 7f9a0d5c2640 1 --2- 172.21.15.27:0/2525244890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a080d0c90 0x7f9a080d1100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009e50 tx=0x7f9a0002bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e636aa5104f86cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.800+0000 7f99ff7fe640 1 -- 172.21.15.27:0/2525244890 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0002cb90 con 0x7f9a080d0c90 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.800+0000 7f99ff7fe640 1 -- 172.21.15.27:0/2525244890 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0002cd30 con 0x7f9a080d0c90 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.800+0000 7f99ff7fe640 1 -- 172.21.15.27:0/2525244890 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00034610 con 0x7f9a080d0c90 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.801+0000 7f9a0f025640 1 -- 172.21.15.27:0/2525244890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a080d0c90 msgr2=0x7f9a080d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.801+0000 7f9a0f025640 1 --2- 172.21.15.27:0/2525244890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a080d0c90 0x7f9a080d1100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009e50 tx=0x7f9a0002bff0 comp rx=0 tx=0).stop 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.801+0000 7f9a0f025640 1 -- 172.21.15.27:0/2525244890 shutdown_connections 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.801+0000 7f9a0f025640 1 --2- 172.21.15.27:0/2525244890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a080d0c90 0x7f9a080d1100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.801+0000 7f9a0f025640 1 --2- 172.21.15.27:0/2525244890 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080b54f0 0x7f9a080b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.801+0000 7f9a0f025640 1 -- 172.21.15.27:0/2525244890 >> 172.21.15.27:0/2525244890 conn(0x7f9a080676c0 msgr2=0x7f9a08067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.801+0000 7f9a0f025640 1 -- 172.21.15.27:0/2525244890 shutdown_connections 2024-04-20T10:56:19.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.801+0000 7f9a0f025640 1 -- 172.21.15.27:0/2525244890 wait complete. 2024-04-20T10:56:19.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.802+0000 7f9a0f025640 1 Processor -- start 2024-04-20T10:56:19.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.803+0000 7f9a0f025640 1 -- start start 2024-04-20T10:56:19.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.803+0000 7f9a0f025640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a080b54f0 0x7f9a08152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:19.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.803+0000 7f9a0f025640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080d0c90 0x7f9a0815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.803+0000 7f9a0f025640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a08155950 0x7f9a0815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.803+0000 7f9a0f025640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a080d1640 con 0x7f9a08155950 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.803+0000 7f9a0f025640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0815c540 con 0x7f9a080b54f0 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.803+0000 7f9a0f025640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0815fb90 con 0x7f9a080d0c90 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0d5c2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a08155950 0x7f9a0815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0d5c2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a08155950 0x7f9a0815e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35442/0 (socket says 172.21.15.27:35442) 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0d5c2640 1 -- 172.21.15.27:0/421291859 learned_addr learned my addr 172.21.15.27:0/421291859 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0cdc1640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a080b54f0 0x7f9a08152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f99fffff640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080d0c90 0x7f9a0815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0d5c2640 1 -- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080d0c90 msgr2=0x7f9a0815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0d5c2640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080d0c90 0x7f9a0815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0d5c2640 1 -- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a080b54f0 msgr2=0x7f9a08152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0d5c2640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a080b54f0 0x7f9a08152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0d5c2640 1 -- 172.21.15.27:0/421291859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0815ecc0 con 0x7f9a08155950 2024-04-20T10:56:19.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0d5c2640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a08155950 0x7f9a0815e780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9a00036040 tx=0x7f9a00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f99fd7fa640 1 -- 172.21.15.27:0/421291859 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00043030 con 0x7f9a08155950 2024-04-20T10:56:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f99fd7fa640 1 -- 172.21.15.27:0/421291859 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0003e450 con 0x7f9a08155950 2024-04-20T10:56:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a08152040 con 0x7f9a08155950 2024-04-20T10:56:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.804+0000 7f99fd7fa640 1 -- 172.21.15.27:0/421291859 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00009510 con 0x7f9a08155950 2024-04-20T10:56:19.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.805+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a08152300 con 0x7f9a08155950 2024-04-20T10:56:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.805+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a081f2340 con 0x7f9a08155950 2024-04-20T10:56:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.806+0000 7f99fd7fa640 1 -- 172.21.15.27:0/421291859 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a00046070 con 0x7f9a08155950 2024-04-20T10:56:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.806+0000 7f99fd7fa640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99f406c8f0 0x7f99f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.807+0000 7f99fd7fa640 1 -- 172.21.15.27:0/421291859 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9a000adc20 con 0x7f9a08155950 2024-04-20T10:56:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.807+0000 7f9a0cdc1640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99f406c8f0 0x7f99f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:19.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.807+0000 7f9a0cdc1640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99f406c8f0 0x7f99f406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9a04000f60 tx=0x7f9a04007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:19.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.811+0000 7f99fd7fa640 1 -- 172.21.15.27:0/421291859 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9a0004d070 con 0x7f9a08155950 2024-04-20T10:56:19.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.996+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a080d0810 con 0x7f9a08155950 2024-04-20T10:56:19.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.997+0000 7f99fd7fa640 1 -- 172.21.15.27:0/421291859 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16569 (secure 0 0 0) 0x7f9a080d0810 con 0x7f9a08155950 2024-04-20T10:56:19.997 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:19.997 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:43.787038+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:56:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99f406c8f0 msgr2=0x7f99f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99f406c8f0 0x7f99f406edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9a04000f60 tx=0x7f9a04007d50 comp rx=0 tx=0).stop 2024-04-20T10:56:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a08155950 msgr2=0x7f9a0815e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a08155950 0x7f9a0815e780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9a00036040 tx=0x7f9a00004a40 comp rx=0 tx=0).stop 2024-04-20T10:56:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 shutdown_connections 2024-04-20T10:56:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99f406c8f0 0x7f99f406edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a08155950 0x7f9a0815e780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a080d0c90 0x7f9a0815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:19.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 --2- 172.21.15.27:0/421291859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a080b54f0 0x7f9a08152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 >> 172.21.15.27:0/421291859 conn(0x7f9a080676c0 msgr2=0x7f9a080b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:20.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 shutdown_connections 2024-04-20T10:56:20.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:19.999+0000 7f9a0f025640 1 -- 172.21.15.27:0/421291859 wait complete. 2024-04-20T10:56:20.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:55:29.717620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:55:39.832461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:55:41.103975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:56:20.010 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:56:20.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.114+0000 7f97e65d0640 1 Processor -- start 2024-04-20T10:56:20.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.114+0000 7f97e65d0640 1 -- start start 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97e65d0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e015f230 0x7f97e015c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97e65d0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97e015f9a0 con 0x7f97e00ccbd0 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97e65d0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97e00cb870 con 0x7f97e015ee50 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97e65d0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f97e00cbb30 con 0x7f97e015f230 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97e4b6d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f97e00ccbd0 0x7f97e0155410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37358/0 (socket says 172.21.15.27:37358) 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97e4b6d640 1 -- 172.21.15.27:0/12498025 learned_addr learned my addr 172.21.15.27:0/12498025 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97df7fe640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e015f230 0x7f97e015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3469378897 0 0) 0x7f97d8001490 con 0x7f97e00ccbd0 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97c8003520 con 0x7f97e00ccbd0 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1570041336 0 0) 0x7f97c8003520 con 0x7f97e00ccbd0 2024-04-20T10:56:20.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e015f230 msgr2=0x7f97e015c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97deffd640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e015f230 0x7f97e015c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 >> v1:172.21.15.137:6789/0 conn(0x7f97e015ee50 legacy=0x7f97e006cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e00cbe10 con 0x7f97e00ccbd0 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f97d8001bc0 con 0x7f97e00ccbd0 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.115+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 >> v1:172.21.15.27:6789/0 conn(0x7f97e00ccbd0 legacy=0x7f97e0155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97deffd640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e015f230 0x7f97c80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97deffd640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c8003f60 0x7f97c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97deffd640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8004d40 0x7f97c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97e00cbb30 con 0x7f97c8003f60 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f97c800bb30 con 0x7f97c8004d40 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f97c800bd30 con 0x7f97e015f230 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97df7fe640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c8003f60 0x7f97c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97dffff640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e015f230 0x7f97c80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97e4b6d640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8004d40 0x7f97c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97df7fe640 1 -- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e015f230 msgr2=0x7f97c80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97df7fe640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e015f230 0x7f97c80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97df7fe640 1 -- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8004d40 msgr2=0x7f97c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97df7fe640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8004d40 0x7f97c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97df7fe640 1 -- 172.21.15.27:0/12498025 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c8003520 con 0x7f97c8003f60 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.116+0000 7f97e4b6d640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8004d40 0x7f97c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97df7fe640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c8003f60 0x7f97c8004800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009aa0 tx=0x7f97c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b12d0e454afa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0039070 con 0x7f97c8003f60 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d002c6b0 con 0x7f97c8003f60 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97deffd640 1 -- 172.21.15.27:0/12498025 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0009670 con 0x7f97c8003f60 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97e65d0640 1 -- 172.21.15.27:0/12498025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c8003f60 msgr2=0x7f97c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97e65d0640 1 --2- 172.21.15.27:0/12498025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97c8003f60 0x7f97c8004800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009aa0 tx=0x7f97c8005830 comp rx=0 tx=0).stop 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97e4b6d640 1 -- 172.21.15.27:0/12498025 reap_dead start 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97e65d0640 1 -- 172.21.15.27:0/12498025 shutdown_connections 2024-04-20T10:56:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97e65d0640 1 -- 172.21.15.27:0/12498025 >> 172.21.15.27:0/12498025 conn(0x7f97e006e010 msgr2=0x7f97e00d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:20.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97e65d0640 1 -- 172.21.15.27:0/12498025 shutdown_connections 2024-04-20T10:56:20.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.117+0000 7f97e65d0640 1 -- 172.21.15.27:0/12498025 wait complete. 2024-04-20T10:56:20.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.119+0000 7f97e65d0640 1 Processor -- start 2024-04-20T10:56:20.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.119+0000 7f97e65d0640 1 -- start start 2024-04-20T10:56:20.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.119+0000 7f97e65d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8003f60 0x7f97e01f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.119+0000 7f97e65d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e00c9ad0 0x7f97e015b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.119+0000 7f97e65d0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e00cf880 0x7f97e00cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.119+0000 7f97e65d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97e00cc6b0 con 0x7f97e00c9ad0 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.119+0000 7f97e65d0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f97e00cc3d0 con 0x7f97c8003f60 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.119+0000 7f97e65d0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f97e015cae0 con 0x7f97e00cf880 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e00c9ad0 0x7f97e015b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e00c9ad0 0x7f97e015b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35458/0 (socket says 172.21.15.27:35458) 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97df7fe640 1 -- 172.21.15.27:0/1494038609 learned_addr learned my addr 172.21.15.27:0/1494038609 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97dffff640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8003f60 0x7f97e01f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97e4b6d640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e00cf880 0x7f97e00cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97df7fe640 1 -- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e00cf880 msgr2=0x7f97e00cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97df7fe640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e00cf880 0x7f97e00cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97df7fe640 1 -- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8003f60 msgr2=0x7f97e01f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97df7fe640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8003f60 0x7f97e01f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.124 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97df7fe640 1 -- 172.21.15.27:0/1494038609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e00d0250 con 0x7f97e00c9ad0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97df7fe640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e00c9ad0 0x7f97e015b920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f97d0037040 tx=0x7f97d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97dcff9640 1 -- 172.21.15.27:0/1494038609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0044020 con 0x7f97e00c9ad0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97dcff9640 1 -- 172.21.15.27:0/1494038609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d00346c0 con 0x7f97e00c9ad0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e01f5740 con 0x7f97e00c9ad0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97dcff9640 1 -- 172.21.15.27:0/1494038609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0039040 con 0x7f97e00c9ad0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.120+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e00ca240 con 0x7f97e00c9ad0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.121+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e004fcf0 con 0x7f97e00c9ad0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.122+0000 7f97dcff9640 1 -- 172.21.15.27:0/1494038609 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97d0047020 con 0x7f97e00c9ad0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.122+0000 7f97dcff9640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f97cc06c920 0x7f97cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.122+0000 7f97dcff9640 1 -- 172.21.15.27:0/1494038609 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f97d00ad7e0 con 0x7f97e00c9ad0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.122+0000 7f97dffff640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f97cc06c920 0x7f97cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.125 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.123+0000 7f97dffff640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f97cc06c920 0x7f97cc06ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97d40017e0 tx=0x7f97d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.126 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.127+0000 7f97dcff9640 1 -- 172.21.15.27:0/1494038609 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f97d004e070 con 0x7f97e00c9ad0 2024-04-20T10:56:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.312+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97e0155f80 con 0x7f97e00c9ad0 2024-04-20T10:56:20.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.312+0000 7f97dcff9640 1 -- 172.21.15.27:0/1494038609 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f97d007b5c0 con 0x7f97e00c9ad0 2024-04-20T10:56:20.312 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-20T10:56:20.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f97cc06c920 msgr2=0x7f97cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f97cc06c920 0x7f97cc06ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97d40017e0 tx=0x7f97d4009040 comp rx=0 tx=0).stop 2024-04-20T10:56:20.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e00c9ad0 msgr2=0x7f97e015b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e00c9ad0 0x7f97e015b920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f97d0037040 tx=0x7f97d0004730 comp rx=0 tx=0).stop 2024-04-20T10:56:20.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 shutdown_connections 2024-04-20T10:56:20.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f97cc06c920 0x7f97cc06ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f97e00cf880 0x7f97e00cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e00c9ad0 0x7f97e015b920 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 --2- 172.21.15.27:0/1494038609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97c8003f60 0x7f97e01f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 >> 172.21.15.27:0/1494038609 conn(0x7f97e006e010 msgr2=0x7f97e0152df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:20.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 shutdown_connections 2024-04-20T10:56:20.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.314+0000 7f97e65d0640 1 -- 172.21.15.27:0/1494038609 wait complete. 2024-04-20T10:56:20.325 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:56:20.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.429+0000 7f438578f640 1 Processor -- start 2024-04-20T10:56:20.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.429+0000 7f438578f640 1 -- start start 2024-04-20T10:56:20.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f438578f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f438015ee50 0x7f438015c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f438578f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015f230 0x7f438006cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f438578f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f438015f9a0 con 0x7f43800ccbd0 2024-04-20T10:56:20.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f438578f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f43800cb870 con 0x7f438015ee50 2024-04-20T10:56:20.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f438578f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f438006d490 con 0x7f438015f230 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437e7fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015f230 0x7f438006cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437effd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f438015ee50 0x7f438015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437e7fc640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015f230 0x7f438006cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:57852/0 (socket says 172.21.15.27:57852) 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437effd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f438015ee50 0x7f438015c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:60708/0 (socket says 172.21.15.27:60708) 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437e7fc640 1 -- 172.21.15.27:0/918160516 learned_addr learned my addr 172.21.15.27:0/918160516 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437f7fe640 1 --1- 172.21.15.27:0/918160516 >> v1:172.21.15.137:6789/0 conn(0x7f43800ccbd0 0x7f4380155410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:32998/0 (socket says 172.21.15.27:32998) 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437e7fc640 1 -- 172.21.15.27:0/918160516 >> v2:172.21.15.137:3300/0 conn(0x7f438015ee50 msgr2=0x7f438015c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437e7fc640 1 --2- 172.21.15.27:0/918160516 >> v2:172.21.15.137:3300/0 conn(0x7f438015ee50 0x7f438015c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437e7fc640 1 -- 172.21.15.27:0/918160516 >> v1:172.21.15.137:6789/0 conn(0x7f43800ccbd0 legacy=0x7f4380155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.430+0000 7f437e7fc640 1 -- 172.21.15.27:0/918160516 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43800cbb30 con 0x7f438015f230 2024-04-20T10:56:20.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.431+0000 7f437e7fc640 1 --2- 172.21.15.27:0/918160516 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015f230 0x7f438006cf50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4374009870 tx=0x7f437402ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=964655c66e3bc491 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.431+0000 7f437dffb640 1 -- 172.21.15.27:0/918160516 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437402f840 con 0x7f438015f230 2024-04-20T10:56:20.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.431+0000 7f437dffb640 1 -- 172.21.15.27:0/918160516 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f437402f9e0 con 0x7f438015f230 2024-04-20T10:56:20.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.431+0000 7f437dffb640 1 -- 172.21.15.27:0/918160516 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437402fdb0 con 0x7f438015f230 2024-04-20T10:56:20.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.432+0000 7f438578f640 1 -- 172.21.15.27:0/918160516 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015f230 msgr2=0x7f438006cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.432+0000 7f438578f640 1 --2- 172.21.15.27:0/918160516 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015f230 0x7f438006cf50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4374009870 tx=0x7f437402ba20 comp rx=0 tx=0).stop 2024-04-20T10:56:20.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.432+0000 7f438578f640 1 -- 172.21.15.27:0/918160516 shutdown_connections 2024-04-20T10:56:20.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.432+0000 7f438578f640 1 --2- 172.21.15.27:0/918160516 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015f230 0x7f438006cf50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.432+0000 7f438578f640 1 --2- 172.21.15.27:0/918160516 >> v2:172.21.15.137:3300/0 conn(0x7f438015ee50 0x7f438015c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.432+0000 7f438578f640 1 -- 172.21.15.27:0/918160516 >> 172.21.15.27:0/918160516 conn(0x7f438006e010 msgr2=0x7f43800d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:20.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.432+0000 7f438578f640 1 -- 172.21.15.27:0/918160516 shutdown_connections 2024-04-20T10:56:20.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.432+0000 7f438578f640 1 -- 172.21.15.27:0/918160516 wait complete. 2024-04-20T10:56:20.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.433+0000 7f438578f640 1 Processor -- start 2024-04-20T10:56:20.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.433+0000 7f438578f640 1 -- start start 2024-04-20T10:56:20.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f438578f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43800ccbd0 0x7f43801f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f438578f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015ee50 0x7f438015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f438578f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438015f230 0x7f4380156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f438578f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43800cc3d0 con 0x7f43800ccbd0 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f438578f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43800cc0f0 con 0x7f438015f230 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f438578f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f43800cc6b0 con 0x7f438015ee50 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f437effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43800ccbd0 0x7f43801f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f437effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43800ccbd0 0x7f43801f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35468/0 (socket says 172.21.15.27:35468) 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f437effd640 1 -- 172.21.15.27:0/4167113541 learned_addr learned my addr 172.21.15.27:0/4167113541 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f437f7fe640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438015f230 0x7f4380156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f437e7fc640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015ee50 0x7f438015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f437effd640 1 -- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015ee50 msgr2=0x7f438015a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.434+0000 7f437effd640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015ee50 0x7f438015a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.435+0000 7f437effd640 1 -- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438015f230 msgr2=0x7f4380156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.435+0000 7f437effd640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438015f230 0x7f4380156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.435+0000 7f437effd640 1 -- 172.21.15.27:0/4167113541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43801f5920 con 0x7f43800ccbd0 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.435+0000 7f437effd640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43800ccbd0 0x7f43801f53e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f437800e9b0 tx=0x7f437800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.436 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.435+0000 7f43677fe640 1 -- 172.21.15.27:0/4167113541 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378017070 con 0x7f43800ccbd0 2024-04-20T10:56:20.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.435+0000 7f43677fe640 1 -- 172.21.15.27:0/4167113541 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f437800c9a0 con 0x7f43800ccbd0 2024-04-20T10:56:20.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.435+0000 7f43677fe640 1 -- 172.21.15.27:0/4167113541 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437800cd90 con 0x7f43800ccbd0 2024-04-20T10:56:20.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.435+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43800d1940 con 0x7f43800ccbd0 2024-04-20T10:56:20.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.435+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43801587d0 con 0x7f43800ccbd0 2024-04-20T10:56:20.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.436+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43800688e0 con 0x7f43800ccbd0 2024-04-20T10:56:20.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.436+0000 7f43677fe640 1 -- 172.21.15.27:0/4167113541 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f437801a070 con 0x7f43800ccbd0 2024-04-20T10:56:20.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.437+0000 7f43677fe640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f436c06c8f0 0x7f436c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.437+0000 7f43677fe640 1 -- 172.21.15.27:0/4167113541 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f437808baa0 con 0x7f43800ccbd0 2024-04-20T10:56:20.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.437+0000 7f437e7fc640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f436c06c8f0 0x7f436c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.437+0000 7f437e7fc640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f436c06c8f0 0x7f436c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4374036000 tx=0x7f4374008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.441+0000 7f43677fe640 1 -- 172.21.15.27:0/4167113541 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4378059880 con 0x7f43800ccbd0 2024-04-20T10:56:20.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.626+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f438015f670 con 0x7f43800ccbd0 2024-04-20T10:56:20.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.627+0000 7f43677fe640 1 -- 172.21.15.27:0/4167113541 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f438015f670 con 0x7f43800ccbd0 2024-04-20T10:56:20.627 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f436c06c8f0 msgr2=0x7f436c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f436c06c8f0 0x7f436c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4374036000 tx=0x7f4374008030 comp rx=0 tx=0).stop 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43800ccbd0 msgr2=0x7f43801f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43800ccbd0 0x7f43801f53e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f437800e9b0 tx=0x7f437800ee70 comp rx=0 tx=0).stop 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 shutdown_connections 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f436c06c8f0 0x7f436c06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f438015f230 0x7f4380156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f438015ee50 0x7f438015a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 --2- 172.21.15.27:0/4167113541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43800ccbd0 0x7f43801f53e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 >> 172.21.15.27:0/4167113541 conn(0x7f438006e010 msgr2=0x7f438015cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 shutdown_connections 2024-04-20T10:56:20.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.629+0000 7f438578f640 1 -- 172.21.15.27:0/4167113541 wait complete. 2024-04-20T10:56:20.639 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:56:20.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.743+0000 7f2864f85640 1 Processor -- start 2024-04-20T10:56:20.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.743+0000 7f2864f85640 1 -- start start 2024-04-20T10:56:20.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f2864f85640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015f230 0x7f286015c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f2864f85640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f286015f9a0 con 0x7f28600ccbd0 2024-04-20T10:56:20.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f2864f85640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28600cb870 con 0x7f286015ee50 2024-04-20T10:56:20.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f2864f85640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f28600cbb30 con 0x7f286015f230 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f285ed9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f28600ccbd0 0x7f2860155410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37366/0 (socket says 172.21.15.27:37366) 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f285ed9d640 1 -- 172.21.15.27:0/590402654 learned_addr learned my addr 172.21.15.27:0/590402654 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f2855d9b640 1 --2- 172.21.15.27:0/590402654 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015f230 0x7f286015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f285dd9b640 1 -- 172.21.15.27:0/590402654 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344216820 0 0) 0x7f2858001490 con 0x7f28600ccbd0 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f285dd9b640 1 -- 172.21.15.27:0/590402654 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f284c003520 con 0x7f28600ccbd0 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f285dd9b640 1 -- 172.21.15.27:0/590402654 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 963325835 0 0) 0x7f28600cb870 con 0x7f286015ee50 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f285dd9b640 1 -- 172.21.15.27:0/590402654 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2858001490 con 0x7f286015ee50 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f2855d9b640 1 -- 172.21.15.27:0/590402654 >> v1:172.21.15.137:6789/0 conn(0x7f286015ee50 legacy=0x7f286006cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f2855d9b640 1 -- 172.21.15.27:0/590402654 >> v1:172.21.15.27:6789/0 conn(0x7f28600ccbd0 legacy=0x7f2860155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.744+0000 7f2855d9b640 1 -- 172.21.15.27:0/590402654 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28600cbe10 con 0x7f286015f230 2024-04-20T10:56:20.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.745+0000 7f2855d9b640 1 --2- 172.21.15.27:0/590402654 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015f230 0x7f286015c180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f28480097d0 tx=0x7f284802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59fd532ea4cc7b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.745+0000 7f285dd9b640 1 -- 172.21.15.27:0/590402654 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848034070 con 0x7f286015f230 2024-04-20T10:56:20.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.745+0000 7f285dd9b640 1 -- 172.21.15.27:0/590402654 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2848030880 con 0x7f286015f230 2024-04-20T10:56:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.745+0000 7f285dd9b640 1 -- 172.21.15.27:0/590402654 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848030c50 con 0x7f286015f230 2024-04-20T10:56:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.746+0000 7f2864f85640 1 -- 172.21.15.27:0/590402654 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015f230 msgr2=0x7f286015c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.746+0000 7f2864f85640 1 --2- 172.21.15.27:0/590402654 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015f230 0x7f286015c180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f28480097d0 tx=0x7f284802b9c0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.746+0000 7f2864f85640 1 -- 172.21.15.27:0/590402654 shutdown_connections 2024-04-20T10:56:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.746+0000 7f2864f85640 1 --2- 172.21.15.27:0/590402654 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015f230 0x7f286015c180 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.746+0000 7f2864f85640 1 -- 172.21.15.27:0/590402654 >> 172.21.15.27:0/590402654 conn(0x7f286006e010 msgr2=0x7f28600d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.746+0000 7f2864f85640 1 -- 172.21.15.27:0/590402654 shutdown_connections 2024-04-20T10:56:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.746+0000 7f2864f85640 1 -- 172.21.15.27:0/590402654 wait complete. 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.747+0000 7f2864f85640 1 Processor -- start 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f2864f85640 1 -- start start 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f2864f85640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28600ccbd0 0x7f286014ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f2864f85640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015ee50 0x7f28600c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f2864f85640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f286015f230 0x7f2860151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f2864f85640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28600cc6b0 con 0x7f28600ccbd0 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f2864f85640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28600cc3d0 con 0x7f286015f230 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f2864f85640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f286015cae0 con 0x7f286015ee50 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f285e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28600ccbd0 0x7f286014ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f285e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28600ccbd0 0x7f286014ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35474/0 (socket says 172.21.15.27:35474) 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.748+0000 7f285e59c640 1 -- 172.21.15.27:0/1814338942 learned_addr learned my addr 172.21.15.27:0/1814338942 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:20.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f285ed9d640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f286015f230 0x7f2860151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f2855d9b640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015ee50 0x7f28600c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f285e59c640 1 -- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015ee50 msgr2=0x7f28600c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f285e59c640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015ee50 0x7f28600c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f285e59c640 1 -- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f286015f230 msgr2=0x7f2860151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f285e59c640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f286015f230 0x7f2860151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f285e59c640 1 -- 172.21.15.27:0/1814338942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286015a530 con 0x7f28600ccbd0 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f285ed9d640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f286015f230 0x7f2860151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f2855d9b640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015ee50 0x7f28600c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f285e59c640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28600ccbd0 0x7f286014ea40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f285000c500 tx=0x7f285000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f28577fe640 1 -- 172.21.15.27:0/1814338942 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28500052d0 con 0x7f28600ccbd0 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f28577fe640 1 -- 172.21.15.27:0/1814338942 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2850005490 con 0x7f28600ccbd0 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f28577fe640 1 -- 172.21.15.27:0/1814338942 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850012870 con 0x7f28600ccbd0 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601f0d40 con 0x7f28600ccbd0 2024-04-20T10:56:20.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.749+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28600c83f0 con 0x7f28600ccbd0 2024-04-20T10:56:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.750+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28600688e0 con 0x7f28600ccbd0 2024-04-20T10:56:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.751+0000 7f28577fe640 1 -- 172.21.15.27:0/1814338942 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2850012a10 con 0x7f28600ccbd0 2024-04-20T10:56:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.751+0000 7f28577fe640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f284006c920 0x7f284006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:20.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.751+0000 7f28577fe640 1 -- 172.21.15.27:0/1814338942 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f285008b7f0 con 0x7f28600ccbd0 2024-04-20T10:56:20.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.751+0000 7f2855d9b640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f284006c920 0x7f284006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:20.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.752+0000 7f2855d9b640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f284006c920 0x7f284006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2848036000 tx=0x7f284803b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:20.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.755+0000 7f28577fe640 1 -- 172.21.15.27:0/1814338942 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f28601f0d40 con 0x7f28600ccbd0 2024-04-20T10:56:20.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.941+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f28600d10f0 con 0x7f28600ccbd0 2024-04-20T10:56:20.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.941+0000 7f28577fe640 1 -- 172.21.15.27:0/1814338942 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f286015a530 con 0x7f28600ccbd0 2024-04-20T10:56:20.941 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-20T10:56:20.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f284006c920 msgr2=0x7f284006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f284006c920 0x7f284006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2848036000 tx=0x7f284803b040 comp rx=0 tx=0).stop 2024-04-20T10:56:20.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28600ccbd0 msgr2=0x7f286014ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:20.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28600ccbd0 0x7f286014ea40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f285000c500 tx=0x7f285000c9c0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 shutdown_connections 2024-04-20T10:56:20.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f284006c920 0x7f284006ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f286015f230 0x7f2860151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f286015ee50 0x7f28600c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 --2- 172.21.15.27:0/1814338942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28600ccbd0 0x7f286014ea40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:20.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 >> 172.21.15.27:0/1814338942 conn(0x7f286006e010 msgr2=0x7f28600d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:20.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 shutdown_connections 2024-04-20T10:56:20.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:20.943+0000 7f2864f85640 1 -- 172.21.15.27:0/1814338942 wait complete. 2024-04-20T10:56:20.955 INFO:tasks.mds_thrash:mds thrasher using random seed: 1713610580 2024-04-20T10:56:20.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:56:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.055+0000 7f23b406d640 1 Processor -- start 2024-04-20T10:56:21.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.056+0000 7f23b406d640 1 -- start start 2024-04-20T10:56:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.056+0000 7f23b406d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f23ac13bab0 0x7f23ac070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.056+0000 7f23b406d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac15cf70 0x7f23ac13e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.056+0000 7f23b406d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23ac13f2c0 con 0x7f23ac071140 2024-04-20T10:56:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.056+0000 7f23b406d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f23ac0cc160 con 0x7f23ac13bab0 2024-04-20T10:56:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.056+0000 7f23b406d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f23ac159360 con 0x7f23ac15cf70 2024-04-20T10:56:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.056+0000 7f23b1608640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f23ac071140 0x7f23ac15c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37374/0 (socket says 172.21.15.27:37374) 2024-04-20T10:56:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.056+0000 7f23b1608640 1 -- 172.21.15.27:0/1614815641 learned_addr learned my addr 172.21.15.27:0/1614815641 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b260a640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac15cf70 0x7f23ac13e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b1e09640 1 --2- 172.21.15.27:0/1614815641 >> v2:172.21.15.137:3300/0 conn(0x7f23ac13bab0 0x7f23ac070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228744127 0 0) 0x7f23ac13f2c0 con 0x7f23ac071140 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2394003520 con 0x7f23ac071140 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 768816896 0 0) 0x7f2394003520 con 0x7f23ac071140 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac15cf70 msgr2=0x7f23ac13e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac15cf70 0x7f23ac13e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 >> v2:172.21.15.137:3300/0 conn(0x7f23ac13bab0 msgr2=0x7f23ac070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 --2- 172.21.15.27:0/1614815641 >> v2:172.21.15.137:3300/0 conn(0x7f23ac13bab0 0x7f23ac070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac159640 con 0x7f23ac071140 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f23a0001670 con 0x7f23ac071140 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 >> v1:172.21.15.27:6789/0 conn(0x7f23ac071140 legacy=0x7f23ac15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23940039b0 0x7f2394003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394004800 0x7f2394004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac15cf70 0x7f23940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac0cc160 con 0x7f23940039b0 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac159360 con 0x7f2394004800 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f239400b9c0 con 0x7f23ac15cf70 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b1e09640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23940039b0 0x7f2394003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b1608640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394004800 0x7f2394004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b260a640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac15cf70 0x7f23940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b1e09640 1 -- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac15cf70 msgr2=0x7f23940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b1e09640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac15cf70 0x7f23940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b1e09640 1 -- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394004800 msgr2=0x7f2394004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b1e09640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394004800 0x7f2394004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.057+0000 7f23b1e09640 1 -- 172.21.15.27:0/1614815641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2394003520 con 0x7f23940039b0 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.058+0000 7f23b1e09640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23940039b0 0x7f2394003dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f239c009fc0 tx=0x7f2394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfadf4856c6bd0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.058+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c039070 con 0x7f23940039b0 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.058+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c02cc60 con 0x7f23940039b0 2024-04-20T10:56:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.058+0000 7f23b0e07640 1 -- 172.21.15.27:0/1614815641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c009c10 con 0x7f23940039b0 2024-04-20T10:56:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.059+0000 7f23b406d640 1 -- 172.21.15.27:0/1614815641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23940039b0 msgr2=0x7f2394003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.059+0000 7f23b406d640 1 --2- 172.21.15.27:0/1614815641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23940039b0 0x7f2394003dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f239c009fc0 tx=0x7f2394005830 comp rx=0 tx=0).stop 2024-04-20T10:56:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.059+0000 7f23b260a640 1 -- 172.21.15.27:0/1614815641 reap_dead start 2024-04-20T10:56:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.059+0000 7f23b406d640 1 -- 172.21.15.27:0/1614815641 shutdown_connections 2024-04-20T10:56:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.059+0000 7f23b406d640 1 -- 172.21.15.27:0/1614815641 >> 172.21.15.27:0/1614815641 conn(0x7f23ac06c070 msgr2=0x7f23ac0676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.059+0000 7f23b406d640 1 -- 172.21.15.27:0/1614815641 shutdown_connections 2024-04-20T10:56:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.059+0000 7f23b406d640 1 -- 172.21.15.27:0/1614815641 wait complete. 2024-04-20T10:56:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.060+0000 7f23b406d640 1 Processor -- start 2024-04-20T10:56:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.060+0000 7f23b406d640 1 -- start start 2024-04-20T10:56:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b406d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23940039b0 0x7f23ac1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b406d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ac156bc0 0x7f23ac1e2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b406d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac1e4cc0 0x7f23ac152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b406d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac13be90 con 0x7f23ac156bc0 2024-04-20T10:56:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b406d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac159c00 con 0x7f23940039b0 2024-04-20T10:56:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b406d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f23ac13c010 con 0x7f23ac1e4cc0 2024-04-20T10:56:21.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1608640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ac156bc0 0x7f23ac1e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1608640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ac156bc0 0x7f23ac1e2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35502/0 (socket says 172.21.15.27:35502) 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1608640 1 -- 172.21.15.27:0/3085006594 learned_addr learned my addr 172.21.15.27:0/3085006594 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1e09640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23940039b0 0x7f23ac1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b260a640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac1e4cc0 0x7f23ac152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1608640 1 -- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac1e4cc0 msgr2=0x7f23ac152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1608640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac1e4cc0 0x7f23ac152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1608640 1 -- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23940039b0 msgr2=0x7f23ac1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1608640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23940039b0 0x7f23ac1571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1608640 1 -- 172.21.15.27:0/3085006594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac157840 con 0x7f23ac156bc0 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b260a640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac1e4cc0 0x7f23ac152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.061+0000 7f23b1608640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ac156bc0 0x7f23ac1e2110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23a0007370 tx=0x7f23a0007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.062+0000 7f23aa7fc640 1 -- 172.21.15.27:0/3085006594 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a001a070 con 0x7f23ac156bc0 2024-04-20T10:56:21.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.062+0000 7f23aa7fc640 1 -- 172.21.15.27:0/3085006594 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a000f940 con 0x7f23ac156bc0 2024-04-20T10:56:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.062+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac157b20 con 0x7f23ac156bc0 2024-04-20T10:56:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.062+0000 7f23aa7fc640 1 -- 172.21.15.27:0/3085006594 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a000fd10 con 0x7f23ac156bc0 2024-04-20T10:56:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.062+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac1f20b0 con 0x7f23ac156bc0 2024-04-20T10:56:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.063+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ac04fcf0 con 0x7f23ac156bc0 2024-04-20T10:56:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.063+0000 7f23aa7fc640 1 -- 172.21.15.27:0/3085006594 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23a001d070 con 0x7f23ac156bc0 2024-04-20T10:56:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.064+0000 7f23aa7fc640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f239806c920 0x7f239806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.064+0000 7f23aa7fc640 1 -- 172.21.15.27:0/3085006594 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f23a008b510 con 0x7f23ac156bc0 2024-04-20T10:56:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.064+0000 7f23b1e09640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f239806c920 0x7f239806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.064+0000 7f23b1e09640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f239806c920 0x7f239806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f239c036000 tx=0x7f239c02e000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:21.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.068+0000 7f23aa7fc640 1 -- 172.21.15.27:0/3085006594 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f23a00599d0 con 0x7f23ac156bc0 2024-04-20T10:56:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.253+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23ac0cc500 con 0x7f23ac156bc0 2024-04-20T10:56:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.254+0000 7f23aa7fc640 1 -- 172.21.15.27:0/3085006594 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16569 (secure 0 0 0) 0x7f23a00592d0 con 0x7f23ac156bc0 2024-04-20T10:56:21.254 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:21.254 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:55:43.787038+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:56:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f239806c920 msgr2=0x7f239806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f239806c920 0x7f239806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f239c036000 tx=0x7f239c02e000 comp rx=0 tx=0).stop 2024-04-20T10:56:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ac156bc0 msgr2=0x7f23ac1e2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ac156bc0 0x7f23ac1e2110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23a0007370 tx=0x7f23a0007830 comp rx=0 tx=0).stop 2024-04-20T10:56:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 shutdown_connections 2024-04-20T10:56:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f239806c920 0x7f239806ede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f23ac1e4cc0 0x7f23ac152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23ac156bc0 0x7f23ac1e2110 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 --2- 172.21.15.27:0/3085006594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23940039b0 0x7f23ac1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 >> 172.21.15.27:0/3085006594 conn(0x7f23ac06c070 msgr2=0x7f23ac15c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 shutdown_connections 2024-04-20T10:56:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.256+0000 7f23b406d640 1 -- 172.21.15.27:0/3085006594 wait complete. 2024-04-20T10:56:21.267 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:55:29.717620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:55:39.832461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:55:41.103975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:56:21.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:56:21.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.371+0000 7f93b2d96640 1 Processor -- start 2024-04-20T10:56:21.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.371+0000 7f93b2d96640 1 -- start start 2024-04-20T10:56:21.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b2d96640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93ac1536b0 0x7f93ac159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b2d96640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f93ac0c8840 0x7f93ac06e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b2d96640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ac15a3a0 con 0x7f93ac06eb40 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b2d96640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f93ac0c8490 con 0x7f93ac1536b0 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b2d96640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f93ac0c90f0 con 0x7f93ac0c8840 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b0b32640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93ac1536b0 0x7f93ac159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b0b32640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93ac1536b0 0x7f93ac159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35506/0 (socket says 172.21.15.27:35506) 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b0b32640 1 -- 172.21.15.27:0/1680626068 learned_addr learned my addr 172.21.15.27:0/1680626068 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93abfff640 1 --2- 172.21.15.27:0/1680626068 >> v2:172.21.15.137:3300/0 conn(0x7f93ac0c8840 0x7f93ac06e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b0b32640 1 -- 172.21.15.27:0/1680626068 >> v2:172.21.15.137:3300/0 conn(0x7f93ac0c8840 msgr2=0x7f93ac06e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b0b32640 1 --2- 172.21.15.27:0/1680626068 >> v2:172.21.15.137:3300/0 conn(0x7f93ac0c8840 0x7f93ac06e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b0b32640 1 -- 172.21.15.27:0/1680626068 >> v1:172.21.15.137:6789/0 conn(0x7f93ac06eb40 legacy=0x7f93ac0c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.372+0000 7f93b0b32640 1 -- 172.21.15.27:0/1680626068 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac0c76d0 con 0x7f93ac1536b0 2024-04-20T10:56:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.373+0000 7f93b0b32640 1 --2- 172.21.15.27:0/1680626068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac1536b0 0x7f93ac159e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f93ac074140 tx=0x7f93a002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77eda9ad488897ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:21.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.373+0000 7f93ab7fe640 1 -- 172.21.15.27:0/1680626068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0034070 con 0x7f93ac1536b0 2024-04-20T10:56:21.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.373+0000 7f93ab7fe640 1 -- 172.21.15.27:0/1680626068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a0030860 con 0x7f93ac1536b0 2024-04-20T10:56:21.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.373+0000 7f93ab7fe640 1 -- 172.21.15.27:0/1680626068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0030c30 con 0x7f93ac1536b0 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.373+0000 7f93b2d96640 1 -- 172.21.15.27:0/1680626068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac1536b0 msgr2=0x7f93ac159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.373+0000 7f93b2d96640 1 --2- 172.21.15.27:0/1680626068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac1536b0 0x7f93ac159e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f93ac074140 tx=0x7f93a002b9a0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.374+0000 7f93b2d96640 1 -- 172.21.15.27:0/1680626068 shutdown_connections 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.374+0000 7f93b2d96640 1 --2- 172.21.15.27:0/1680626068 >> v2:172.21.15.137:3300/0 conn(0x7f93ac0c8840 0x7f93ac06e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.374+0000 7f93b2d96640 1 --2- 172.21.15.27:0/1680626068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac1536b0 0x7f93ac159e60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.374+0000 7f93b2d96640 1 -- 172.21.15.27:0/1680626068 >> 172.21.15.27:0/1680626068 conn(0x7f93ac0c9630 msgr2=0x7f93ac0c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.374+0000 7f93b2d96640 1 -- 172.21.15.27:0/1680626068 shutdown_connections 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.374+0000 7f93b2d96640 1 -- 172.21.15.27:0/1680626068 wait complete. 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.375+0000 7f93b2d96640 1 Processor -- start 2024-04-20T10:56:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.375+0000 7f93b2d96640 1 -- start start 2024-04-20T10:56:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b2d96640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac06eb40 0x7f93ac0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b2d96640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93ac0c8840 0x7f93ac14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b2d96640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ac1536b0 0x7f93ac1f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b2d96640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0c7f70 con 0x7f93ac06eb40 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b2d96640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac0c7c90 con 0x7f93ac1536b0 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b2d96640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f93ac153a90 con 0x7f93ac0c8840 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b0b32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac06eb40 0x7f93ac0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b0b32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac06eb40 0x7f93ac0ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35516/0 (socket says 172.21.15.27:35516) 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b0b32640 1 -- 172.21.15.27:0/2291107128 learned_addr learned my addr 172.21.15.27:0/2291107128 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b1333640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ac1536b0 0x7f93ac1f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93abfff640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93ac0c8840 0x7f93ac14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b0b32640 1 -- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93ac0c8840 msgr2=0x7f93ac14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b0b32640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93ac0c8840 0x7f93ac14d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b0b32640 1 -- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ac1536b0 msgr2=0x7f93ac1f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b0b32640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ac1536b0 0x7f93ac1f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b0b32640 1 -- 172.21.15.27:0/2291107128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac0720e0 con 0x7f93ac06eb40 2024-04-20T10:56:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93abfff640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93ac0c8840 0x7f93ac14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b0b32640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac06eb40 0x7f93ac0ca930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f93a0000c00 tx=0x7f93a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93a97fa640 1 -- 172.21.15.27:0/2291107128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0034040 con 0x7f93ac06eb40 2024-04-20T10:56:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93a97fa640 1 -- 172.21.15.27:0/2291107128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a0039d90 con 0x7f93ac06eb40 2024-04-20T10:56:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ac072380 con 0x7f93ac06eb40 2024-04-20T10:56:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93a97fa640 1 -- 172.21.15.27:0/2291107128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0043610 con 0x7f93ac06eb40 2024-04-20T10:56:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.376+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ac072920 con 0x7f93ac06eb40 2024-04-20T10:56:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.377+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ac06c970 con 0x7f93ac06eb40 2024-04-20T10:56:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.378+0000 7f93a97fa640 1 -- 172.21.15.27:0/2291107128 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93a004a060 con 0x7f93ac06eb40 2024-04-20T10:56:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.378+0000 7f93a97fa640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939006c8f0 0x7f939006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.378+0000 7f93a97fa640 1 -- 172.21.15.27:0/2291107128 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f93a00add00 con 0x7f93ac06eb40 2024-04-20T10:56:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.379+0000 7f93abfff640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939006c8f0 0x7f939006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.379+0000 7f93abfff640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939006c8f0 0x7f939006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f93ac1500e0 tx=0x7f9398008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.383+0000 7f93a97fa640 1 -- 172.21.15.27:0/2291107128 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f93ac06c970 con 0x7f93ac06eb40 2024-04-20T10:56:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.567+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f93ac06f730 con 0x7f93ac06eb40 2024-04-20T10:56:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.568+0000 7f93a97fa640 1 -- 172.21.15.27:0/2291107128 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f93a003e070 con 0x7f93ac06eb40 2024-04-20T10:56:21.568 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939006c8f0 msgr2=0x7f939006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939006c8f0 0x7f939006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f93ac1500e0 tx=0x7f9398008040 comp rx=0 tx=0).stop 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac06eb40 msgr2=0x7f93ac0ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac06eb40 0x7f93ac0ca930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f93a0000c00 tx=0x7f93a00046c0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 shutdown_connections 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939006c8f0 0x7f939006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ac1536b0 0x7f93ac1f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93ac0c8840 0x7f93ac14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 --2- 172.21.15.27:0/2291107128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93ac06eb40 0x7f93ac0ca930 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 >> 172.21.15.27:0/2291107128 conn(0x7f93ac0c9630 msgr2=0x7f93ac06c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 shutdown_connections 2024-04-20T10:56:21.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.570+0000 7f93b2d96640 1 -- 172.21.15.27:0/2291107128 wait complete. 2024-04-20T10:56:21.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:56:21.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.684+0000 7f76dafd5640 1 Processor -- start 2024-04-20T10:56:21.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.684+0000 7f76dafd5640 1 -- start start 2024-04-20T10:56:21.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76dafd5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d40c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76dafd5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d40cd910 con 0x7f76d415f8b0 2024-04-20T10:56:21.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76dafd5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d40cc5a0 con 0x7f76d40cd4f0 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76dafd5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f76d40cc350 con 0x7f76d415edc0 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d9572640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f76d415f8b0 0x7f76d4155260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37384/0 (socket says 172.21.15.27:37384) 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d9572640 1 -- 172.21.15.27:0/2092272424 learned_addr learned my addr 172.21.15.27:0/2092272424 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d8d71640 1 --2- 172.21.15.27:0/2092272424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d40c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d37fe640 1 -- 172.21.15.27:0/2092272424 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715672601 0 0) 0x7f76cc001490 con 0x7f76d415f8b0 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d37fe640 1 -- 172.21.15.27:0/2092272424 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76c4003520 con 0x7f76d415f8b0 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d37fe640 1 -- 172.21.15.27:0/2092272424 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2161114974 0 0) 0x7f76d40cc5a0 con 0x7f76d40cd4f0 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d37fe640 1 -- 172.21.15.27:0/2092272424 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76cc001490 con 0x7f76d40cd4f0 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d8d71640 1 -- 172.21.15.27:0/2092272424 >> v1:172.21.15.137:6789/0 conn(0x7f76d40cd4f0 legacy=0x7f76d415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d8d71640 1 -- 172.21.15.27:0/2092272424 >> v1:172.21.15.27:6789/0 conn(0x7f76d415f8b0 legacy=0x7f76d4155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d37fe640 1 -- 172.21.15.27:0/2092272424 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3190209452 0 0) 0x7f76c4003520 con 0x7f76d415f8b0 2024-04-20T10:56:21.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.685+0000 7f76d8d71640 1 -- 172.21.15.27:0/2092272424 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d40cc880 con 0x7f76d415edc0 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.686+0000 7f76d8d71640 1 --2- 172.21.15.27:0/2092272424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d40c9620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76c80097d0 tx=0x7f76c802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1732c072ad26d3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.686+0000 7f76d37fe640 1 -- 172.21.15.27:0/2092272424 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc001bc0 con 0x7f76d415edc0 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.686+0000 7f76d37fe640 1 -- 172.21.15.27:0/2092272424 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c8037070 con 0x7f76d415edc0 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.686+0000 7f76d37fe640 1 -- 172.21.15.27:0/2092272424 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8031780 con 0x7f76d415edc0 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.687+0000 7f76dafd5640 1 -- 172.21.15.27:0/2092272424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 msgr2=0x7f76d40c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.687+0000 7f76dafd5640 1 --2- 172.21.15.27:0/2092272424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d40c9620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f76c80097d0 tx=0x7f76c802b8a0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.687+0000 7f76dafd5640 1 -- 172.21.15.27:0/2092272424 shutdown_connections 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.687+0000 7f76dafd5640 1 --2- 172.21.15.27:0/2092272424 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d40c9620 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.687+0000 7f76dafd5640 1 -- 172.21.15.27:0/2092272424 >> 172.21.15.27:0/2092272424 conn(0x7f76d406fc90 msgr2=0x7f76d4070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.687+0000 7f76dafd5640 1 -- 172.21.15.27:0/2092272424 shutdown_connections 2024-04-20T10:56:21.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.687+0000 7f76dafd5640 1 -- 172.21.15.27:0/2092272424 wait complete. 2024-04-20T10:56:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.688+0000 7f76dafd5640 1 Processor -- start 2024-04-20T10:56:21.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.688+0000 7f76dafd5640 1 -- start start 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76dafd5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d40cd4f0 0x7f76d41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76dafd5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d4157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76dafd5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d415e520 0x7f76d4157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76dafd5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40c3670 con 0x7f76d40cd4f0 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76dafd5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40c3390 con 0x7f76d415e520 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76dafd5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f76d40c8b70 con 0x7f76d415edc0 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d8d71640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d40cd4f0 0x7f76d41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d8d71640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d40cd4f0 0x7f76d41552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35530/0 (socket says 172.21.15.27:35530) 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d8d71640 1 -- 172.21.15.27:0/3683740938 learned_addr learned my addr 172.21.15.27:0/3683740938 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d9572640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d415e520 0x7f76d4157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d3fff640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d4157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d8d71640 1 -- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 msgr2=0x7f76d4157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d8d71640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d4157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d8d71640 1 -- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d415e520 msgr2=0x7f76d4157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d8d71640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d415e520 0x7f76d4157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d8d71640 1 -- 172.21.15.27:0/3683740938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d40cab40 con 0x7f76d40cd4f0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.689+0000 7f76d3fff640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d4157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.690+0000 7f76d8d71640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d40cd4f0 0x7f76d41552a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76c8000c00 tx=0x7f76c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.690+0000 7f76d17fa640 1 -- 172.21.15.27:0/3683740938 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8032610 con 0x7f76d40cd4f0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.690+0000 7f76d17fa640 1 -- 172.21.15.27:0/3683740938 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c80327b0 con 0x7f76d40cd4f0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.690+0000 7f76d17fa640 1 -- 172.21.15.27:0/3683740938 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8032ba0 con 0x7f76d40cd4f0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.690+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d4156f40 con 0x7f76d40cd4f0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.690+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41f19f0 con 0x7f76d40cd4f0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.691+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d40695b0 con 0x7f76d40cd4f0 2024-04-20T10:56:21.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.691+0000 7f76d17fa640 1 -- 172.21.15.27:0/3683740938 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76c804c060 con 0x7f76d40cd4f0 2024-04-20T10:56:21.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.692+0000 7f76d17fa640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f76b806c8f0 0x7f76b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:21.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.692+0000 7f76d17fa640 1 -- 172.21.15.27:0/3683740938 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f76c80ad490 con 0x7f76d40cd4f0 2024-04-20T10:56:21.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.692+0000 7f76d3fff640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f76b806c8f0 0x7f76b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:21.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.693+0000 7f76d3fff640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f76b806c8f0 0x7f76b806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f76d40b3ae0 tx=0x7f76c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:21.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.696+0000 7f76d17fa640 1 -- 172.21.15.27:0/3683740938 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f76d40695b0 con 0x7f76d40cd4f0 2024-04-20T10:56:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.882+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f76d404fcf0 con 0x7f76d40cd4f0 2024-04-20T10:56:21.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.883+0000 7f76d17fa640 1 -- 172.21.15.27:0/3683740938 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f76c8040070 con 0x7f76d40cd4f0 2024-04-20T10:56:21.883 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-20T10:56:21.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.884+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f76b806c8f0 msgr2=0x7f76b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.884+0000 7f76dafd5640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f76b806c8f0 0x7f76b806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f76d40b3ae0 tx=0x7f76c0009040 comp rx=0 tx=0).stop 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.884+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d40cd4f0 msgr2=0x7f76d41552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.884+0000 7f76dafd5640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d40cd4f0 0x7f76d41552a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76c8000c00 tx=0x7f76c80046c0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.885+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 shutdown_connections 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.885+0000 7f76dafd5640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f76b806c8f0 0x7f76b806edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.885+0000 7f76dafd5640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76d415e520 0x7f76d4157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.885+0000 7f76dafd5640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76d415edc0 0x7f76d4157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.885+0000 7f76dafd5640 1 --2- 172.21.15.27:0/3683740938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76d40cd4f0 0x7f76d41552a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.885+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 >> 172.21.15.27:0/3683740938 conn(0x7f76d406fc90 msgr2=0x7f76d4075070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.885+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 shutdown_connections 2024-04-20T10:56:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:21.885+0000 7f76dafd5640 1 -- 172.21.15.27:0/3683740938 wait complete. 2024-04-20T10:56:21.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:56:22.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.001+0000 7f5a3fd50640 1 Processor -- start 2024-04-20T10:56:22.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.001+0000 7f5a3fd50640 1 -- start start 2024-04-20T10:56:22.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.002+0000 7f5a3fd50640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a38155a20 0x7f5a380c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.002+0000 7f5a3fd50640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 0x7f5a380cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.002+0000 7f5a3fd50640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a3815f210 con 0x7f5a380c62e0 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.002+0000 7f5a3fd50640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5a38155e00 con 0x7f5a38155a20 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.002+0000 7f5a3fd50640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3815ba30 con 0x7f5a3815ec40 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.002+0000 7f5a3daec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a38155a20 0x7f5a380c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.002+0000 7f5a3daec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a38155a20 0x7f5a380c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35544/0 (socket says 172.21.15.27:35544) 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.002+0000 7f5a3d2eb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5a380c62e0 0x7f5a3815bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37396/0 (socket says 172.21.15.27:37396) 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.002+0000 7f5a3daec640 1 -- 172.21.15.27:0/1135782320 learned_addr learned my addr 172.21.15.27:0/1135782320 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.003+0000 7f5a3e2ed640 1 --2- 172.21.15.27:0/1135782320 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 0x7f5a380cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.003+0000 7f5a3daec640 1 -- 172.21.15.27:0/1135782320 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 msgr2=0x7f5a380cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.003+0000 7f5a3daec640 1 --2- 172.21.15.27:0/1135782320 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 0x7f5a380cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.003+0000 7f5a3daec640 1 -- 172.21.15.27:0/1135782320 >> v1:172.21.15.27:6789/0 conn(0x7f5a380c62e0 legacy=0x7f5a3815bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.003+0000 7f5a3daec640 1 -- 172.21.15.27:0/1135782320 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3815f4f0 con 0x7f5a38155a20 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.003+0000 7f5a3daec640 1 --2- 172.21.15.27:0/1135782320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 0x7f5a380c5bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009870 tx=0x7f5a3002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f53612338e258f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.003+0000 7f5a3caea640 1 -- 172.21.15.27:0/1135782320 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3002f840 con 0x7f5a38155a20 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3caea640 1 -- 172.21.15.27:0/1135782320 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3002f9e0 con 0x7f5a38155a20 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3caea640 1 -- 172.21.15.27:0/1135782320 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3002fdb0 con 0x7f5a38155a20 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3fd50640 1 -- 172.21.15.27:0/1135782320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 msgr2=0x7f5a380c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3fd50640 1 --2- 172.21.15.27:0/1135782320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 0x7f5a380c5bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009870 tx=0x7f5a3002ba20 comp rx=0 tx=0).stop 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3fd50640 1 -- 172.21.15.27:0/1135782320 shutdown_connections 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3fd50640 1 --2- 172.21.15.27:0/1135782320 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 0x7f5a380cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3fd50640 1 --2- 172.21.15.27:0/1135782320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 0x7f5a380c5bb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3fd50640 1 -- 172.21.15.27:0/1135782320 >> 172.21.15.27:0/1135782320 conn(0x7f5a380678a0 msgr2=0x7f5a38067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:22.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3fd50640 1 -- 172.21.15.27:0/1135782320 shutdown_connections 2024-04-20T10:56:22.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.004+0000 7f5a3fd50640 1 -- 172.21.15.27:0/1135782320 wait complete. 2024-04-20T10:56:22.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.005+0000 7f5a3fd50640 1 Processor -- start 2024-04-20T10:56:22.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.005+0000 7f5a3fd50640 1 -- start start 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3fd50640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a380c62e0 0x7f5a381e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3fd50640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 0x7f5a381f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3fd50640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 0x7f5a38159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3fd50640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3815c5d0 con 0x7f5a38155a20 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3fd50640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3815fab0 con 0x7f5a380c62e0 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3fd50640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5a380c67c0 con 0x7f5a3815ec40 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3d2eb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 0x7f5a381f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3d2eb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 0x7f5a381f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35554/0 (socket says 172.21.15.27:35554) 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3d2eb640 1 -- 172.21.15.27:0/2772335065 learned_addr learned my addr 172.21.15.27:0/2772335065 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3e2ed640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 0x7f5a38159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3daec640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a380c62e0 0x7f5a381e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3d2eb640 1 -- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 msgr2=0x7f5a38159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3d2eb640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 0x7f5a38159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3d2eb640 1 -- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a380c62e0 msgr2=0x7f5a381e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3d2eb640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a380c62e0 0x7f5a381e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3d2eb640 1 -- 172.21.15.27:0/2772335065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a381f5e50 con 0x7f5a38155a20 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3e2ed640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 0x7f5a38159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.006+0000 7f5a3daec640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a380c62e0 0x7f5a381e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.007+0000 7f5a3d2eb640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 0x7f5a381f5910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00c3f0 tx=0x7f5a2c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.007+0000 7f5a367fc640 1 -- 172.21.15.27:0/2772335065 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0074e0 con 0x7f5a38155a20 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.007+0000 7f5a367fc640 1 -- 172.21.15.27:0/2772335065 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a2c007680 con 0x7f5a38155a20 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.007+0000 7f5a367fc640 1 -- 172.21.15.27:0/2772335065 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c004a00 con 0x7f5a38155a20 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.007+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a381e5cb0 con 0x7f5a38155a20 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.007+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3806db00 con 0x7f5a38155a20 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.008+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a380b42c0 con 0x7f5a38155a20 2024-04-20T10:56:22.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.008+0000 7f5a367fc640 1 -- 172.21.15.27:0/2772335065 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a2c01a020 con 0x7f5a38155a20 2024-04-20T10:56:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.009+0000 7f5a367fc640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a2406c920 0x7f5a2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.009+0000 7f5a367fc640 1 -- 172.21.15.27:0/2772335065 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f5a2c08b3b0 con 0x7f5a38155a20 2024-04-20T10:56:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.009+0000 7f5a3daec640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a2406c920 0x7f5a2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.009+0000 7f5a3daec640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a2406c920 0x7f5a2406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5a30036000 tx=0x7f5a30039040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.013+0000 7f5a367fc640 1 -- 172.21.15.27:0/2772335065 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5a380b42c0 con 0x7f5a38155a20 2024-04-20T10:56:22.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.201+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5a3804fcf0 con 0x7f5a38155a20 2024-04-20T10:56:22.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.201+0000 7f5a367fc640 1 -- 172.21.15.27:0/2772335065 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f5a2c010300 con 0x7f5a38155a20 2024-04-20T10:56:22.201 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-20T10:56:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a2406c920 msgr2=0x7f5a2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a2406c920 0x7f5a2406ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5a30036000 tx=0x7f5a30039040 comp rx=0 tx=0).stop 2024-04-20T10:56:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 msgr2=0x7f5a381f5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 0x7f5a381f5910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00c3f0 tx=0x7f5a2c00c8b0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 shutdown_connections 2024-04-20T10:56:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a2406c920 0x7f5a2406ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a3815ec40 0x7f5a38159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a38155a20 0x7f5a381f5910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 --2- 172.21.15.27:0/2772335065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a380c62e0 0x7f5a381e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 >> 172.21.15.27:0/2772335065 conn(0x7f5a380678a0 msgr2=0x7f5a380c7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 shutdown_connections 2024-04-20T10:56:22.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.203+0000 7f5a3fd50640 1 -- 172.21.15.27:0/2772335065 wait complete. 2024-04-20T10:56:22.214 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2024-04-20T10:56:22.214 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:56:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.315+0000 7fcd34d9f640 1 Processor -- start 2024-04-20T10:56:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.315+0000 7fcd34d9f640 1 -- start start 2024-04-20T10:56:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.316+0000 7fcd34d9f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcd30152680 0x7fcd3015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.316+0000 7fcd34d9f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcd300c2ea0 0x7fcd30162b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.316+0000 7fcd34d9f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd300b3f70 con 0x7fcd3015c510 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.316+0000 7fcd34d9f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fcd301587d0 con 0x7fcd300c2ea0 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.316+0000 7fcd34d9f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fcd30158ab0 con 0x7fcd30152680 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.316+0000 7fcd2ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcd300c2ea0 0x7fcd30162b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.317+0000 7fcd2dd9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcd3015c510 0x7fcd300c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37408/0 (socket says 172.21.15.27:37408) 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.317+0000 7fcd2dd9b640 1 -- 172.21.15.27:0/2767153248 learned_addr learned my addr 172.21.15.27:0/2767153248 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.317+0000 7fcd2e59c640 1 --2- 172.21.15.27:0/2767153248 >> v2:172.21.15.137:3300/0 conn(0x7fcd30152680 0x7fcd3015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.317+0000 7fcd2ed9d640 1 -- 172.21.15.27:0/2767153248 >> v2:172.21.15.137:3300/0 conn(0x7fcd30152680 msgr2=0x7fcd3015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.317+0000 7fcd2ed9d640 1 --2- 172.21.15.27:0/2767153248 >> v2:172.21.15.137:3300/0 conn(0x7fcd30152680 0x7fcd3015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.317+0000 7fcd2ed9d640 1 -- 172.21.15.27:0/2767153248 >> v1:172.21.15.27:6789/0 conn(0x7fcd3015c510 legacy=0x7fcd300c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.317+0000 7fcd2ed9d640 1 -- 172.21.15.27:0/2767153248 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd30158d90 con 0x7fcd300c2ea0 2024-04-20T10:56:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.317+0000 7fcd2d59a640 1 -- 172.21.15.27:0/2767153248 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3552525951 0 0) 0x7fcd300b3f70 con 0x7fcd3015c510 2024-04-20T10:56:22.318 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.317+0000 7fcd2e59c640 1 --2- 172.21.15.27:0/2767153248 >> v2:172.21.15.137:3300/0 conn(0x7fcd30152680 0x7fcd3015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd2ed9d640 1 --2- 172.21.15.27:0/2767153248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd300c2ea0 0x7fcd30162b60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcd3006db10 tx=0x7fcd2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fa32e6f66259cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd2d59a640 1 -- 172.21.15.27:0/2767153248 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28039070 con 0x7fcd300c2ea0 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd2d59a640 1 -- 172.21.15.27:0/2767153248 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2802cc80 con 0x7fcd300c2ea0 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd2d59a640 1 -- 172.21.15.27:0/2767153248 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28034660 con 0x7fcd300c2ea0 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd34d9f640 1 -- 172.21.15.27:0/2767153248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd300c2ea0 msgr2=0x7fcd30162b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd34d9f640 1 --2- 172.21.15.27:0/2767153248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd300c2ea0 0x7fcd30162b60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcd3006db10 tx=0x7fcd2802bed0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd34d9f640 1 -- 172.21.15.27:0/2767153248 shutdown_connections 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd34d9f640 1 --2- 172.21.15.27:0/2767153248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd300c2ea0 0x7fcd30162b60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd34d9f640 1 --2- 172.21.15.27:0/2767153248 >> v2:172.21.15.137:3300/0 conn(0x7fcd30152680 0x7fcd3015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd34d9f640 1 -- 172.21.15.27:0/2767153248 >> 172.21.15.27:0/2767153248 conn(0x7fcd3006fb50 msgr2=0x7fcd3006ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd34d9f640 1 -- 172.21.15.27:0/2767153248 shutdown_connections 2024-04-20T10:56:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.318+0000 7fcd34d9f640 1 -- 172.21.15.27:0/2767153248 wait complete. 2024-04-20T10:56:22.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.320+0000 7fcd34d9f640 1 Processor -- start 2024-04-20T10:56:22.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.320+0000 7fcd34d9f640 1 -- start start 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.320+0000 7fcd34d9f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd300c2ea0 0x7fcd300cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.320+0000 7fcd34d9f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcd30152680 0x7fcd301e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.320+0000 7fcd34d9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd3015c510 0x7fcd301e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.320+0000 7fcd34d9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3015b7a0 con 0x7fcd3015c510 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.320+0000 7fcd34d9f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd300c9a50 con 0x7fcd300c2ea0 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.320+0000 7fcd34d9f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fcd300c93e0 con 0x7fcd30152680 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd3015c510 0x7fcd301e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd3015c510 0x7fcd301e57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35566/0 (socket says 172.21.15.27:35566) 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2ed9d640 1 -- 172.21.15.27:0/1792653774 learned_addr learned my addr 172.21.15.27:0/1792653774 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2e59c640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd300c2ea0 0x7fcd300cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2ed9d640 1 -- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcd30152680 msgr2=0x7fcd301e4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2ed9d640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcd30152680 0x7fcd301e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2ed9d640 1 -- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd300c2ea0 msgr2=0x7fcd300cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2ed9d640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd300c2ea0 0x7fcd300cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2ed9d640 1 -- 172.21.15.27:0/1792653774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd300cbd60 con 0x7fcd3015c510 2024-04-20T10:56:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2e59c640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd300c2ea0 0x7fcd300cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd2ed9d640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd3015c510 0x7fcd301e57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009e80 tx=0x7fcd28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd1effd640 1 -- 172.21.15.27:0/1792653774 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28039050 con 0x7fcd3015c510 2024-04-20T10:56:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd1effd640 1 -- 172.21.15.27:0/1792653774 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd28045070 con 0x7fcd3015c510 2024-04-20T10:56:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd1effd640 1 -- 172.21.15.27:0/1792653774 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2803e830 con 0x7fcd3015c510 2024-04-20T10:56:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3015b580 con 0x7fcd3015c510 2024-04-20T10:56:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.321+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd301518e0 con 0x7fcd3015c510 2024-04-20T10:56:22.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.322+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3004fcf0 con 0x7fcd3015c510 2024-04-20T10:56:22.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.323+0000 7fcd1effd640 1 -- 172.21.15.27:0/1792653774 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd2803e9d0 con 0x7fcd3015c510 2024-04-20T10:56:22.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.323+0000 7fcd1effd640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcd1006c920 0x7fcd1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.323+0000 7fcd1effd640 1 -- 172.21.15.27:0/1792653774 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7fcd280ae980 con 0x7fcd3015c510 2024-04-20T10:56:22.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.323+0000 7fcd2e59c640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcd1006c920 0x7fcd1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.324 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.324+0000 7fcd2e59c640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcd1006c920 0x7fcd1006ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcd20006fd0 tx=0x7fcd20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.328+0000 7fcd1effd640 1 -- 172.21.15.27:0/1792653774 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcd3015b580 con 0x7fcd3015c510 2024-04-20T10:56:22.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.554+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd3014fc80 con 0x7fcd3015c510 2024-04-20T10:56:22.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.554+0000 7fcd1effd640 1 -- 172.21.15.27:0/1792653774 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7fcd28032e20 con 0x7fcd3015c510 2024-04-20T10:56:22.554 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:22.555 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:56:22.555 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcd1006c920 msgr2=0x7fcd1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcd1006c920 0x7fcd1006ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcd20006fd0 tx=0x7fcd20008040 comp rx=0 tx=0).stop 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd3015c510 msgr2=0x7fcd301e57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd3015c510 0x7fcd301e57c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009e80 tx=0x7fcd28004a40 comp rx=0 tx=0).stop 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 shutdown_connections 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcd1006c920 0x7fcd1006ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd3015c510 0x7fcd301e57c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcd30152680 0x7fcd301e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 --2- 172.21.15.27:0/1792653774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd300c2ea0 0x7fcd300cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 >> 172.21.15.27:0/1792653774 conn(0x7fcd3006fb50 msgr2=0x7fcd300c43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 shutdown_connections 2024-04-20T10:56:22.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.557+0000 7fcd34d9f640 1 -- 172.21.15.27:0/1792653774 wait complete. 2024-04-20T10:56:22.568 INFO:tasks.mds_thrash:Ready to start thrashing 2024-04-20T10:56:22.568 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2024-04-20T10:56:22.568 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:56:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.669+0000 7f63be414640 1 Processor -- start 2024-04-20T10:56:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.669+0000 7f63be414640 1 -- start start 2024-04-20T10:56:22.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63be414640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63b8152260 0x7f63b80719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63be414640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b8152640 0x7f63b80c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63be414640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b815ba80 con 0x7f63b80cd6f0 2024-04-20T10:56:22.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63be414640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f63b815bd60 con 0x7f63b8152260 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63be414640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f63b815c040 con 0x7f63b8152640 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63b7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63b8152260 0x7f63b80719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63b7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63b8152260 0x7f63b80719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35578/0 (socket says 172.21.15.27:35578) 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63bc9b1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f63b80cd6f0 0x7f63b8158760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37412/0 (socket says 172.21.15.27:37412) 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63b7fff640 1 -- 172.21.15.27:0/526899727 learned_addr learned my addr 172.21.15.27:0/526899727 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63b77fe640 1 --2- 172.21.15.27:0/526899727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b8152640 0x7f63b80c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63b7fff640 1 -- 172.21.15.27:0/526899727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b8152640 msgr2=0x7f63b80c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63b7fff640 1 --2- 172.21.15.27:0/526899727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b8152640 0x7f63b80c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63b7fff640 1 -- 172.21.15.27:0/526899727 >> v1:172.21.15.27:6789/0 conn(0x7f63b80cd6f0 legacy=0x7f63b8158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63b7fff640 1 -- 172.21.15.27:0/526899727 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b815c320 con 0x7f63b8152260 2024-04-20T10:56:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.670+0000 7f63b6ffd640 1 -- 172.21.15.27:0/526899727 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3595989558 0 0) 0x7f63b815ba80 con 0x7f63b80cd6f0 2024-04-20T10:56:22.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.671+0000 7f63b7fff640 1 --2- 172.21.15.27:0/526899727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152260 0x7f63b80719c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009870 tx=0x7f63a002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0867842d66cd539 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.672 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.671+0000 7f63b6ffd640 1 -- 172.21.15.27:0/526899727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a002f840 con 0x7f63b8152260 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.671+0000 7f63b6ffd640 1 -- 172.21.15.27:0/526899727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a002f9e0 con 0x7f63b8152260 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.671+0000 7f63b6ffd640 1 -- 172.21.15.27:0/526899727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a002fdb0 con 0x7f63b8152260 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.672+0000 7f63be414640 1 -- 172.21.15.27:0/526899727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152260 msgr2=0x7f63b80719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.672+0000 7f63be414640 1 --2- 172.21.15.27:0/526899727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152260 0x7f63b80719c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009870 tx=0x7f63a002ba20 comp rx=0 tx=0).stop 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.672+0000 7f63be414640 1 -- 172.21.15.27:0/526899727 shutdown_connections 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.672+0000 7f63be414640 1 --2- 172.21.15.27:0/526899727 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b8152640 0x7f63b80c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.672+0000 7f63be414640 1 --2- 172.21.15.27:0/526899727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152260 0x7f63b80719c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.672+0000 7f63be414640 1 -- 172.21.15.27:0/526899727 >> 172.21.15.27:0/526899727 conn(0x7f63b80ccbd0 msgr2=0x7f63b80ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.672+0000 7f63be414640 1 -- 172.21.15.27:0/526899727 shutdown_connections 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.672+0000 7f63be414640 1 -- 172.21.15.27:0/526899727 wait complete. 2024-04-20T10:56:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.673+0000 7f63be414640 1 Processor -- start 2024-04-20T10:56:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.673+0000 7f63be414640 1 -- start start 2024-04-20T10:56:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63be414640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b80cd6f0 0x7f63b8147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63be414640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63b8152260 0x7f63b81efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63be414640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152640 0x7f63b8144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63be414640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f63b815c600 con 0x7f63b8152640 2024-04-20T10:56:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63be414640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8074260 con 0x7f63b8152260 2024-04-20T10:56:22.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63be414640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f63b815b720 con 0x7f63b80cd6f0 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63bc9b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152640 0x7f63b8144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63bc9b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152640 0x7f63b8144410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35586/0 (socket says 172.21.15.27:35586) 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63bc9b1640 1 -- 172.21.15.27:0/491139664 learned_addr learned my addr 172.21.15.27:0/491139664 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63b7fff640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b80cd6f0 0x7f63b8147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63b77fe640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63b8152260 0x7f63b81efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63bc9b1640 1 -- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b80cd6f0 msgr2=0x7f63b8147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63bc9b1640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b80cd6f0 0x7f63b8147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63bc9b1640 1 -- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63b8152260 msgr2=0x7f63b81efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63bc9b1640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63b8152260 0x7f63b81efe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63bc9b1640 1 -- 172.21.15.27:0/491139664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b81f2a90 con 0x7f63b8152640 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63b7fff640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b80cd6f0 0x7f63b8147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.674+0000 7f63b77fe640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63b8152260 0x7f63b81efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.675+0000 7f63bc9b1640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152640 0x7f63b8144410 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63b000c700 tx=0x7f63b000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.675+0000 7f63b4ff9640 1 -- 172.21.15.27:0/491139664 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0018070 con 0x7f63b8152640 2024-04-20T10:56:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.675+0000 7f63b4ff9640 1 -- 172.21.15.27:0/491139664 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b000ce10 con 0x7f63b8152640 2024-04-20T10:56:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.675+0000 7f63b4ff9640 1 -- 172.21.15.27:0/491139664 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b000a880 con 0x7f63b8152640 2024-04-20T10:56:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.675+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b81eea70 con 0x7f63b8152640 2024-04-20T10:56:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.675+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b81ef030 con 0x7f63b8152640 2024-04-20T10:56:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.676+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b804fcf0 con 0x7f63b8152640 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.676+0000 7f63b4ff9640 1 -- 172.21.15.27:0/491139664 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63b000aa20 con 0x7f63b8152640 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.677+0000 7f63b4ff9640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f639806c8f0 0x7f639806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.677+0000 7f63b4ff9640 1 -- 172.21.15.27:0/491139664 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f63b008b1a0 con 0x7f63b8152640 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.677+0000 7f63b7fff640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f639806c8f0 0x7f639806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:22.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.678+0000 7f63b7fff640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f639806c8f0 0x7f639806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63a0036000 tx=0x7f63a0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.681+0000 7f63b4ff9640 1 -- 172.21.15.27:0/491139664 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f63b0059550 con 0x7f63b8152640 2024-04-20T10:56:22.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.866+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63b80c4620 con 0x7f639806c8f0 2024-04-20T10:56:22.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.873+0000 7f63b4ff9640 1 -- 172.21.15.27:0/491139664 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251928 (secure 0 0 0) 0x7f63a002f8b0 con 0x7f639806c8f0 2024-04-20T10:56:22.875 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:22.877 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T10:56:22.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f639806c8f0 msgr2=0x7f639806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.878 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f639806c8f0 0x7f639806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63a0036000 tx=0x7f63a0008030 comp rx=0 tx=0).stop 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152640 msgr2=0x7f63b8144410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152640 0x7f63b8144410 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63b000c700 tx=0x7f63b000cbc0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 shutdown_connections 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f639806c8f0 0x7f639806edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63b8152640 0x7f63b8144410 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63b8152260 0x7f63b81efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 --2- 172.21.15.27:0/491139664 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f63b80cd6f0 0x7f63b8147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 >> 172.21.15.27:0/491139664 conn(0x7f63b80ccbd0 msgr2=0x7f63b8158760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 shutdown_connections 2024-04-20T10:56:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:22.876+0000 7f63be414640 1 -- 172.21.15.27:0/491139664 wait complete. 2024-04-20T10:56:22.887 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-20T10:56:21.897549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598914,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":368,"num_write_kb":2482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1918,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.188698"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287393+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T10:55:44.287393+0000","last_peered":"2024-04-20T10:55:44.287393+0000","last_clean":"2024-04-20T10:55:44.287393+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T10:55:44.287393+0000","last_undegraded":"2024-04-20T10:55:44.287393+0000","last_fullsized":"2024-04-20T10:55:44.287393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797629+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T10:55:43.797629+0000","last_peered":"2024-04-20T10:55:43.797629+0000","last_clean":"2024-04-20T10:55:43.797629+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T10:55:43.797629+0000","last_undegraded":"2024-04-20T10:55:43.797629+0000","last_fullsized":"2024-04-20T10:55:43.797629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285486+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T10:55:44.285486+0000","last_peered":"2024-04-20T10:55:44.285486+0000","last_clean":"2024-04-20T10:55:44.285486+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T10:55:44.285486+0000","last_undegraded":"2024-04-20T10:55:44.285486+0000","last_fullsized":"2024-04-20T10:55:44.285486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286383+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T10:55:44.286383+0000","last_peered":"2024-04-20T10:55:44.286383+0000","last_clean":"2024-04-20T10:55:44.286383+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T10:55:44.286383+0000","last_undegraded":"2024-04-20T10:55:44.286383+0000","last_fullsized":"2024-04-20T10:55:44.286383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286289+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T10:55:44.286289+0000","last_peered":"2024-04-20T10:55:44.286289+0000","last_clean":"2024-04-20T10:55:44.286289+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T10:55:44.286289+0000","last_undegraded":"2024-04-20T10:55:44.286289+0000","last_fullsized":"2024-04-20T10:55:44.286289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287329+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T10:55:44.287329+0000","last_peered":"2024-04-20T10:55:44.287329+0000","last_clean":"2024-04-20T10:55:44.287329+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T10:55:44.287329+0000","last_undegraded":"2024-04-20T10:55:44.287329+0000","last_fullsized":"2024-04-20T10:55:44.287329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287439+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T10:55:44.287439+0000","last_peered":"2024-04-20T10:55:44.287439+0000","last_clean":"2024-04-20T10:55:44.287439+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T10:55:44.287439+0000","last_undegraded":"2024-04-20T10:55:44.287439+0000","last_fullsized":"2024-04-20T10:55:44.287439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792859+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T10:55:43.792859+0000","last_peered":"2024-04-20T10:55:43.792859+0000","last_clean":"2024-04-20T10:55:43.792859+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T10:55:43.792859+0000","last_undegraded":"2024-04-20T10:55:43.792859+0000","last_fullsized":"2024-04-20T10:55:43.792859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286907+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T10:55:44.286907+0000","last_peered":"2024-04-20T10:55:44.286907+0000","last_clean":"2024-04-20T10:55:44.286907+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T10:55:44.286907+0000","last_undegraded":"2024-04-20T10:55:44.286907+0000","last_fullsized":"2024-04-20T10:55:44.286907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286541+0000","last_change":"2024-04-20T10:55:42.802716+0000","last_active":"2024-04-20T10:55:44.286541+0000","last_peered":"2024-04-20T10:55:44.286541+0000","last_clean":"2024-04-20T10:55:44.286541+0000","last_became_active":"2024-04-20T10:55:42.802418+0000","last_became_peered":"2024-04-20T10:55:42.802418+0000","last_unstale":"2024-04-20T10:55:44.286541+0000","last_undegraded":"2024-04-20T10:55:44.286541+0000","last_fullsized":"2024-04-20T10:55:44.286541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:12:22.186243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286228+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T10:55:44.286228+0000","last_peered":"2024-04-20T10:55:44.286228+0000","last_clean":"2024-04-20T10:55:44.286228+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T10:55:44.286228+0000","last_undegraded":"2024-04-20T10:55:44.286228+0000","last_fullsized":"2024-04-20T10:55:44.286228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801002+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T10:55:43.801002+0000","last_peered":"2024-04-20T10:55:43.801002+0000","last_clean":"2024-04-20T10:55:43.801002+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T10:55:43.801002+0000","last_undegraded":"2024-04-20T10:55:43.801002+0000","last_fullsized":"2024-04-20T10:55:43.801002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793050+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T10:55:43.793050+0000","last_peered":"2024-04-20T10:55:43.793050+0000","last_clean":"2024-04-20T10:55:43.793050+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T10:55:43.793050+0000","last_undegraded":"2024-04-20T10:55:43.793050+0000","last_fullsized":"2024-04-20T10:55:43.793050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797332+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T10:55:43.797332+0000","last_peered":"2024-04-20T10:55:43.797332+0000","last_clean":"2024-04-20T10:55:43.797332+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T10:55:43.797332+0000","last_undegraded":"2024-04-20T10:55:43.797332+0000","last_fullsized":"2024-04-20T10:55:43.797332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285851+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T10:55:44.285851+0000","last_peered":"2024-04-20T10:55:44.285851+0000","last_clean":"2024-04-20T10:55:44.285851+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T10:55:44.285851+0000","last_undegraded":"2024-04-20T10:55:44.285851+0000","last_fullsized":"2024-04-20T10:55:44.285851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286491+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T10:55:44.286491+0000","last_peered":"2024-04-20T10:55:44.286491+0000","last_clean":"2024-04-20T10:55:44.286491+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T10:55:44.286491+0000","last_undegraded":"2024-04-20T10:55:44.286491+0000","last_fullsized":"2024-04-20T10:55:44.286491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285217+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T10:55:44.285217+0000","last_peered":"2024-04-20T10:55:44.285217+0000","last_clean":"2024-04-20T10:55:44.285217+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T10:55:44.285217+0000","last_undegraded":"2024-04-20T10:55:44.285217+0000","last_fullsized":"2024-04-20T10:55:44.285217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287175+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T10:55:44.287175+0000","last_peered":"2024-04-20T10:55:44.287175+0000","last_clean":"2024-04-20T10:55:44.287175+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T10:55:44.287175+0000","last_undegraded":"2024-04-20T10:55:44.287175+0000","last_fullsized":"2024-04-20T10:55:44.287175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286994+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T10:55:44.286994+0000","last_peered":"2024-04-20T10:55:44.286994+0000","last_clean":"2024-04-20T10:55:44.286994+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T10:55:44.286994+0000","last_undegraded":"2024-04-20T10:55:44.286994+0000","last_fullsized":"2024-04-20T10:55:44.286994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285751+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T10:55:44.285751+0000","last_peered":"2024-04-20T10:55:44.285751+0000","last_clean":"2024-04-20T10:55:44.285751+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T10:55:44.285751+0000","last_undegraded":"2024-04-20T10:55:44.285751+0000","last_fullsized":"2024-04-20T10:55:44.285751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287484+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T10:55:44.287484+0000","last_peered":"2024-04-20T10:55:44.287484+0000","last_clean":"2024-04-20T10:55:44.287484+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T10:55:44.287484+0000","last_undegraded":"2024-04-20T10:55:44.287484+0000","last_fullsized":"2024-04-20T10:55:44.287484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792921+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T10:55:43.792921+0000","last_peered":"2024-04-20T10:55:43.792921+0000","last_clean":"2024-04-20T10:55:43.792921+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T10:55:43.792921+0000","last_undegraded":"2024-04-20T10:55:43.792921+0000","last_fullsized":"2024-04-20T10:55:43.792921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793499+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T10:55:43.793499+0000","last_peered":"2024-04-20T10:55:43.793499+0000","last_clean":"2024-04-20T10:55:43.793499+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T10:55:43.793499+0000","last_undegraded":"2024-04-20T10:55:43.793499+0000","last_fullsized":"2024-04-20T10:55:43.793499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286683+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T10:55:44.286683+0000","last_peered":"2024-04-20T10:55:44.286683+0000","last_clean":"2024-04-20T10:55:44.286683+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T10:55:44.286683+0000","last_undegraded":"2024-04-20T10:55:44.286683+0000","last_fullsized":"2024-04-20T10:55:44.286683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797439+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T10:55:43.797439+0000","last_peered":"2024-04-20T10:55:43.797439+0000","last_clean":"2024-04-20T10:55:43.797439+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T10:55:43.797439+0000","last_undegraded":"2024-04-20T10:55:43.797439+0000","last_fullsized":"2024-04-20T10:55:43.797439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793232+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T10:55:43.793232+0000","last_peered":"2024-04-20T10:55:43.793232+0000","last_clean":"2024-04-20T10:55:43.793232+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T10:55:43.793232+0000","last_undegraded":"2024-04-20T10:55:43.793232+0000","last_fullsized":"2024-04-20T10:55:43.793232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285757+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T10:55:44.285757+0000","last_peered":"2024-04-20T10:55:44.285757+0000","last_clean":"2024-04-20T10:55:44.285757+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T10:55:44.285757+0000","last_undegraded":"2024-04-20T10:55:44.285757+0000","last_fullsized":"2024-04-20T10:55:44.285757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287493+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T10:55:44.287493+0000","last_peered":"2024-04-20T10:55:44.287493+0000","last_clean":"2024-04-20T10:55:44.287493+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T10:55:44.287493+0000","last_undegraded":"2024-04-20T10:55:44.287493+0000","last_fullsized":"2024-04-20T10:55:44.287493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285309+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T10:55:44.285309+0000","last_peered":"2024-04-20T10:55:44.285309+0000","last_clean":"2024-04-20T10:55:44.285309+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T10:55:44.285309+0000","last_undegraded":"2024-04-20T10:55:44.285309+0000","last_fullsized":"2024-04-20T10:55:44.285309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285746+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T10:55:44.285746+0000","last_peered":"2024-04-20T10:55:44.285746+0000","last_clean":"2024-04-20T10:55:44.285746+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T10:55:44.285746+0000","last_undegraded":"2024-04-20T10:55:44.285746+0000","last_fullsized":"2024-04-20T10:55:44.285746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800661+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T10:55:43.800661+0000","last_peered":"2024-04-20T10:55:43.800661+0000","last_clean":"2024-04-20T10:55:43.800661+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T10:55:43.800661+0000","last_undegraded":"2024-04-20T10:55:43.800661+0000","last_fullsized":"2024-04-20T10:55:43.800661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.831445+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T10:55:52.831445+0000","last_peered":"2024-04-20T10:55:52.831445+0000","last_clean":"2024-04-20T10:55:52.831445+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T10:55:52.831445+0000","last_undegraded":"2024-04-20T10:55:52.831445+0000","last_fullsized":"2024-04-20T10:55:52.831445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285840+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T10:55:44.285840+0000","last_peered":"2024-04-20T10:55:44.285840+0000","last_clean":"2024-04-20T10:55:44.285840+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T10:55:44.285840+0000","last_undegraded":"2024-04-20T10:55:44.285840+0000","last_fullsized":"2024-04-20T10:55:44.285840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864139+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T10:56:04.864139+0000","last_peered":"2024-04-20T10:56:04.864139+0000","last_clean":"2024-04-20T10:56:04.864139+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T10:56:04.864139+0000","last_undegraded":"2024-04-20T10:56:04.864139+0000","last_fullsized":"2024-04-20T10:56:04.864139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:58.849736+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T10:55:58.849736+0000","last_peered":"2024-04-20T10:55:58.849736+0000","last_clean":"2024-04-20T10:55:58.849736+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T10:55:58.849736+0000","last_undegraded":"2024-04-20T10:55:58.849736+0000","last_fullsized":"2024-04-20T10:55:58.849736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792877+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T10:55:43.792877+0000","last_peered":"2024-04-20T10:55:43.792877+0000","last_clean":"2024-04-20T10:55:43.792877+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T10:55:43.792877+0000","last_undegraded":"2024-04-20T10:55:43.792877+0000","last_fullsized":"2024-04-20T10:55:43.792877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.867514+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T10:56:04.867514+0000","last_peered":"2024-04-20T10:56:04.867514+0000","last_clean":"2024-04-20T10:56:04.867514+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T10:56:04.867514+0000","last_undegraded":"2024-04-20T10:56:04.867514+0000","last_fullsized":"2024-04-20T10:56:04.867514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797544+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T10:55:43.797544+0000","last_peered":"2024-04-20T10:55:43.797544+0000","last_clean":"2024-04-20T10:55:43.797544+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T10:55:43.797544+0000","last_undegraded":"2024-04-20T10:55:43.797544+0000","last_fullsized":"2024-04-20T10:55:43.797544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864804+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T10:56:04.864804+0000","last_peered":"2024-04-20T10:56:04.864804+0000","last_clean":"2024-04-20T10:56:04.864804+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T10:56:04.864804+0000","last_undegraded":"2024-04-20T10:56:04.864804+0000","last_fullsized":"2024-04-20T10:56:04.864804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287062+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T10:55:44.287062+0000","last_peered":"2024-04-20T10:55:44.287062+0000","last_clean":"2024-04-20T10:55:44.287062+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T10:55:44.287062+0000","last_undegraded":"2024-04-20T10:55:44.287062+0000","last_fullsized":"2024-04-20T10:55:44.287062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.877295+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T10:56:10.877295+0000","last_peered":"2024-04-20T10:56:10.877295+0000","last_clean":"2024-04-20T10:56:10.877295+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T10:56:10.877295+0000","last_undegraded":"2024-04-20T10:56:10.877295+0000","last_fullsized":"2024-04-20T10:56:10.877295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285669+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T10:55:44.285669+0000","last_peered":"2024-04-20T10:55:44.285669+0000","last_clean":"2024-04-20T10:55:44.285669+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T10:55:44.285669+0000","last_undegraded":"2024-04-20T10:55:44.285669+0000","last_fullsized":"2024-04-20T10:55:44.285669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.828790+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T10:55:52.828790+0000","last_peered":"2024-04-20T10:55:52.828790+0000","last_clean":"2024-04-20T10:55:52.828790+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T10:55:52.828790+0000","last_undegraded":"2024-04-20T10:55:52.828790+0000","last_fullsized":"2024-04-20T10:55:52.828790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801052+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T10:55:43.801052+0000","last_peered":"2024-04-20T10:55:43.801052+0000","last_clean":"2024-04-20T10:55:43.801052+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T10:55:43.801052+0000","last_undegraded":"2024-04-20T10:55:43.801052+0000","last_fullsized":"2024-04-20T10:55:43.801052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:19.323845+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T10:56:19.323845+0000","last_peered":"2024-04-20T10:56:19.323845+0000","last_clean":"2024-04-20T10:56:19.323845+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T10:56:19.323845+0000","last_undegraded":"2024-04-20T10:56:19.323845+0000","last_fullsized":"2024-04-20T10:56:19.323845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287336+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T10:55:44.287336+0000","last_peered":"2024-04-20T10:55:44.287336+0000","last_clean":"2024-04-20T10:55:44.287336+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T10:55:44.287336+0000","last_undegraded":"2024-04-20T10:55:44.287336+0000","last_fullsized":"2024-04-20T10:55:44.287336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.873583+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T10:56:10.873583+0000","last_peered":"2024-04-20T10:56:10.873583+0000","last_clean":"2024-04-20T10:56:10.873583+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T10:56:10.873583+0000","last_undegraded":"2024-04-20T10:56:10.873583+0000","last_fullsized":"2024-04-20T10:56:10.873583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285622+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T10:55:44.285622+0000","last_peered":"2024-04-20T10:55:44.285622+0000","last_clean":"2024-04-20T10:55:44.285622+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T10:55:44.285622+0000","last_undegraded":"2024-04-20T10:55:44.285622+0000","last_fullsized":"2024-04-20T10:55:44.285622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:05.462951+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T10:56:05.462951+0000","last_peered":"2024-04-20T10:56:05.462951+0000","last_clean":"2024-04-20T10:56:05.462951+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T10:56:05.462951+0000","last_undegraded":"2024-04-20T10:56:05.462951+0000","last_fullsized":"2024-04-20T10:56:05.462951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286631+0000","last_change":"2024-04-20T10:55:42.805169+0000","last_active":"2024-04-20T10:55:44.286631+0000","last_peered":"2024-04-20T10:55:44.286631+0000","last_clean":"2024-04-20T10:55:44.286631+0000","last_became_active":"2024-04-20T10:55:42.804573+0000","last_became_peered":"2024-04-20T10:55:42.804573+0000","last_unstale":"2024-04-20T10:55:44.286631+0000","last_undegraded":"2024-04-20T10:55:44.286631+0000","last_fullsized":"2024-04-20T10:55:44.286631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:16.103447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864036+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T10:56:04.864036+0000","last_peered":"2024-04-20T10:56:04.864036+0000","last_clean":"2024-04-20T10:56:04.864036+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T10:56:04.864036+0000","last_undegraded":"2024-04-20T10:56:04.864036+0000","last_fullsized":"2024-04-20T10:56:04.864036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285762+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T10:55:44.285762+0000","last_peered":"2024-04-20T10:55:44.285762+0000","last_clean":"2024-04-20T10:55:44.285762+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T10:55:44.285762+0000","last_undegraded":"2024-04-20T10:55:44.285762+0000","last_fullsized":"2024-04-20T10:55:44.285762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864145+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T10:56:04.864145+0000","last_peered":"2024-04-20T10:56:04.864145+0000","last_clean":"2024-04-20T10:56:04.864145+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T10:56:04.864145+0000","last_undegraded":"2024-04-20T10:56:04.864145+0000","last_fullsized":"2024-04-20T10:56:04.864145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287518+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T10:55:44.287518+0000","last_peered":"2024-04-20T10:55:44.287518+0000","last_clean":"2024-04-20T10:55:44.287518+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T10:55:44.287518+0000","last_undegraded":"2024-04-20T10:55:44.287518+0000","last_fullsized":"2024-04-20T10:55:44.287518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.786618+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T10:55:47.786618+0000","last_peered":"2024-04-20T10:55:47.786618+0000","last_clean":"2024-04-20T10:55:47.786618+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T10:55:47.786618+0000","last_undegraded":"2024-04-20T10:55:47.786618+0000","last_fullsized":"2024-04-20T10:55:47.786618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286460+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T10:55:44.286460+0000","last_peered":"2024-04-20T10:55:44.286460+0000","last_clean":"2024-04-20T10:55:44.286460+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T10:55:44.286460+0000","last_undegraded":"2024-04-20T10:55:44.286460+0000","last_fullsized":"2024-04-20T10:55:44.286460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.873255+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T10:56:10.873255+0000","last_peered":"2024-04-20T10:56:10.873255+0000","last_clean":"2024-04-20T10:56:10.873255+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T10:56:10.873255+0000","last_undegraded":"2024-04-20T10:56:10.873255+0000","last_fullsized":"2024-04-20T10:56:10.873255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287067+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T10:55:44.287067+0000","last_peered":"2024-04-20T10:55:44.287067+0000","last_clean":"2024-04-20T10:55:44.287067+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T10:55:44.287067+0000","last_undegraded":"2024-04-20T10:55:44.287067+0000","last_fullsized":"2024-04-20T10:55:44.287067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864274+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T10:56:04.864274+0000","last_peered":"2024-04-20T10:56:04.864274+0000","last_clean":"2024-04-20T10:56:04.864274+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T10:56:04.864274+0000","last_undegraded":"2024-04-20T10:56:04.864274+0000","last_fullsized":"2024-04-20T10:56:04.864274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286721+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T10:55:44.286721+0000","last_peered":"2024-04-20T10:55:44.286721+0000","last_clean":"2024-04-20T10:55:44.286721+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T10:55:44.286721+0000","last_undegraded":"2024-04-20T10:55:44.286721+0000","last_fullsized":"2024-04-20T10:55:44.286721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864812+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T10:56:04.864812+0000","last_peered":"2024-04-20T10:56:04.864812+0000","last_clean":"2024-04-20T10:56:04.864812+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T10:56:04.864812+0000","last_undegraded":"2024-04-20T10:56:04.864812+0000","last_fullsized":"2024-04-20T10:56:04.864812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286934+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T10:55:44.286934+0000","last_peered":"2024-04-20T10:55:44.286934+0000","last_clean":"2024-04-20T10:55:44.286934+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T10:55:44.286934+0000","last_undegraded":"2024-04-20T10:55:44.286934+0000","last_fullsized":"2024-04-20T10:55:44.286934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864501+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T10:56:04.864501+0000","last_peered":"2024-04-20T10:56:04.864501+0000","last_clean":"2024-04-20T10:56:04.864501+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T10:56:04.864501+0000","last_undegraded":"2024-04-20T10:56:04.864501+0000","last_fullsized":"2024-04-20T10:56:04.864501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793252+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T10:55:43.793252+0000","last_peered":"2024-04-20T10:55:43.793252+0000","last_clean":"2024-04-20T10:55:43.793252+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T10:55:43.793252+0000","last_undegraded":"2024-04-20T10:55:43.793252+0000","last_fullsized":"2024-04-20T10:55:43.793252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:58.848667+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T10:55:58.848667+0000","last_peered":"2024-04-20T10:55:58.848667+0000","last_clean":"2024-04-20T10:55:58.848667+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T10:55:58.848667+0000","last_undegraded":"2024-04-20T10:55:58.848667+0000","last_fullsized":"2024-04-20T10:55:58.848667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285651+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T10:55:44.285651+0000","last_peered":"2024-04-20T10:55:44.285651+0000","last_clean":"2024-04-20T10:55:44.285651+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T10:55:44.285651+0000","last_undegraded":"2024-04-20T10:55:44.285651+0000","last_fullsized":"2024-04-20T10:55:44.285651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:58.848610+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T10:55:58.848610+0000","last_peered":"2024-04-20T10:55:58.848610+0000","last_clean":"2024-04-20T10:55:58.848610+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T10:55:58.848610+0000","last_undegraded":"2024-04-20T10:55:58.848610+0000","last_fullsized":"2024-04-20T10:55:58.848610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797319+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T10:55:43.797319+0000","last_peered":"2024-04-20T10:55:43.797319+0000","last_clean":"2024-04-20T10:55:43.797319+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T10:55:43.797319+0000","last_undegraded":"2024-04-20T10:55:43.797319+0000","last_fullsized":"2024-04-20T10:55:43.797319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876180+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T10:56:10.876180+0000","last_peered":"2024-04-20T10:56:10.876180+0000","last_clean":"2024-04-20T10:56:10.876180+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T10:56:10.876180+0000","last_undegraded":"2024-04-20T10:56:10.876180+0000","last_fullsized":"2024-04-20T10:56:10.876180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800914+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T10:55:43.800914+0000","last_peered":"2024-04-20T10:55:43.800914+0000","last_clean":"2024-04-20T10:55:43.800914+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T10:55:43.800914+0000","last_undegraded":"2024-04-20T10:55:43.800914+0000","last_fullsized":"2024-04-20T10:55:43.800914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.731030+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T10:55:44.731030+0000","last_peered":"2024-04-20T10:55:44.731030+0000","last_clean":"2024-04-20T10:55:44.731030+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T10:55:44.731030+0000","last_undegraded":"2024-04-20T10:55:44.731030+0000","last_fullsized":"2024-04-20T10:55:44.731030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287087+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T10:55:44.287087+0000","last_peered":"2024-04-20T10:55:44.287087+0000","last_clean":"2024-04-20T10:55:44.287087+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T10:55:44.287087+0000","last_undegraded":"2024-04-20T10:55:44.287087+0000","last_fullsized":"2024-04-20T10:55:44.287087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864814+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T10:56:04.864814+0000","last_peered":"2024-04-20T10:56:04.864814+0000","last_clean":"2024-04-20T10:56:04.864814+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T10:56:04.864814+0000","last_undegraded":"2024-04-20T10:56:04.864814+0000","last_fullsized":"2024-04-20T10:56:04.864814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797411+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T10:55:43.797411+0000","last_peered":"2024-04-20T10:55:43.797411+0000","last_clean":"2024-04-20T10:55:43.797411+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T10:55:43.797411+0000","last_undegraded":"2024-04-20T10:55:43.797411+0000","last_fullsized":"2024-04-20T10:55:43.797411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.867493+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T10:56:04.867493+0000","last_peered":"2024-04-20T10:56:04.867493+0000","last_clean":"2024-04-20T10:56:04.867493+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T10:56:04.867493+0000","last_undegraded":"2024-04-20T10:56:04.867493+0000","last_fullsized":"2024-04-20T10:56:04.867493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793562+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T10:55:43.793562+0000","last_peered":"2024-04-20T10:55:43.793562+0000","last_clean":"2024-04-20T10:55:43.793562+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T10:55:43.793562+0000","last_undegraded":"2024-04-20T10:55:43.793562+0000","last_fullsized":"2024-04-20T10:55:43.793562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.873382+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T10:56:10.873382+0000","last_peered":"2024-04-20T10:56:10.873382+0000","last_clean":"2024-04-20T10:56:10.873382+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T10:56:10.873382+0000","last_undegraded":"2024-04-20T10:56:10.873382+0000","last_fullsized":"2024-04-20T10:56:10.873382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285397+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T10:55:44.285397+0000","last_peered":"2024-04-20T10:55:44.285397+0000","last_clean":"2024-04-20T10:55:44.285397+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T10:55:44.285397+0000","last_undegraded":"2024-04-20T10:55:44.285397+0000","last_fullsized":"2024-04-20T10:55:44.285397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.825832+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T10:55:52.825832+0000","last_peered":"2024-04-20T10:55:52.825832+0000","last_clean":"2024-04-20T10:55:52.825832+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T10:55:52.825832+0000","last_undegraded":"2024-04-20T10:55:52.825832+0000","last_fullsized":"2024-04-20T10:55:52.825832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800994+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T10:55:43.800994+0000","last_peered":"2024-04-20T10:55:43.800994+0000","last_clean":"2024-04-20T10:55:43.800994+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T10:55:43.800994+0000","last_undegraded":"2024-04-20T10:55:43.800994+0000","last_fullsized":"2024-04-20T10:55:43.800994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876458+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T10:56:10.876458+0000","last_peered":"2024-04-20T10:56:10.876458+0000","last_clean":"2024-04-20T10:56:10.876458+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T10:56:10.876458+0000","last_undegraded":"2024-04-20T10:56:10.876458+0000","last_fullsized":"2024-04-20T10:56:10.876458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793024+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T10:55:43.793024+0000","last_peered":"2024-04-20T10:55:43.793024+0000","last_clean":"2024-04-20T10:55:43.793024+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T10:55:43.793024+0000","last_undegraded":"2024-04-20T10:55:43.793024+0000","last_fullsized":"2024-04-20T10:55:43.793024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:05.384095+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T10:56:05.384095+0000","last_peered":"2024-04-20T10:56:05.384095+0000","last_clean":"2024-04-20T10:56:05.384095+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T10:56:05.384095+0000","last_undegraded":"2024-04-20T10:56:05.384095+0000","last_fullsized":"2024-04-20T10:56:05.384095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793105+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T10:55:43.793105+0000","last_peered":"2024-04-20T10:55:43.793105+0000","last_clean":"2024-04-20T10:55:43.793105+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T10:55:43.793105+0000","last_undegraded":"2024-04-20T10:55:43.793105+0000","last_fullsized":"2024-04-20T10:55:43.793105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.825429+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T10:55:52.825429+0000","last_peered":"2024-04-20T10:55:52.825429+0000","last_clean":"2024-04-20T10:55:52.825429+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T10:55:52.825429+0000","last_undegraded":"2024-04-20T10:55:52.825429+0000","last_fullsized":"2024-04-20T10:55:52.825429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286369+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T10:55:44.286369+0000","last_peered":"2024-04-20T10:55:44.286369+0000","last_clean":"2024-04-20T10:55:44.286369+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T10:55:44.286369+0000","last_undegraded":"2024-04-20T10:55:44.286369+0000","last_fullsized":"2024-04-20T10:55:44.286369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286936+0000","last_change":"2024-04-20T10:55:42.792905+0000","last_active":"2024-04-20T10:55:44.286936+0000","last_peered":"2024-04-20T10:55:44.286936+0000","last_clean":"2024-04-20T10:55:44.286936+0000","last_became_active":"2024-04-20T10:55:42.792587+0000","last_became_peered":"2024-04-20T10:55:42.792587+0000","last_unstale":"2024-04-20T10:55:44.286936+0000","last_undegraded":"2024-04-20T10:55:44.286936+0000","last_fullsized":"2024-04-20T10:55:44.286936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:02.987448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876702+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T10:56:10.876702+0000","last_peered":"2024-04-20T10:56:10.876702+0000","last_clean":"2024-04-20T10:56:10.876702+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T10:56:10.876702+0000","last_undegraded":"2024-04-20T10:56:10.876702+0000","last_fullsized":"2024-04-20T10:56:10.876702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286560+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T10:55:44.286560+0000","last_peered":"2024-04-20T10:55:44.286560+0000","last_clean":"2024-04-20T10:55:44.286560+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T10:55:44.286560+0000","last_undegraded":"2024-04-20T10:55:44.286560+0000","last_fullsized":"2024-04-20T10:55:44.286560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.877051+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T10:56:10.877051+0000","last_peered":"2024-04-20T10:56:10.877051+0000","last_clean":"2024-04-20T10:56:10.877051+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T10:56:10.877051+0000","last_undegraded":"2024-04-20T10:56:10.877051+0000","last_fullsized":"2024-04-20T10:56:10.877051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.788752+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T10:55:47.788752+0000","last_peered":"2024-04-20T10:55:47.788752+0000","last_clean":"2024-04-20T10:55:47.788752+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T10:55:47.788752+0000","last_undegraded":"2024-04-20T10:55:47.788752+0000","last_fullsized":"2024-04-20T10:55:47.788752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793370+0000","last_change":"2024-04-20T10:55:42.802227+0000","last_active":"2024-04-20T10:55:43.793370+0000","last_peered":"2024-04-20T10:55:43.793370+0000","last_clean":"2024-04-20T10:55:43.793370+0000","last_became_active":"2024-04-20T10:55:42.801944+0000","last_became_peered":"2024-04-20T10:55:42.801944+0000","last_unstale":"2024-04-20T10:55:43.793370+0000","last_undegraded":"2024-04-20T10:55:43.793370+0000","last_fullsized":"2024-04-20T10:55:43.793370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:29.862643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876246+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T10:56:10.876246+0000","last_peered":"2024-04-20T10:56:10.876246+0000","last_clean":"2024-04-20T10:56:10.876246+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T10:56:10.876246+0000","last_undegraded":"2024-04-20T10:56:10.876246+0000","last_fullsized":"2024-04-20T10:56:10.876246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8546,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":209,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":17092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:56:22.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:56:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.027+0000 7f34ad34a640 1 Processor -- start 2024-04-20T10:56:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.027+0000 7f34ad34a640 1 -- start start 2024-04-20T10:56:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34ad34a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 0x7f34a80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34ad34a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a80b5e70 con 0x7f34a80d0c90 2024-04-20T10:56:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34ad34a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a815b9c0 con 0x7f34a8155950 2024-04-20T10:56:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34ad34a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f34a815bca0 con 0x7f34a80b54f0 2024-04-20T10:56:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34a77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f34a80d0c90 0x7f34a80d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37416/0 (socket says 172.21.15.27:37416) 2024-04-20T10:56:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34a77fe640 1 -- 172.21.15.27:0/1014818233 learned_addr learned my addr 172.21.15.27:0/1014818233 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34a67fc640 1 --2- 172.21.15.27:0/1014818233 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 0x7f34a80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34a5ffb640 1 -- 172.21.15.27:0/1014818233 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356935701 0 0) 0x7f34a80b5e70 con 0x7f34a80d0c90 2024-04-20T10:56:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34a5ffb640 1 -- 172.21.15.27:0/1014818233 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3494003520 con 0x7f34a80d0c90 2024-04-20T10:56:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.028+0000 7f34a5ffb640 1 -- 172.21.15.27:0/1014818233 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 948232509 0 0) 0x7f34a815b9c0 con 0x7f34a8155950 2024-04-20T10:56:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.029+0000 7f34a5ffb640 1 -- 172.21.15.27:0/1014818233 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34a80b5e70 con 0x7f34a8155950 2024-04-20T10:56:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.029+0000 7f34a67fc640 1 -- 172.21.15.27:0/1014818233 >> v1:172.21.15.137:6789/0 conn(0x7f34a8155950 legacy=0x7f34a80cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.029+0000 7f34a5ffb640 1 -- 172.21.15.27:0/1014818233 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2507434661 0 0) 0x7f3494003520 con 0x7f34a80d0c90 2024-04-20T10:56:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.029+0000 7f34a67fc640 1 -- 172.21.15.27:0/1014818233 >> v1:172.21.15.27:6789/0 conn(0x7f34a80d0c90 legacy=0x7f34a80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.029+0000 7f34a67fc640 1 -- 172.21.15.27:0/1014818233 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a815bf80 con 0x7f34a80b54f0 2024-04-20T10:56:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.029+0000 7f34a67fc640 1 --2- 172.21.15.27:0/1014818233 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 0x7f34a80b5930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f34900097d0 tx=0x7f349002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=190829bbe9b79fac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34a5ffb640 1 -- 172.21.15.27:0/1014818233 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980019f0 con 0x7f34a80b54f0 2024-04-20T10:56:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34a5ffb640 1 -- 172.21.15.27:0/1014818233 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3490037070 con 0x7f34a80b54f0 2024-04-20T10:56:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34a5ffb640 1 -- 172.21.15.27:0/1014818233 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490031780 con 0x7f34a80b54f0 2024-04-20T10:56:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34ad34a640 1 -- 172.21.15.27:0/1014818233 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 msgr2=0x7f34a80b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34ad34a640 1 --2- 172.21.15.27:0/1014818233 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 0x7f34a80b5930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f34900097d0 tx=0x7f349002b8a0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34ad34a640 1 -- 172.21.15.27:0/1014818233 shutdown_connections 2024-04-20T10:56:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34ad34a640 1 --2- 172.21.15.27:0/1014818233 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 0x7f34a80b5930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34ad34a640 1 -- 172.21.15.27:0/1014818233 >> 172.21.15.27:0/1014818233 conn(0x7f34a80676c0 msgr2=0x7f34a8067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34ad34a640 1 -- 172.21.15.27:0/1014818233 shutdown_connections 2024-04-20T10:56:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.030+0000 7f34ad34a640 1 -- 172.21.15.27:0/1014818233 wait complete. 2024-04-20T10:56:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.031+0000 7f34ad34a640 1 Processor -- start 2024-04-20T10:56:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34ad34a640 1 -- start start 2024-04-20T10:56:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34ad34a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 0x7f34a80c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34ad34a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a80d0c90 0x7f34a80c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34ad34a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a8155950 0x7f34a81508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34ad34a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34a815fb90 con 0x7f34a8155950 2024-04-20T10:56:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34ad34a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34a815c540 con 0x7f34a80d0c90 2024-04-20T10:56:23.034 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34ad34a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f34a815fd10 con 0x7f34a80b54f0 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a8155950 0x7f34a81508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34a77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a8155950 0x7f34a81508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35592/0 (socket says 172.21.15.27:35592) 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.032+0000 7f34a77fe640 1 -- 172.21.15.27:0/2919087579 learned_addr learned my addr 172.21.15.27:0/2919087579 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34a67fc640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a80d0c90 0x7f34a80c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34a6ffd640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 0x7f34a80c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34a77fe640 1 -- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 msgr2=0x7f34a80c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34a77fe640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 0x7f34a80c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34a77fe640 1 -- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a80d0c90 msgr2=0x7f34a80c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34a77fe640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a80d0c90 0x7f34a80c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34a77fe640 1 -- 172.21.15.27:0/2919087579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a80c7a40 con 0x7f34a8155950 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34a77fe640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a8155950 0x7f34a81508d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f349800cca0 tx=0x7f349800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f349f7fe640 1 -- 172.21.15.27:0/2919087579 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349800ce50 con 0x7f34a8155950 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f349f7fe640 1 -- 172.21.15.27:0/2919087579 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3498012400 con 0x7f34a8155950 2024-04-20T10:56:23.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a815af70 con 0x7f34a8155950 2024-04-20T10:56:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f349f7fe640 1 -- 172.21.15.27:0/2919087579 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980127d0 con 0x7f34a8155950 2024-04-20T10:56:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.033+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a815b530 con 0x7f34a8155950 2024-04-20T10:56:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.034+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a80686d0 con 0x7f34a8155950 2024-04-20T10:56:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.035+0000 7f349f7fe640 1 -- 172.21.15.27:0/2919087579 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3498012970 con 0x7f34a8155950 2024-04-20T10:56:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.035+0000 7f349f7fe640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f348806cac0 0x7f348806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.035+0000 7f349f7fe640 1 -- 172.21.15.27:0/2919087579 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f349808c2f0 con 0x7f34a8155950 2024-04-20T10:56:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.035+0000 7f34a6ffd640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f348806cac0 0x7f348806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.036+0000 7f34a6ffd640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f348806cac0 0x7f348806ef80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34a0001650 tx=0x7f34a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.039+0000 7f349f7fe640 1 -- 172.21.15.27:0/2919087579 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f34a80686d0 con 0x7f34a8155950 2024-04-20T10:56:23.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.225+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34a815b770 con 0x7f348806cac0 2024-04-20T10:56:23.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.232+0000 7f349f7fe640 1 -- 172.21.15.27:0/2919087579 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f34a815b770 con 0x7f348806cac0 2024-04-20T10:56:23.235 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:23.236 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T10:56:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f348806cac0 msgr2=0x7f348806ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f348806cac0 0x7f348806ef80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34a0001650 tx=0x7f34a0009040 comp rx=0 tx=0).stop 2024-04-20T10:56:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a8155950 msgr2=0x7f34a81508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a8155950 0x7f34a81508d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f349800cca0 tx=0x7f349800a430 comp rx=0 tx=0).stop 2024-04-20T10:56:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 shutdown_connections 2024-04-20T10:56:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f348806cac0 0x7f348806ef80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a8155950 0x7f34a81508d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a80d0c90 0x7f34a80c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 --2- 172.21.15.27:0/2919087579 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f34a80b54f0 0x7f34a80c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 >> 172.21.15.27:0/2919087579 conn(0x7f34a80676c0 msgr2=0x7f34a80cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:23.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 shutdown_connections 2024-04-20T10:56:23.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.235+0000 7f34ad34a640 1 -- 172.21.15.27:0/2919087579 wait complete. 2024-04-20T10:56:23.246 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-04-20T10:56:22.899439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598914,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":368,"num_write_kb":2482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1918,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.190082"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287393+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T10:55:44.287393+0000","last_peered":"2024-04-20T10:55:44.287393+0000","last_clean":"2024-04-20T10:55:44.287393+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T10:55:44.287393+0000","last_undegraded":"2024-04-20T10:55:44.287393+0000","last_fullsized":"2024-04-20T10:55:44.287393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797629+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T10:55:43.797629+0000","last_peered":"2024-04-20T10:55:43.797629+0000","last_clean":"2024-04-20T10:55:43.797629+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T10:55:43.797629+0000","last_undegraded":"2024-04-20T10:55:43.797629+0000","last_fullsized":"2024-04-20T10:55:43.797629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285486+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T10:55:44.285486+0000","last_peered":"2024-04-20T10:55:44.285486+0000","last_clean":"2024-04-20T10:55:44.285486+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T10:55:44.285486+0000","last_undegraded":"2024-04-20T10:55:44.285486+0000","last_fullsized":"2024-04-20T10:55:44.285486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286383+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T10:55:44.286383+0000","last_peered":"2024-04-20T10:55:44.286383+0000","last_clean":"2024-04-20T10:55:44.286383+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T10:55:44.286383+0000","last_undegraded":"2024-04-20T10:55:44.286383+0000","last_fullsized":"2024-04-20T10:55:44.286383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286289+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T10:55:44.286289+0000","last_peered":"2024-04-20T10:55:44.286289+0000","last_clean":"2024-04-20T10:55:44.286289+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T10:55:44.286289+0000","last_undegraded":"2024-04-20T10:55:44.286289+0000","last_fullsized":"2024-04-20T10:55:44.286289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287329+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T10:55:44.287329+0000","last_peered":"2024-04-20T10:55:44.287329+0000","last_clean":"2024-04-20T10:55:44.287329+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T10:55:44.287329+0000","last_undegraded":"2024-04-20T10:55:44.287329+0000","last_fullsized":"2024-04-20T10:55:44.287329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287439+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T10:55:44.287439+0000","last_peered":"2024-04-20T10:55:44.287439+0000","last_clean":"2024-04-20T10:55:44.287439+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T10:55:44.287439+0000","last_undegraded":"2024-04-20T10:55:44.287439+0000","last_fullsized":"2024-04-20T10:55:44.287439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792859+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T10:55:43.792859+0000","last_peered":"2024-04-20T10:55:43.792859+0000","last_clean":"2024-04-20T10:55:43.792859+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T10:55:43.792859+0000","last_undegraded":"2024-04-20T10:55:43.792859+0000","last_fullsized":"2024-04-20T10:55:43.792859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286907+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T10:55:44.286907+0000","last_peered":"2024-04-20T10:55:44.286907+0000","last_clean":"2024-04-20T10:55:44.286907+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T10:55:44.286907+0000","last_undegraded":"2024-04-20T10:55:44.286907+0000","last_fullsized":"2024-04-20T10:55:44.286907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286541+0000","last_change":"2024-04-20T10:55:42.802716+0000","last_active":"2024-04-20T10:55:44.286541+0000","last_peered":"2024-04-20T10:55:44.286541+0000","last_clean":"2024-04-20T10:55:44.286541+0000","last_became_active":"2024-04-20T10:55:42.802418+0000","last_became_peered":"2024-04-20T10:55:42.802418+0000","last_unstale":"2024-04-20T10:55:44.286541+0000","last_undegraded":"2024-04-20T10:55:44.286541+0000","last_fullsized":"2024-04-20T10:55:44.286541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:12:22.186243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286228+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T10:55:44.286228+0000","last_peered":"2024-04-20T10:55:44.286228+0000","last_clean":"2024-04-20T10:55:44.286228+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T10:55:44.286228+0000","last_undegraded":"2024-04-20T10:55:44.286228+0000","last_fullsized":"2024-04-20T10:55:44.286228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801002+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T10:55:43.801002+0000","last_peered":"2024-04-20T10:55:43.801002+0000","last_clean":"2024-04-20T10:55:43.801002+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T10:55:43.801002+0000","last_undegraded":"2024-04-20T10:55:43.801002+0000","last_fullsized":"2024-04-20T10:55:43.801002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793050+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T10:55:43.793050+0000","last_peered":"2024-04-20T10:55:43.793050+0000","last_clean":"2024-04-20T10:55:43.793050+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T10:55:43.793050+0000","last_undegraded":"2024-04-20T10:55:43.793050+0000","last_fullsized":"2024-04-20T10:55:43.793050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797332+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T10:55:43.797332+0000","last_peered":"2024-04-20T10:55:43.797332+0000","last_clean":"2024-04-20T10:55:43.797332+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T10:55:43.797332+0000","last_undegraded":"2024-04-20T10:55:43.797332+0000","last_fullsized":"2024-04-20T10:55:43.797332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285851+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T10:55:44.285851+0000","last_peered":"2024-04-20T10:55:44.285851+0000","last_clean":"2024-04-20T10:55:44.285851+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T10:55:44.285851+0000","last_undegraded":"2024-04-20T10:55:44.285851+0000","last_fullsized":"2024-04-20T10:55:44.285851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286491+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T10:55:44.286491+0000","last_peered":"2024-04-20T10:55:44.286491+0000","last_clean":"2024-04-20T10:55:44.286491+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T10:55:44.286491+0000","last_undegraded":"2024-04-20T10:55:44.286491+0000","last_fullsized":"2024-04-20T10:55:44.286491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285217+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T10:55:44.285217+0000","last_peered":"2024-04-20T10:55:44.285217+0000","last_clean":"2024-04-20T10:55:44.285217+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T10:55:44.285217+0000","last_undegraded":"2024-04-20T10:55:44.285217+0000","last_fullsized":"2024-04-20T10:55:44.285217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287175+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T10:55:44.287175+0000","last_peered":"2024-04-20T10:55:44.287175+0000","last_clean":"2024-04-20T10:55:44.287175+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T10:55:44.287175+0000","last_undegraded":"2024-04-20T10:55:44.287175+0000","last_fullsized":"2024-04-20T10:55:44.287175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286994+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T10:55:44.286994+0000","last_peered":"2024-04-20T10:55:44.286994+0000","last_clean":"2024-04-20T10:55:44.286994+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T10:55:44.286994+0000","last_undegraded":"2024-04-20T10:55:44.286994+0000","last_fullsized":"2024-04-20T10:55:44.286994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285751+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T10:55:44.285751+0000","last_peered":"2024-04-20T10:55:44.285751+0000","last_clean":"2024-04-20T10:55:44.285751+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T10:55:44.285751+0000","last_undegraded":"2024-04-20T10:55:44.285751+0000","last_fullsized":"2024-04-20T10:55:44.285751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287484+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T10:55:44.287484+0000","last_peered":"2024-04-20T10:55:44.287484+0000","last_clean":"2024-04-20T10:55:44.287484+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T10:55:44.287484+0000","last_undegraded":"2024-04-20T10:55:44.287484+0000","last_fullsized":"2024-04-20T10:55:44.287484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792921+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T10:55:43.792921+0000","last_peered":"2024-04-20T10:55:43.792921+0000","last_clean":"2024-04-20T10:55:43.792921+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T10:55:43.792921+0000","last_undegraded":"2024-04-20T10:55:43.792921+0000","last_fullsized":"2024-04-20T10:55:43.792921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793499+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T10:55:43.793499+0000","last_peered":"2024-04-20T10:55:43.793499+0000","last_clean":"2024-04-20T10:55:43.793499+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T10:55:43.793499+0000","last_undegraded":"2024-04-20T10:55:43.793499+0000","last_fullsized":"2024-04-20T10:55:43.793499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286683+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T10:55:44.286683+0000","last_peered":"2024-04-20T10:55:44.286683+0000","last_clean":"2024-04-20T10:55:44.286683+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T10:55:44.286683+0000","last_undegraded":"2024-04-20T10:55:44.286683+0000","last_fullsized":"2024-04-20T10:55:44.286683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797439+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T10:55:43.797439+0000","last_peered":"2024-04-20T10:55:43.797439+0000","last_clean":"2024-04-20T10:55:43.797439+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T10:55:43.797439+0000","last_undegraded":"2024-04-20T10:55:43.797439+0000","last_fullsized":"2024-04-20T10:55:43.797439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793232+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T10:55:43.793232+0000","last_peered":"2024-04-20T10:55:43.793232+0000","last_clean":"2024-04-20T10:55:43.793232+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T10:55:43.793232+0000","last_undegraded":"2024-04-20T10:55:43.793232+0000","last_fullsized":"2024-04-20T10:55:43.793232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285757+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T10:55:44.285757+0000","last_peered":"2024-04-20T10:55:44.285757+0000","last_clean":"2024-04-20T10:55:44.285757+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T10:55:44.285757+0000","last_undegraded":"2024-04-20T10:55:44.285757+0000","last_fullsized":"2024-04-20T10:55:44.285757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287493+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T10:55:44.287493+0000","last_peered":"2024-04-20T10:55:44.287493+0000","last_clean":"2024-04-20T10:55:44.287493+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T10:55:44.287493+0000","last_undegraded":"2024-04-20T10:55:44.287493+0000","last_fullsized":"2024-04-20T10:55:44.287493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285309+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T10:55:44.285309+0000","last_peered":"2024-04-20T10:55:44.285309+0000","last_clean":"2024-04-20T10:55:44.285309+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T10:55:44.285309+0000","last_undegraded":"2024-04-20T10:55:44.285309+0000","last_fullsized":"2024-04-20T10:55:44.285309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285746+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T10:55:44.285746+0000","last_peered":"2024-04-20T10:55:44.285746+0000","last_clean":"2024-04-20T10:55:44.285746+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T10:55:44.285746+0000","last_undegraded":"2024-04-20T10:55:44.285746+0000","last_fullsized":"2024-04-20T10:55:44.285746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800661+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T10:55:43.800661+0000","last_peered":"2024-04-20T10:55:43.800661+0000","last_clean":"2024-04-20T10:55:43.800661+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T10:55:43.800661+0000","last_undegraded":"2024-04-20T10:55:43.800661+0000","last_fullsized":"2024-04-20T10:55:43.800661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.831445+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T10:55:52.831445+0000","last_peered":"2024-04-20T10:55:52.831445+0000","last_clean":"2024-04-20T10:55:52.831445+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T10:55:52.831445+0000","last_undegraded":"2024-04-20T10:55:52.831445+0000","last_fullsized":"2024-04-20T10:55:52.831445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285840+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T10:55:44.285840+0000","last_peered":"2024-04-20T10:55:44.285840+0000","last_clean":"2024-04-20T10:55:44.285840+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T10:55:44.285840+0000","last_undegraded":"2024-04-20T10:55:44.285840+0000","last_fullsized":"2024-04-20T10:55:44.285840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864139+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T10:56:04.864139+0000","last_peered":"2024-04-20T10:56:04.864139+0000","last_clean":"2024-04-20T10:56:04.864139+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T10:56:04.864139+0000","last_undegraded":"2024-04-20T10:56:04.864139+0000","last_fullsized":"2024-04-20T10:56:04.864139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:58.849736+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T10:55:58.849736+0000","last_peered":"2024-04-20T10:55:58.849736+0000","last_clean":"2024-04-20T10:55:58.849736+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T10:55:58.849736+0000","last_undegraded":"2024-04-20T10:55:58.849736+0000","last_fullsized":"2024-04-20T10:55:58.849736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792877+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T10:55:43.792877+0000","last_peered":"2024-04-20T10:55:43.792877+0000","last_clean":"2024-04-20T10:55:43.792877+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T10:55:43.792877+0000","last_undegraded":"2024-04-20T10:55:43.792877+0000","last_fullsized":"2024-04-20T10:55:43.792877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.867514+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T10:56:04.867514+0000","last_peered":"2024-04-20T10:56:04.867514+0000","last_clean":"2024-04-20T10:56:04.867514+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T10:56:04.867514+0000","last_undegraded":"2024-04-20T10:56:04.867514+0000","last_fullsized":"2024-04-20T10:56:04.867514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797544+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T10:55:43.797544+0000","last_peered":"2024-04-20T10:55:43.797544+0000","last_clean":"2024-04-20T10:55:43.797544+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T10:55:43.797544+0000","last_undegraded":"2024-04-20T10:55:43.797544+0000","last_fullsized":"2024-04-20T10:55:43.797544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864804+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T10:56:04.864804+0000","last_peered":"2024-04-20T10:56:04.864804+0000","last_clean":"2024-04-20T10:56:04.864804+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T10:56:04.864804+0000","last_undegraded":"2024-04-20T10:56:04.864804+0000","last_fullsized":"2024-04-20T10:56:04.864804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287062+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T10:55:44.287062+0000","last_peered":"2024-04-20T10:55:44.287062+0000","last_clean":"2024-04-20T10:55:44.287062+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T10:55:44.287062+0000","last_undegraded":"2024-04-20T10:55:44.287062+0000","last_fullsized":"2024-04-20T10:55:44.287062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.877295+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T10:56:10.877295+0000","last_peered":"2024-04-20T10:56:10.877295+0000","last_clean":"2024-04-20T10:56:10.877295+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T10:56:10.877295+0000","last_undegraded":"2024-04-20T10:56:10.877295+0000","last_fullsized":"2024-04-20T10:56:10.877295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285669+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T10:55:44.285669+0000","last_peered":"2024-04-20T10:55:44.285669+0000","last_clean":"2024-04-20T10:55:44.285669+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T10:55:44.285669+0000","last_undegraded":"2024-04-20T10:55:44.285669+0000","last_fullsized":"2024-04-20T10:55:44.285669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.828790+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T10:55:52.828790+0000","last_peered":"2024-04-20T10:55:52.828790+0000","last_clean":"2024-04-20T10:55:52.828790+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T10:55:52.828790+0000","last_undegraded":"2024-04-20T10:55:52.828790+0000","last_fullsized":"2024-04-20T10:55:52.828790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801052+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T10:55:43.801052+0000","last_peered":"2024-04-20T10:55:43.801052+0000","last_clean":"2024-04-20T10:55:43.801052+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T10:55:43.801052+0000","last_undegraded":"2024-04-20T10:55:43.801052+0000","last_fullsized":"2024-04-20T10:55:43.801052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:19.323845+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T10:56:19.323845+0000","last_peered":"2024-04-20T10:56:19.323845+0000","last_clean":"2024-04-20T10:56:19.323845+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T10:56:19.323845+0000","last_undegraded":"2024-04-20T10:56:19.323845+0000","last_fullsized":"2024-04-20T10:56:19.323845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287336+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T10:55:44.287336+0000","last_peered":"2024-04-20T10:55:44.287336+0000","last_clean":"2024-04-20T10:55:44.287336+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T10:55:44.287336+0000","last_undegraded":"2024-04-20T10:55:44.287336+0000","last_fullsized":"2024-04-20T10:55:44.287336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.873583+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T10:56:10.873583+0000","last_peered":"2024-04-20T10:56:10.873583+0000","last_clean":"2024-04-20T10:56:10.873583+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T10:56:10.873583+0000","last_undegraded":"2024-04-20T10:56:10.873583+0000","last_fullsized":"2024-04-20T10:56:10.873583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285622+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T10:55:44.285622+0000","last_peered":"2024-04-20T10:55:44.285622+0000","last_clean":"2024-04-20T10:55:44.285622+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T10:55:44.285622+0000","last_undegraded":"2024-04-20T10:55:44.285622+0000","last_fullsized":"2024-04-20T10:55:44.285622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:05.462951+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T10:56:05.462951+0000","last_peered":"2024-04-20T10:56:05.462951+0000","last_clean":"2024-04-20T10:56:05.462951+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T10:56:05.462951+0000","last_undegraded":"2024-04-20T10:56:05.462951+0000","last_fullsized":"2024-04-20T10:56:05.462951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286631+0000","last_change":"2024-04-20T10:55:42.805169+0000","last_active":"2024-04-20T10:55:44.286631+0000","last_peered":"2024-04-20T10:55:44.286631+0000","last_clean":"2024-04-20T10:55:44.286631+0000","last_became_active":"2024-04-20T10:55:42.804573+0000","last_became_peered":"2024-04-20T10:55:42.804573+0000","last_unstale":"2024-04-20T10:55:44.286631+0000","last_undegraded":"2024-04-20T10:55:44.286631+0000","last_fullsized":"2024-04-20T10:55:44.286631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:16.103447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864036+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T10:56:04.864036+0000","last_peered":"2024-04-20T10:56:04.864036+0000","last_clean":"2024-04-20T10:56:04.864036+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T10:56:04.864036+0000","last_undegraded":"2024-04-20T10:56:04.864036+0000","last_fullsized":"2024-04-20T10:56:04.864036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285762+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T10:55:44.285762+0000","last_peered":"2024-04-20T10:55:44.285762+0000","last_clean":"2024-04-20T10:55:44.285762+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T10:55:44.285762+0000","last_undegraded":"2024-04-20T10:55:44.285762+0000","last_fullsized":"2024-04-20T10:55:44.285762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864145+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T10:56:04.864145+0000","last_peered":"2024-04-20T10:56:04.864145+0000","last_clean":"2024-04-20T10:56:04.864145+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T10:56:04.864145+0000","last_undegraded":"2024-04-20T10:56:04.864145+0000","last_fullsized":"2024-04-20T10:56:04.864145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287518+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T10:55:44.287518+0000","last_peered":"2024-04-20T10:55:44.287518+0000","last_clean":"2024-04-20T10:55:44.287518+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T10:55:44.287518+0000","last_undegraded":"2024-04-20T10:55:44.287518+0000","last_fullsized":"2024-04-20T10:55:44.287518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.786618+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T10:55:47.786618+0000","last_peered":"2024-04-20T10:55:47.786618+0000","last_clean":"2024-04-20T10:55:47.786618+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T10:55:47.786618+0000","last_undegraded":"2024-04-20T10:55:47.786618+0000","last_fullsized":"2024-04-20T10:55:47.786618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286460+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T10:55:44.286460+0000","last_peered":"2024-04-20T10:55:44.286460+0000","last_clean":"2024-04-20T10:55:44.286460+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T10:55:44.286460+0000","last_undegraded":"2024-04-20T10:55:44.286460+0000","last_fullsized":"2024-04-20T10:55:44.286460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.873255+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T10:56:10.873255+0000","last_peered":"2024-04-20T10:56:10.873255+0000","last_clean":"2024-04-20T10:56:10.873255+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T10:56:10.873255+0000","last_undegraded":"2024-04-20T10:56:10.873255+0000","last_fullsized":"2024-04-20T10:56:10.873255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287067+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T10:55:44.287067+0000","last_peered":"2024-04-20T10:55:44.287067+0000","last_clean":"2024-04-20T10:55:44.287067+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T10:55:44.287067+0000","last_undegraded":"2024-04-20T10:55:44.287067+0000","last_fullsized":"2024-04-20T10:55:44.287067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864274+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T10:56:04.864274+0000","last_peered":"2024-04-20T10:56:04.864274+0000","last_clean":"2024-04-20T10:56:04.864274+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T10:56:04.864274+0000","last_undegraded":"2024-04-20T10:56:04.864274+0000","last_fullsized":"2024-04-20T10:56:04.864274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286721+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T10:55:44.286721+0000","last_peered":"2024-04-20T10:55:44.286721+0000","last_clean":"2024-04-20T10:55:44.286721+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T10:55:44.286721+0000","last_undegraded":"2024-04-20T10:55:44.286721+0000","last_fullsized":"2024-04-20T10:55:44.286721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864812+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T10:56:04.864812+0000","last_peered":"2024-04-20T10:56:04.864812+0000","last_clean":"2024-04-20T10:56:04.864812+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T10:56:04.864812+0000","last_undegraded":"2024-04-20T10:56:04.864812+0000","last_fullsized":"2024-04-20T10:56:04.864812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286934+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T10:55:44.286934+0000","last_peered":"2024-04-20T10:55:44.286934+0000","last_clean":"2024-04-20T10:55:44.286934+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T10:55:44.286934+0000","last_undegraded":"2024-04-20T10:55:44.286934+0000","last_fullsized":"2024-04-20T10:55:44.286934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864501+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T10:56:04.864501+0000","last_peered":"2024-04-20T10:56:04.864501+0000","last_clean":"2024-04-20T10:56:04.864501+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T10:56:04.864501+0000","last_undegraded":"2024-04-20T10:56:04.864501+0000","last_fullsized":"2024-04-20T10:56:04.864501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793252+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T10:55:43.793252+0000","last_peered":"2024-04-20T10:55:43.793252+0000","last_clean":"2024-04-20T10:55:43.793252+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T10:55:43.793252+0000","last_undegraded":"2024-04-20T10:55:43.793252+0000","last_fullsized":"2024-04-20T10:55:43.793252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:58.848667+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T10:55:58.848667+0000","last_peered":"2024-04-20T10:55:58.848667+0000","last_clean":"2024-04-20T10:55:58.848667+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T10:55:58.848667+0000","last_undegraded":"2024-04-20T10:55:58.848667+0000","last_fullsized":"2024-04-20T10:55:58.848667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285651+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T10:55:44.285651+0000","last_peered":"2024-04-20T10:55:44.285651+0000","last_clean":"2024-04-20T10:55:44.285651+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T10:55:44.285651+0000","last_undegraded":"2024-04-20T10:55:44.285651+0000","last_fullsized":"2024-04-20T10:55:44.285651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:58.848610+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T10:55:58.848610+0000","last_peered":"2024-04-20T10:55:58.848610+0000","last_clean":"2024-04-20T10:55:58.848610+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T10:55:58.848610+0000","last_undegraded":"2024-04-20T10:55:58.848610+0000","last_fullsized":"2024-04-20T10:55:58.848610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797319+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T10:55:43.797319+0000","last_peered":"2024-04-20T10:55:43.797319+0000","last_clean":"2024-04-20T10:55:43.797319+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T10:55:43.797319+0000","last_undegraded":"2024-04-20T10:55:43.797319+0000","last_fullsized":"2024-04-20T10:55:43.797319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876180+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T10:56:10.876180+0000","last_peered":"2024-04-20T10:56:10.876180+0000","last_clean":"2024-04-20T10:56:10.876180+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T10:56:10.876180+0000","last_undegraded":"2024-04-20T10:56:10.876180+0000","last_fullsized":"2024-04-20T10:56:10.876180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800914+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T10:55:43.800914+0000","last_peered":"2024-04-20T10:55:43.800914+0000","last_clean":"2024-04-20T10:55:43.800914+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T10:55:43.800914+0000","last_undegraded":"2024-04-20T10:55:43.800914+0000","last_fullsized":"2024-04-20T10:55:43.800914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.731030+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T10:55:44.731030+0000","last_peered":"2024-04-20T10:55:44.731030+0000","last_clean":"2024-04-20T10:55:44.731030+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T10:55:44.731030+0000","last_undegraded":"2024-04-20T10:55:44.731030+0000","last_fullsized":"2024-04-20T10:55:44.731030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287087+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T10:55:44.287087+0000","last_peered":"2024-04-20T10:55:44.287087+0000","last_clean":"2024-04-20T10:55:44.287087+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T10:55:44.287087+0000","last_undegraded":"2024-04-20T10:55:44.287087+0000","last_fullsized":"2024-04-20T10:55:44.287087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864814+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T10:56:04.864814+0000","last_peered":"2024-04-20T10:56:04.864814+0000","last_clean":"2024-04-20T10:56:04.864814+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T10:56:04.864814+0000","last_undegraded":"2024-04-20T10:56:04.864814+0000","last_fullsized":"2024-04-20T10:56:04.864814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797411+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T10:55:43.797411+0000","last_peered":"2024-04-20T10:55:43.797411+0000","last_clean":"2024-04-20T10:55:43.797411+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T10:55:43.797411+0000","last_undegraded":"2024-04-20T10:55:43.797411+0000","last_fullsized":"2024-04-20T10:55:43.797411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.867493+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T10:56:04.867493+0000","last_peered":"2024-04-20T10:56:04.867493+0000","last_clean":"2024-04-20T10:56:04.867493+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T10:56:04.867493+0000","last_undegraded":"2024-04-20T10:56:04.867493+0000","last_fullsized":"2024-04-20T10:56:04.867493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793562+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T10:55:43.793562+0000","last_peered":"2024-04-20T10:55:43.793562+0000","last_clean":"2024-04-20T10:55:43.793562+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T10:55:43.793562+0000","last_undegraded":"2024-04-20T10:55:43.793562+0000","last_fullsized":"2024-04-20T10:55:43.793562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.873382+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T10:56:10.873382+0000","last_peered":"2024-04-20T10:56:10.873382+0000","last_clean":"2024-04-20T10:56:10.873382+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T10:56:10.873382+0000","last_undegraded":"2024-04-20T10:56:10.873382+0000","last_fullsized":"2024-04-20T10:56:10.873382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285397+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T10:55:44.285397+0000","last_peered":"2024-04-20T10:55:44.285397+0000","last_clean":"2024-04-20T10:55:44.285397+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T10:55:44.285397+0000","last_undegraded":"2024-04-20T10:55:44.285397+0000","last_fullsized":"2024-04-20T10:55:44.285397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.825832+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T10:55:52.825832+0000","last_peered":"2024-04-20T10:55:52.825832+0000","last_clean":"2024-04-20T10:55:52.825832+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T10:55:52.825832+0000","last_undegraded":"2024-04-20T10:55:52.825832+0000","last_fullsized":"2024-04-20T10:55:52.825832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800994+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T10:55:43.800994+0000","last_peered":"2024-04-20T10:55:43.800994+0000","last_clean":"2024-04-20T10:55:43.800994+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T10:55:43.800994+0000","last_undegraded":"2024-04-20T10:55:43.800994+0000","last_fullsized":"2024-04-20T10:55:43.800994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876458+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T10:56:10.876458+0000","last_peered":"2024-04-20T10:56:10.876458+0000","last_clean":"2024-04-20T10:56:10.876458+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T10:56:10.876458+0000","last_undegraded":"2024-04-20T10:56:10.876458+0000","last_fullsized":"2024-04-20T10:56:10.876458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793024+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T10:55:43.793024+0000","last_peered":"2024-04-20T10:55:43.793024+0000","last_clean":"2024-04-20T10:55:43.793024+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T10:55:43.793024+0000","last_undegraded":"2024-04-20T10:55:43.793024+0000","last_fullsized":"2024-04-20T10:55:43.793024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:05.384095+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T10:56:05.384095+0000","last_peered":"2024-04-20T10:56:05.384095+0000","last_clean":"2024-04-20T10:56:05.384095+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T10:56:05.384095+0000","last_undegraded":"2024-04-20T10:56:05.384095+0000","last_fullsized":"2024-04-20T10:56:05.384095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793105+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T10:55:43.793105+0000","last_peered":"2024-04-20T10:55:43.793105+0000","last_clean":"2024-04-20T10:55:43.793105+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T10:55:43.793105+0000","last_undegraded":"2024-04-20T10:55:43.793105+0000","last_fullsized":"2024-04-20T10:55:43.793105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.825429+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T10:55:52.825429+0000","last_peered":"2024-04-20T10:55:52.825429+0000","last_clean":"2024-04-20T10:55:52.825429+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T10:55:52.825429+0000","last_undegraded":"2024-04-20T10:55:52.825429+0000","last_fullsized":"2024-04-20T10:55:52.825429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286369+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T10:55:44.286369+0000","last_peered":"2024-04-20T10:55:44.286369+0000","last_clean":"2024-04-20T10:55:44.286369+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T10:55:44.286369+0000","last_undegraded":"2024-04-20T10:55:44.286369+0000","last_fullsized":"2024-04-20T10:55:44.286369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286936+0000","last_change":"2024-04-20T10:55:42.792905+0000","last_active":"2024-04-20T10:55:44.286936+0000","last_peered":"2024-04-20T10:55:44.286936+0000","last_clean":"2024-04-20T10:55:44.286936+0000","last_became_active":"2024-04-20T10:55:42.792587+0000","last_became_peered":"2024-04-20T10:55:42.792587+0000","last_unstale":"2024-04-20T10:55:44.286936+0000","last_undegraded":"2024-04-20T10:55:44.286936+0000","last_fullsized":"2024-04-20T10:55:44.286936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:02.987448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876702+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T10:56:10.876702+0000","last_peered":"2024-04-20T10:56:10.876702+0000","last_clean":"2024-04-20T10:56:10.876702+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T10:56:10.876702+0000","last_undegraded":"2024-04-20T10:56:10.876702+0000","last_fullsized":"2024-04-20T10:56:10.876702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286560+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T10:55:44.286560+0000","last_peered":"2024-04-20T10:55:44.286560+0000","last_clean":"2024-04-20T10:55:44.286560+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T10:55:44.286560+0000","last_undegraded":"2024-04-20T10:55:44.286560+0000","last_fullsized":"2024-04-20T10:55:44.286560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.877051+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T10:56:10.877051+0000","last_peered":"2024-04-20T10:56:10.877051+0000","last_clean":"2024-04-20T10:56:10.877051+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T10:56:10.877051+0000","last_undegraded":"2024-04-20T10:56:10.877051+0000","last_fullsized":"2024-04-20T10:56:10.877051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.788752+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T10:55:47.788752+0000","last_peered":"2024-04-20T10:55:47.788752+0000","last_clean":"2024-04-20T10:55:47.788752+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T10:55:47.788752+0000","last_undegraded":"2024-04-20T10:55:47.788752+0000","last_fullsized":"2024-04-20T10:55:47.788752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793370+0000","last_change":"2024-04-20T10:55:42.802227+0000","last_active":"2024-04-20T10:55:43.793370+0000","last_peered":"2024-04-20T10:55:43.793370+0000","last_clean":"2024-04-20T10:55:43.793370+0000","last_became_active":"2024-04-20T10:55:42.801944+0000","last_became_peered":"2024-04-20T10:55:42.801944+0000","last_unstale":"2024-04-20T10:55:43.793370+0000","last_undegraded":"2024-04-20T10:55:43.793370+0000","last_fullsized":"2024-04-20T10:55:43.793370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:29.862643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876246+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T10:56:10.876246+0000","last_peered":"2024-04-20T10:56:10.876246+0000","last_clean":"2024-04-20T10:56:10.876246+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T10:56:10.876246+0000","last_undegraded":"2024-04-20T10:56:10.876246+0000","last_fullsized":"2024-04-20T10:56:10.876246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8546,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":209,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":17092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:56:23.250 INFO:tasks.mds_thrash.ceph_manager:clean! 2024-04-20T10:56:23.250 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:56:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.386+0000 7f5d1d69a640 1 Processor -- start 2024-04-20T10:56:23.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.386+0000 7f5d1d69a640 1 -- start start 2024-04-20T10:56:23.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d1d69a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5d18151f40 0x7f5d1806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d1d69a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18152320 0x7f5d1806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d1d69a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d180c9970 con 0x7f5d1815e990 2024-04-20T10:56:23.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d1d69a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5d18070210 con 0x7f5d18151f40 2024-04-20T10:56:23.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d1d69a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5d1815b750 con 0x7f5d18152320 2024-04-20T10:56:23.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d16ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5d18151f40 0x7f5d1806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d16ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5d18151f40 0x7f5d1806f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35596/0 (socket says 172.21.15.27:35596) 2024-04-20T10:56:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d16ffd640 1 -- 172.21.15.27:0/1637260024 learned_addr learned my addr 172.21.15.27:0/1637260024 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d167fc640 1 --2- 172.21.15.27:0/1637260024 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18152320 0x7f5d1806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d16ffd640 1 -- 172.21.15.27:0/1637260024 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18152320 msgr2=0x7f5d1806fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d16ffd640 1 --2- 172.21.15.27:0/1637260024 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18152320 0x7f5d1806fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d16ffd640 1 -- 172.21.15.27:0/1637260024 >> v1:172.21.15.137:6789/0 conn(0x7f5d1815e990 legacy=0x7f5d1806ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:23.389 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.387+0000 7f5d16ffd640 1 -- 172.21.15.27:0/1637260024 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1815ba30 con 0x7f5d18151f40 2024-04-20T10:56:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.388+0000 7f5d16ffd640 1 --2- 172.21.15.27:0/1637260024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18151f40 0x7f5d1806f5c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009870 tx=0x7f5d1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fa484f6f5ba5ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.388+0000 7f5d15ffb640 1 -- 172.21.15.27:0/1637260024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10034070 con 0x7f5d18151f40 2024-04-20T10:56:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.388+0000 7f5d15ffb640 1 -- 172.21.15.27:0/1637260024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d10030840 con 0x7f5d18151f40 2024-04-20T10:56:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.388+0000 7f5d15ffb640 1 -- 172.21.15.27:0/1637260024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10030c10 con 0x7f5d18151f40 2024-04-20T10:56:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.389+0000 7f5d1d69a640 1 -- 172.21.15.27:0/1637260024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18151f40 msgr2=0x7f5d1806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.389+0000 7f5d1d69a640 1 --2- 172.21.15.27:0/1637260024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18151f40 0x7f5d1806f5c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009870 tx=0x7f5d1002b980 comp rx=0 tx=0).stop 2024-04-20T10:56:23.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.389+0000 7f5d1d69a640 1 -- 172.21.15.27:0/1637260024 shutdown_connections 2024-04-20T10:56:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.389+0000 7f5d1d69a640 1 --2- 172.21.15.27:0/1637260024 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18152320 0x7f5d1806fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.389+0000 7f5d1d69a640 1 --2- 172.21.15.27:0/1637260024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18151f40 0x7f5d1806f5c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.389+0000 7f5d1d69a640 1 -- 172.21.15.27:0/1637260024 >> 172.21.15.27:0/1637260024 conn(0x7f5d180cee90 msgr2=0x7f5d180cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.389+0000 7f5d1d69a640 1 -- 172.21.15.27:0/1637260024 shutdown_connections 2024-04-20T10:56:23.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.389+0000 7f5d1d69a640 1 -- 172.21.15.27:0/1637260024 wait complete. 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.390+0000 7f5d1d69a640 1 Processor -- start 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.390+0000 7f5d1d69a640 1 -- start start 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d1d69a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18151f40 0x7f5d181f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d1d69a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18152320 0x7f5d180c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d1d69a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d1815e990 0x7f5d181e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d1d69a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1815c2d0 con 0x7f5d18152320 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d1d69a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1815bff0 con 0x7f5d1815e990 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d1d69a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5d1806f0a0 con 0x7f5d18151f40 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d167fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18152320 0x7f5d180c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d167fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18152320 0x7f5d180c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35602/0 (socket says 172.21.15.27:35602) 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d167fc640 1 -- 172.21.15.27:0/2948695469 learned_addr learned my addr 172.21.15.27:0/2948695469 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d16ffd640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18151f40 0x7f5d181f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d177fe640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d1815e990 0x7f5d181e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d167fc640 1 -- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18151f40 msgr2=0x7f5d181f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d167fc640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18151f40 0x7f5d181f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d167fc640 1 -- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d1815e990 msgr2=0x7f5d181e4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d167fc640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d1815e990 0x7f5d181e4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d167fc640 1 -- 172.21.15.27:0/2948695469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d181f1bb0 con 0x7f5d18152320 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d167fc640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18152320 0x7f5d180c5450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000d7d0 tx=0x7f5d0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d0f7fe640 1 -- 172.21.15.27:0/2948695469 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000b6c0 con 0x7f5d18152320 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d0f7fe640 1 -- 172.21.15.27:0/2948695469 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d00019070 con 0x7f5d18152320 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.391+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d18150eb0 con 0x7f5d18152320 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.392+0000 7f5d0f7fe640 1 -- 172.21.15.27:0/2948695469 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000ba80 con 0x7f5d18152320 2024-04-20T10:56:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.392+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d1814f860 con 0x7f5d18152320 2024-04-20T10:56:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.393+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d1804fcf0 con 0x7f5d18152320 2024-04-20T10:56:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.393+0000 7f5d0f7fe640 1 -- 172.21.15.27:0/2948695469 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d0000bc20 con 0x7f5d18152320 2024-04-20T10:56:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.394+0000 7f5d0f7fe640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5cf806c8f0 0x7f5cf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.394+0000 7f5d0f7fe640 1 -- 172.21.15.27:0/2948695469 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5d0008bff0 con 0x7f5d18152320 2024-04-20T10:56:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.394+0000 7f5d16ffd640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5cf806c8f0 0x7f5cf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.394+0000 7f5d16ffd640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5cf806c8f0 0x7f5cf806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5d10036000 tx=0x7f5d10008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:23.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.398+0000 7f5d0f7fe640 1 -- 172.21.15.27:0/2948695469 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5d181f1bb0 con 0x7f5d18152320 2024-04-20T10:56:23.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.579+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d180b4ec0 con 0x7f5cf806c8f0 2024-04-20T10:56:23.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.586+0000 7f5d0f7fe640 1 -- 172.21.15.27:0/2948695469 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f5d180b4ec0 con 0x7f5cf806c8f0 2024-04-20T10:56:23.589 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:23.590 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5cf806c8f0 msgr2=0x7f5cf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5cf806c8f0 0x7f5cf806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5d10036000 tx=0x7f5d10008030 comp rx=0 tx=0).stop 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18152320 msgr2=0x7f5d180c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18152320 0x7f5d180c5450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000d7d0 tx=0x7f5d0000dc90 comp rx=0 tx=0).stop 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 shutdown_connections 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5cf806c8f0 0x7f5cf806edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d1815e990 0x7f5d181e4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d18152320 0x7f5d180c5450 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 --2- 172.21.15.27:0/2948695469 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5d18151f40 0x7f5d181f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 >> 172.21.15.27:0/2948695469 conn(0x7f5d180cee90 msgr2=0x7f5d1806f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 shutdown_connections 2024-04-20T10:56:23.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.589+0000 7f5d1d69a640 1 -- 172.21.15.27:0/2948695469 wait complete. 2024-04-20T10:56:23.600 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-04-20T10:56:22.899439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598914,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":368,"num_write_kb":2482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219144,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755576,"statfs":{"total":773094113280,"available":772869709824,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1918,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.190082"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287393+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T10:55:44.287393+0000","last_peered":"2024-04-20T10:55:44.287393+0000","last_clean":"2024-04-20T10:55:44.287393+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T10:55:44.287393+0000","last_undegraded":"2024-04-20T10:55:44.287393+0000","last_fullsized":"2024-04-20T10:55:44.287393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797629+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T10:55:43.797629+0000","last_peered":"2024-04-20T10:55:43.797629+0000","last_clean":"2024-04-20T10:55:43.797629+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T10:55:43.797629+0000","last_undegraded":"2024-04-20T10:55:43.797629+0000","last_fullsized":"2024-04-20T10:55:43.797629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285486+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T10:55:44.285486+0000","last_peered":"2024-04-20T10:55:44.285486+0000","last_clean":"2024-04-20T10:55:44.285486+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T10:55:44.285486+0000","last_undegraded":"2024-04-20T10:55:44.285486+0000","last_fullsized":"2024-04-20T10:55:44.285486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286383+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T10:55:44.286383+0000","last_peered":"2024-04-20T10:55:44.286383+0000","last_clean":"2024-04-20T10:55:44.286383+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T10:55:44.286383+0000","last_undegraded":"2024-04-20T10:55:44.286383+0000","last_fullsized":"2024-04-20T10:55:44.286383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286289+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T10:55:44.286289+0000","last_peered":"2024-04-20T10:55:44.286289+0000","last_clean":"2024-04-20T10:55:44.286289+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T10:55:44.286289+0000","last_undegraded":"2024-04-20T10:55:44.286289+0000","last_fullsized":"2024-04-20T10:55:44.286289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287329+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T10:55:44.287329+0000","last_peered":"2024-04-20T10:55:44.287329+0000","last_clean":"2024-04-20T10:55:44.287329+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T10:55:44.287329+0000","last_undegraded":"2024-04-20T10:55:44.287329+0000","last_fullsized":"2024-04-20T10:55:44.287329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287439+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T10:55:44.287439+0000","last_peered":"2024-04-20T10:55:44.287439+0000","last_clean":"2024-04-20T10:55:44.287439+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T10:55:44.287439+0000","last_undegraded":"2024-04-20T10:55:44.287439+0000","last_fullsized":"2024-04-20T10:55:44.287439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792859+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T10:55:43.792859+0000","last_peered":"2024-04-20T10:55:43.792859+0000","last_clean":"2024-04-20T10:55:43.792859+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T10:55:43.792859+0000","last_undegraded":"2024-04-20T10:55:43.792859+0000","last_fullsized":"2024-04-20T10:55:43.792859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286907+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T10:55:44.286907+0000","last_peered":"2024-04-20T10:55:44.286907+0000","last_clean":"2024-04-20T10:55:44.286907+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T10:55:44.286907+0000","last_undegraded":"2024-04-20T10:55:44.286907+0000","last_fullsized":"2024-04-20T10:55:44.286907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286541+0000","last_change":"2024-04-20T10:55:42.802716+0000","last_active":"2024-04-20T10:55:44.286541+0000","last_peered":"2024-04-20T10:55:44.286541+0000","last_clean":"2024-04-20T10:55:44.286541+0000","last_became_active":"2024-04-20T10:55:42.802418+0000","last_became_peered":"2024-04-20T10:55:42.802418+0000","last_unstale":"2024-04-20T10:55:44.286541+0000","last_undegraded":"2024-04-20T10:55:44.286541+0000","last_fullsized":"2024-04-20T10:55:44.286541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:12:22.186243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286228+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T10:55:44.286228+0000","last_peered":"2024-04-20T10:55:44.286228+0000","last_clean":"2024-04-20T10:55:44.286228+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T10:55:44.286228+0000","last_undegraded":"2024-04-20T10:55:44.286228+0000","last_fullsized":"2024-04-20T10:55:44.286228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801002+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T10:55:43.801002+0000","last_peered":"2024-04-20T10:55:43.801002+0000","last_clean":"2024-04-20T10:55:43.801002+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T10:55:43.801002+0000","last_undegraded":"2024-04-20T10:55:43.801002+0000","last_fullsized":"2024-04-20T10:55:43.801002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793050+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T10:55:43.793050+0000","last_peered":"2024-04-20T10:55:43.793050+0000","last_clean":"2024-04-20T10:55:43.793050+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T10:55:43.793050+0000","last_undegraded":"2024-04-20T10:55:43.793050+0000","last_fullsized":"2024-04-20T10:55:43.793050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797332+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T10:55:43.797332+0000","last_peered":"2024-04-20T10:55:43.797332+0000","last_clean":"2024-04-20T10:55:43.797332+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T10:55:43.797332+0000","last_undegraded":"2024-04-20T10:55:43.797332+0000","last_fullsized":"2024-04-20T10:55:43.797332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285851+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T10:55:44.285851+0000","last_peered":"2024-04-20T10:55:44.285851+0000","last_clean":"2024-04-20T10:55:44.285851+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T10:55:44.285851+0000","last_undegraded":"2024-04-20T10:55:44.285851+0000","last_fullsized":"2024-04-20T10:55:44.285851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286491+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T10:55:44.286491+0000","last_peered":"2024-04-20T10:55:44.286491+0000","last_clean":"2024-04-20T10:55:44.286491+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T10:55:44.286491+0000","last_undegraded":"2024-04-20T10:55:44.286491+0000","last_fullsized":"2024-04-20T10:55:44.286491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285217+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T10:55:44.285217+0000","last_peered":"2024-04-20T10:55:44.285217+0000","last_clean":"2024-04-20T10:55:44.285217+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T10:55:44.285217+0000","last_undegraded":"2024-04-20T10:55:44.285217+0000","last_fullsized":"2024-04-20T10:55:44.285217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287175+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T10:55:44.287175+0000","last_peered":"2024-04-20T10:55:44.287175+0000","last_clean":"2024-04-20T10:55:44.287175+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T10:55:44.287175+0000","last_undegraded":"2024-04-20T10:55:44.287175+0000","last_fullsized":"2024-04-20T10:55:44.287175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286994+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T10:55:44.286994+0000","last_peered":"2024-04-20T10:55:44.286994+0000","last_clean":"2024-04-20T10:55:44.286994+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T10:55:44.286994+0000","last_undegraded":"2024-04-20T10:55:44.286994+0000","last_fullsized":"2024-04-20T10:55:44.286994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285751+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T10:55:44.285751+0000","last_peered":"2024-04-20T10:55:44.285751+0000","last_clean":"2024-04-20T10:55:44.285751+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T10:55:44.285751+0000","last_undegraded":"2024-04-20T10:55:44.285751+0000","last_fullsized":"2024-04-20T10:55:44.285751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287484+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T10:55:44.287484+0000","last_peered":"2024-04-20T10:55:44.287484+0000","last_clean":"2024-04-20T10:55:44.287484+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T10:55:44.287484+0000","last_undegraded":"2024-04-20T10:55:44.287484+0000","last_fullsized":"2024-04-20T10:55:44.287484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792921+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T10:55:43.792921+0000","last_peered":"2024-04-20T10:55:43.792921+0000","last_clean":"2024-04-20T10:55:43.792921+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T10:55:43.792921+0000","last_undegraded":"2024-04-20T10:55:43.792921+0000","last_fullsized":"2024-04-20T10:55:43.792921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793499+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T10:55:43.793499+0000","last_peered":"2024-04-20T10:55:43.793499+0000","last_clean":"2024-04-20T10:55:43.793499+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T10:55:43.793499+0000","last_undegraded":"2024-04-20T10:55:43.793499+0000","last_fullsized":"2024-04-20T10:55:43.793499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286683+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T10:55:44.286683+0000","last_peered":"2024-04-20T10:55:44.286683+0000","last_clean":"2024-04-20T10:55:44.286683+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T10:55:44.286683+0000","last_undegraded":"2024-04-20T10:55:44.286683+0000","last_fullsized":"2024-04-20T10:55:44.286683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797439+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T10:55:43.797439+0000","last_peered":"2024-04-20T10:55:43.797439+0000","last_clean":"2024-04-20T10:55:43.797439+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T10:55:43.797439+0000","last_undegraded":"2024-04-20T10:55:43.797439+0000","last_fullsized":"2024-04-20T10:55:43.797439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793232+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T10:55:43.793232+0000","last_peered":"2024-04-20T10:55:43.793232+0000","last_clean":"2024-04-20T10:55:43.793232+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T10:55:43.793232+0000","last_undegraded":"2024-04-20T10:55:43.793232+0000","last_fullsized":"2024-04-20T10:55:43.793232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285757+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T10:55:44.285757+0000","last_peered":"2024-04-20T10:55:44.285757+0000","last_clean":"2024-04-20T10:55:44.285757+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T10:55:44.285757+0000","last_undegraded":"2024-04-20T10:55:44.285757+0000","last_fullsized":"2024-04-20T10:55:44.285757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287493+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T10:55:44.287493+0000","last_peered":"2024-04-20T10:55:44.287493+0000","last_clean":"2024-04-20T10:55:44.287493+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T10:55:44.287493+0000","last_undegraded":"2024-04-20T10:55:44.287493+0000","last_fullsized":"2024-04-20T10:55:44.287493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285309+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T10:55:44.285309+0000","last_peered":"2024-04-20T10:55:44.285309+0000","last_clean":"2024-04-20T10:55:44.285309+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T10:55:44.285309+0000","last_undegraded":"2024-04-20T10:55:44.285309+0000","last_fullsized":"2024-04-20T10:55:44.285309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285746+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T10:55:44.285746+0000","last_peered":"2024-04-20T10:55:44.285746+0000","last_clean":"2024-04-20T10:55:44.285746+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T10:55:44.285746+0000","last_undegraded":"2024-04-20T10:55:44.285746+0000","last_fullsized":"2024-04-20T10:55:44.285746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800661+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T10:55:43.800661+0000","last_peered":"2024-04-20T10:55:43.800661+0000","last_clean":"2024-04-20T10:55:43.800661+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T10:55:43.800661+0000","last_undegraded":"2024-04-20T10:55:43.800661+0000","last_fullsized":"2024-04-20T10:55:43.800661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.831445+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T10:55:52.831445+0000","last_peered":"2024-04-20T10:55:52.831445+0000","last_clean":"2024-04-20T10:55:52.831445+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T10:55:52.831445+0000","last_undegraded":"2024-04-20T10:55:52.831445+0000","last_fullsized":"2024-04-20T10:55:52.831445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285840+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T10:55:44.285840+0000","last_peered":"2024-04-20T10:55:44.285840+0000","last_clean":"2024-04-20T10:55:44.285840+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T10:55:44.285840+0000","last_undegraded":"2024-04-20T10:55:44.285840+0000","last_fullsized":"2024-04-20T10:55:44.285840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864139+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T10:56:04.864139+0000","last_peered":"2024-04-20T10:56:04.864139+0000","last_clean":"2024-04-20T10:56:04.864139+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T10:56:04.864139+0000","last_undegraded":"2024-04-20T10:56:04.864139+0000","last_fullsized":"2024-04-20T10:56:04.864139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:58.849736+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T10:55:58.849736+0000","last_peered":"2024-04-20T10:55:58.849736+0000","last_clean":"2024-04-20T10:55:58.849736+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T10:55:58.849736+0000","last_undegraded":"2024-04-20T10:55:58.849736+0000","last_fullsized":"2024-04-20T10:55:58.849736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.792877+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T10:55:43.792877+0000","last_peered":"2024-04-20T10:55:43.792877+0000","last_clean":"2024-04-20T10:55:43.792877+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T10:55:43.792877+0000","last_undegraded":"2024-04-20T10:55:43.792877+0000","last_fullsized":"2024-04-20T10:55:43.792877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.867514+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T10:56:04.867514+0000","last_peered":"2024-04-20T10:56:04.867514+0000","last_clean":"2024-04-20T10:56:04.867514+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T10:56:04.867514+0000","last_undegraded":"2024-04-20T10:56:04.867514+0000","last_fullsized":"2024-04-20T10:56:04.867514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797544+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T10:55:43.797544+0000","last_peered":"2024-04-20T10:55:43.797544+0000","last_clean":"2024-04-20T10:55:43.797544+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T10:55:43.797544+0000","last_undegraded":"2024-04-20T10:55:43.797544+0000","last_fullsized":"2024-04-20T10:55:43.797544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864804+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T10:56:04.864804+0000","last_peered":"2024-04-20T10:56:04.864804+0000","last_clean":"2024-04-20T10:56:04.864804+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T10:56:04.864804+0000","last_undegraded":"2024-04-20T10:56:04.864804+0000","last_fullsized":"2024-04-20T10:56:04.864804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287062+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T10:55:44.287062+0000","last_peered":"2024-04-20T10:55:44.287062+0000","last_clean":"2024-04-20T10:55:44.287062+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T10:55:44.287062+0000","last_undegraded":"2024-04-20T10:55:44.287062+0000","last_fullsized":"2024-04-20T10:55:44.287062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.877295+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T10:56:10.877295+0000","last_peered":"2024-04-20T10:56:10.877295+0000","last_clean":"2024-04-20T10:56:10.877295+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T10:56:10.877295+0000","last_undegraded":"2024-04-20T10:56:10.877295+0000","last_fullsized":"2024-04-20T10:56:10.877295+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285669+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T10:55:44.285669+0000","last_peered":"2024-04-20T10:55:44.285669+0000","last_clean":"2024-04-20T10:55:44.285669+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T10:55:44.285669+0000","last_undegraded":"2024-04-20T10:55:44.285669+0000","last_fullsized":"2024-04-20T10:55:44.285669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.828790+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T10:55:52.828790+0000","last_peered":"2024-04-20T10:55:52.828790+0000","last_clean":"2024-04-20T10:55:52.828790+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T10:55:52.828790+0000","last_undegraded":"2024-04-20T10:55:52.828790+0000","last_fullsized":"2024-04-20T10:55:52.828790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.801052+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T10:55:43.801052+0000","last_peered":"2024-04-20T10:55:43.801052+0000","last_clean":"2024-04-20T10:55:43.801052+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T10:55:43.801052+0000","last_undegraded":"2024-04-20T10:55:43.801052+0000","last_fullsized":"2024-04-20T10:55:43.801052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:19.323845+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T10:56:19.323845+0000","last_peered":"2024-04-20T10:56:19.323845+0000","last_clean":"2024-04-20T10:56:19.323845+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T10:56:19.323845+0000","last_undegraded":"2024-04-20T10:56:19.323845+0000","last_fullsized":"2024-04-20T10:56:19.323845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287336+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T10:55:44.287336+0000","last_peered":"2024-04-20T10:55:44.287336+0000","last_clean":"2024-04-20T10:55:44.287336+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T10:55:44.287336+0000","last_undegraded":"2024-04-20T10:55:44.287336+0000","last_fullsized":"2024-04-20T10:55:44.287336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.873583+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T10:56:10.873583+0000","last_peered":"2024-04-20T10:56:10.873583+0000","last_clean":"2024-04-20T10:56:10.873583+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T10:56:10.873583+0000","last_undegraded":"2024-04-20T10:56:10.873583+0000","last_fullsized":"2024-04-20T10:56:10.873583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285622+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T10:55:44.285622+0000","last_peered":"2024-04-20T10:55:44.285622+0000","last_clean":"2024-04-20T10:55:44.285622+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T10:55:44.285622+0000","last_undegraded":"2024-04-20T10:55:44.285622+0000","last_fullsized":"2024-04-20T10:55:44.285622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:05.462951+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T10:56:05.462951+0000","last_peered":"2024-04-20T10:56:05.462951+0000","last_clean":"2024-04-20T10:56:05.462951+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T10:56:05.462951+0000","last_undegraded":"2024-04-20T10:56:05.462951+0000","last_fullsized":"2024-04-20T10:56:05.462951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286631+0000","last_change":"2024-04-20T10:55:42.805169+0000","last_active":"2024-04-20T10:55:44.286631+0000","last_peered":"2024-04-20T10:55:44.286631+0000","last_clean":"2024-04-20T10:55:44.286631+0000","last_became_active":"2024-04-20T10:55:42.804573+0000","last_became_peered":"2024-04-20T10:55:42.804573+0000","last_unstale":"2024-04-20T10:55:44.286631+0000","last_undegraded":"2024-04-20T10:55:44.286631+0000","last_fullsized":"2024-04-20T10:55:44.286631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:16.103447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864036+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T10:56:04.864036+0000","last_peered":"2024-04-20T10:56:04.864036+0000","last_clean":"2024-04-20T10:56:04.864036+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T10:56:04.864036+0000","last_undegraded":"2024-04-20T10:56:04.864036+0000","last_fullsized":"2024-04-20T10:56:04.864036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285762+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T10:55:44.285762+0000","last_peered":"2024-04-20T10:55:44.285762+0000","last_clean":"2024-04-20T10:55:44.285762+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T10:55:44.285762+0000","last_undegraded":"2024-04-20T10:55:44.285762+0000","last_fullsized":"2024-04-20T10:55:44.285762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864145+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T10:56:04.864145+0000","last_peered":"2024-04-20T10:56:04.864145+0000","last_clean":"2024-04-20T10:56:04.864145+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T10:56:04.864145+0000","last_undegraded":"2024-04-20T10:56:04.864145+0000","last_fullsized":"2024-04-20T10:56:04.864145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287518+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T10:55:44.287518+0000","last_peered":"2024-04-20T10:55:44.287518+0000","last_clean":"2024-04-20T10:55:44.287518+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T10:55:44.287518+0000","last_undegraded":"2024-04-20T10:55:44.287518+0000","last_fullsized":"2024-04-20T10:55:44.287518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.786618+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T10:55:47.786618+0000","last_peered":"2024-04-20T10:55:47.786618+0000","last_clean":"2024-04-20T10:55:47.786618+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T10:55:47.786618+0000","last_undegraded":"2024-04-20T10:55:47.786618+0000","last_fullsized":"2024-04-20T10:55:47.786618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286460+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T10:55:44.286460+0000","last_peered":"2024-04-20T10:55:44.286460+0000","last_clean":"2024-04-20T10:55:44.286460+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T10:55:44.286460+0000","last_undegraded":"2024-04-20T10:55:44.286460+0000","last_fullsized":"2024-04-20T10:55:44.286460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.873255+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T10:56:10.873255+0000","last_peered":"2024-04-20T10:56:10.873255+0000","last_clean":"2024-04-20T10:56:10.873255+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T10:56:10.873255+0000","last_undegraded":"2024-04-20T10:56:10.873255+0000","last_fullsized":"2024-04-20T10:56:10.873255+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287067+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T10:55:44.287067+0000","last_peered":"2024-04-20T10:55:44.287067+0000","last_clean":"2024-04-20T10:55:44.287067+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T10:55:44.287067+0000","last_undegraded":"2024-04-20T10:55:44.287067+0000","last_fullsized":"2024-04-20T10:55:44.287067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864274+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T10:56:04.864274+0000","last_peered":"2024-04-20T10:56:04.864274+0000","last_clean":"2024-04-20T10:56:04.864274+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T10:56:04.864274+0000","last_undegraded":"2024-04-20T10:56:04.864274+0000","last_fullsized":"2024-04-20T10:56:04.864274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286721+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T10:55:44.286721+0000","last_peered":"2024-04-20T10:55:44.286721+0000","last_clean":"2024-04-20T10:55:44.286721+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T10:55:44.286721+0000","last_undegraded":"2024-04-20T10:55:44.286721+0000","last_fullsized":"2024-04-20T10:55:44.286721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864812+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T10:56:04.864812+0000","last_peered":"2024-04-20T10:56:04.864812+0000","last_clean":"2024-04-20T10:56:04.864812+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T10:56:04.864812+0000","last_undegraded":"2024-04-20T10:56:04.864812+0000","last_fullsized":"2024-04-20T10:56:04.864812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286934+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T10:55:44.286934+0000","last_peered":"2024-04-20T10:55:44.286934+0000","last_clean":"2024-04-20T10:55:44.286934+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T10:55:44.286934+0000","last_undegraded":"2024-04-20T10:55:44.286934+0000","last_fullsized":"2024-04-20T10:55:44.286934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286005+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T10:55:44.286005+0000","last_peered":"2024-04-20T10:55:44.286005+0000","last_clean":"2024-04-20T10:55:44.286005+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T10:55:44.286005+0000","last_undegraded":"2024-04-20T10:55:44.286005+0000","last_fullsized":"2024-04-20T10:55:44.286005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864501+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T10:56:04.864501+0000","last_peered":"2024-04-20T10:56:04.864501+0000","last_clean":"2024-04-20T10:56:04.864501+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T10:56:04.864501+0000","last_undegraded":"2024-04-20T10:56:04.864501+0000","last_fullsized":"2024-04-20T10:56:04.864501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793252+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T10:55:43.793252+0000","last_peered":"2024-04-20T10:55:43.793252+0000","last_clean":"2024-04-20T10:55:43.793252+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T10:55:43.793252+0000","last_undegraded":"2024-04-20T10:55:43.793252+0000","last_fullsized":"2024-04-20T10:55:43.793252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:58.848667+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T10:55:58.848667+0000","last_peered":"2024-04-20T10:55:58.848667+0000","last_clean":"2024-04-20T10:55:58.848667+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T10:55:58.848667+0000","last_undegraded":"2024-04-20T10:55:58.848667+0000","last_fullsized":"2024-04-20T10:55:58.848667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285651+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T10:55:44.285651+0000","last_peered":"2024-04-20T10:55:44.285651+0000","last_clean":"2024-04-20T10:55:44.285651+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T10:55:44.285651+0000","last_undegraded":"2024-04-20T10:55:44.285651+0000","last_fullsized":"2024-04-20T10:55:44.285651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:58.848610+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T10:55:58.848610+0000","last_peered":"2024-04-20T10:55:58.848610+0000","last_clean":"2024-04-20T10:55:58.848610+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T10:55:58.848610+0000","last_undegraded":"2024-04-20T10:55:58.848610+0000","last_fullsized":"2024-04-20T10:55:58.848610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797319+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T10:55:43.797319+0000","last_peered":"2024-04-20T10:55:43.797319+0000","last_clean":"2024-04-20T10:55:43.797319+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T10:55:43.797319+0000","last_undegraded":"2024-04-20T10:55:43.797319+0000","last_fullsized":"2024-04-20T10:55:43.797319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876180+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T10:56:10.876180+0000","last_peered":"2024-04-20T10:56:10.876180+0000","last_clean":"2024-04-20T10:56:10.876180+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T10:56:10.876180+0000","last_undegraded":"2024-04-20T10:56:10.876180+0000","last_fullsized":"2024-04-20T10:56:10.876180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800914+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T10:55:43.800914+0000","last_peered":"2024-04-20T10:55:43.800914+0000","last_clean":"2024-04-20T10:55:43.800914+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T10:55:43.800914+0000","last_undegraded":"2024-04-20T10:55:43.800914+0000","last_fullsized":"2024-04-20T10:55:43.800914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.731030+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T10:55:44.731030+0000","last_peered":"2024-04-20T10:55:44.731030+0000","last_clean":"2024-04-20T10:55:44.731030+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T10:55:44.731030+0000","last_undegraded":"2024-04-20T10:55:44.731030+0000","last_fullsized":"2024-04-20T10:55:44.731030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.287087+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T10:55:44.287087+0000","last_peered":"2024-04-20T10:55:44.287087+0000","last_clean":"2024-04-20T10:55:44.287087+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T10:55:44.287087+0000","last_undegraded":"2024-04-20T10:55:44.287087+0000","last_fullsized":"2024-04-20T10:55:44.287087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.864814+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T10:56:04.864814+0000","last_peered":"2024-04-20T10:56:04.864814+0000","last_clean":"2024-04-20T10:56:04.864814+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T10:56:04.864814+0000","last_undegraded":"2024-04-20T10:56:04.864814+0000","last_fullsized":"2024-04-20T10:56:04.864814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.797411+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T10:55:43.797411+0000","last_peered":"2024-04-20T10:55:43.797411+0000","last_clean":"2024-04-20T10:55:43.797411+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T10:55:43.797411+0000","last_undegraded":"2024-04-20T10:55:43.797411+0000","last_fullsized":"2024-04-20T10:55:43.797411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:04.867493+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T10:56:04.867493+0000","last_peered":"2024-04-20T10:56:04.867493+0000","last_clean":"2024-04-20T10:56:04.867493+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T10:56:04.867493+0000","last_undegraded":"2024-04-20T10:56:04.867493+0000","last_fullsized":"2024-04-20T10:56:04.867493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793562+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T10:55:43.793562+0000","last_peered":"2024-04-20T10:55:43.793562+0000","last_clean":"2024-04-20T10:55:43.793562+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T10:55:43.793562+0000","last_undegraded":"2024-04-20T10:55:43.793562+0000","last_fullsized":"2024-04-20T10:55:43.793562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.873382+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T10:56:10.873382+0000","last_peered":"2024-04-20T10:56:10.873382+0000","last_clean":"2024-04-20T10:56:10.873382+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T10:56:10.873382+0000","last_undegraded":"2024-04-20T10:56:10.873382+0000","last_fullsized":"2024-04-20T10:56:10.873382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285397+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T10:55:44.285397+0000","last_peered":"2024-04-20T10:55:44.285397+0000","last_clean":"2024-04-20T10:55:44.285397+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T10:55:44.285397+0000","last_undegraded":"2024-04-20T10:55:44.285397+0000","last_fullsized":"2024-04-20T10:55:44.285397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.825832+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T10:55:52.825832+0000","last_peered":"2024-04-20T10:55:52.825832+0000","last_clean":"2024-04-20T10:55:52.825832+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T10:55:52.825832+0000","last_undegraded":"2024-04-20T10:55:52.825832+0000","last_fullsized":"2024-04-20T10:55:52.825832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.800994+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T10:55:43.800994+0000","last_peered":"2024-04-20T10:55:43.800994+0000","last_clean":"2024-04-20T10:55:43.800994+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T10:55:43.800994+0000","last_undegraded":"2024-04-20T10:55:43.800994+0000","last_fullsized":"2024-04-20T10:55:43.800994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876458+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T10:56:10.876458+0000","last_peered":"2024-04-20T10:56:10.876458+0000","last_clean":"2024-04-20T10:56:10.876458+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T10:56:10.876458+0000","last_undegraded":"2024-04-20T10:56:10.876458+0000","last_fullsized":"2024-04-20T10:56:10.876458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793024+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T10:55:43.793024+0000","last_peered":"2024-04-20T10:55:43.793024+0000","last_clean":"2024-04-20T10:55:43.793024+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T10:55:43.793024+0000","last_undegraded":"2024-04-20T10:55:43.793024+0000","last_fullsized":"2024-04-20T10:55:43.793024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:05.384095+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T10:56:05.384095+0000","last_peered":"2024-04-20T10:56:05.384095+0000","last_clean":"2024-04-20T10:56:05.384095+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T10:56:05.384095+0000","last_undegraded":"2024-04-20T10:56:05.384095+0000","last_fullsized":"2024-04-20T10:56:05.384095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793105+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T10:55:43.793105+0000","last_peered":"2024-04-20T10:55:43.793105+0000","last_clean":"2024-04-20T10:55:43.793105+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T10:55:43.793105+0000","last_undegraded":"2024-04-20T10:55:43.793105+0000","last_fullsized":"2024-04-20T10:55:43.793105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:52.825429+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T10:55:52.825429+0000","last_peered":"2024-04-20T10:55:52.825429+0000","last_clean":"2024-04-20T10:55:52.825429+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T10:55:52.825429+0000","last_undegraded":"2024-04-20T10:55:52.825429+0000","last_fullsized":"2024-04-20T10:55:52.825429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286369+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T10:55:44.286369+0000","last_peered":"2024-04-20T10:55:44.286369+0000","last_clean":"2024-04-20T10:55:44.286369+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T10:55:44.286369+0000","last_undegraded":"2024-04-20T10:55:44.286369+0000","last_fullsized":"2024-04-20T10:55:44.286369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286936+0000","last_change":"2024-04-20T10:55:42.792905+0000","last_active":"2024-04-20T10:55:44.286936+0000","last_peered":"2024-04-20T10:55:44.286936+0000","last_clean":"2024-04-20T10:55:44.286936+0000","last_became_active":"2024-04-20T10:55:42.792587+0000","last_became_peered":"2024-04-20T10:55:42.792587+0000","last_unstale":"2024-04-20T10:55:44.286936+0000","last_undegraded":"2024-04-20T10:55:44.286936+0000","last_fullsized":"2024-04-20T10:55:44.286936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:02.987448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876702+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T10:56:10.876702+0000","last_peered":"2024-04-20T10:56:10.876702+0000","last_clean":"2024-04-20T10:56:10.876702+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T10:56:10.876702+0000","last_undegraded":"2024-04-20T10:56:10.876702+0000","last_fullsized":"2024-04-20T10:56:10.876702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.286560+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T10:55:44.286560+0000","last_peered":"2024-04-20T10:55:44.286560+0000","last_clean":"2024-04-20T10:55:44.286560+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T10:55:44.286560+0000","last_undegraded":"2024-04-20T10:55:44.286560+0000","last_fullsized":"2024-04-20T10:55:44.286560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.877051+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T10:56:10.877051+0000","last_peered":"2024-04-20T10:56:10.877051+0000","last_clean":"2024-04-20T10:56:10.877051+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T10:56:10.877051+0000","last_undegraded":"2024-04-20T10:56:10.877051+0000","last_fullsized":"2024-04-20T10:56:10.877051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:44.285506+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T10:55:44.285506+0000","last_peered":"2024-04-20T10:55:44.285506+0000","last_clean":"2024-04-20T10:55:44.285506+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T10:55:44.285506+0000","last_undegraded":"2024-04-20T10:55:44.285506+0000","last_fullsized":"2024-04-20T10:55:44.285506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:47.788752+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T10:55:47.788752+0000","last_peered":"2024-04-20T10:55:47.788752+0000","last_clean":"2024-04-20T10:55:47.788752+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T10:55:47.788752+0000","last_undegraded":"2024-04-20T10:55:47.788752+0000","last_fullsized":"2024-04-20T10:55:47.788752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:55:43.793370+0000","last_change":"2024-04-20T10:55:42.802227+0000","last_active":"2024-04-20T10:55:43.793370+0000","last_peered":"2024-04-20T10:55:43.793370+0000","last_clean":"2024-04-20T10:55:43.793370+0000","last_became_active":"2024-04-20T10:55:42.801944+0000","last_became_peered":"2024-04-20T10:55:42.801944+0000","last_unstale":"2024-04-20T10:55:43.793370+0000","last_undegraded":"2024-04-20T10:55:43.793370+0000","last_fullsized":"2024-04-20T10:55:43.793370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:29:29.862643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T10:56:10.876246+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T10:56:10.876246+0000","last_peered":"2024-04-20T10:56:10.876246+0000","last_clean":"2024-04-20T10:56:10.876246+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T10:56:10.876246+0000","last_undegraded":"2024-04-20T10:56:10.876246+0000","last_fullsized":"2024-04-20T10:56:10.876246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8546,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":209,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":17092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:56:23.604 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:56:23.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.740+0000 7fb75b3bd640 1 Processor -- start 2024-04-20T10:56:23.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.740+0000 7fb75b3bd640 1 -- start start 2024-04-20T10:56:23.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.741+0000 7fb75b3bd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb754073970 0x7fb7541552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.741+0000 7fb75b3bd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb754073fc0 con 0x7fb75415ed10 2024-04-20T10:56:23.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.741+0000 7fb75b3bd640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb754073d50 con 0x7fb754072130 2024-04-20T10:56:23.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.741+0000 7fb75b3bd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb7540c3bc0 con 0x7fb754073970 2024-04-20T10:56:23.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.741+0000 7fb75995a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb754073970 0x7fb7541552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.741+0000 7fb759159640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb75415ed10 0x7fb7540c87f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37424/0 (socket says 172.21.15.27:37424) 2024-04-20T10:56:23.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.741+0000 7fb759159640 1 -- 172.21.15.27:0/3386243983 learned_addr learned my addr 172.21.15.27:0/3386243983 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:23.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.741+0000 7fb75995a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb754073970 0x7fb7541552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35618/0 (socket says 172.21.15.27:35618) 2024-04-20T10:56:23.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.742+0000 7fb75995a640 1 -- 172.21.15.27:0/3386243983 >> v1:172.21.15.137:6789/0 conn(0x7fb754072130 legacy=0x7fb75415f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:23.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.742+0000 7fb75995a640 1 -- 172.21.15.27:0/3386243983 >> v1:172.21.15.27:6789/0 conn(0x7fb75415ed10 legacy=0x7fb7540c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.742+0000 7fb75995a640 1 -- 172.21.15.27:0/3386243983 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7540c2fb0 con 0x7fb754073970 2024-04-20T10:56:23.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.742+0000 7fb75995a640 1 --2- 172.21.15.27:0/3386243983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754073970 0x7fb7541552f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb7440097a0 tx=0x7fb74402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f83b860c377f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:23.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb753fff640 1 -- 172.21.15.27:0/3386243983 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744035020 con 0x7fb754073970 2024-04-20T10:56:23.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb753fff640 1 -- 172.21.15.27:0/3386243983 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb744031830 con 0x7fb754073970 2024-04-20T10:56:23.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb753fff640 1 -- 172.21.15.27:0/3386243983 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744031c00 con 0x7fb754073970 2024-04-20T10:56:23.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb75b3bd640 1 -- 172.21.15.27:0/3386243983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754073970 msgr2=0x7fb7541552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb75b3bd640 1 --2- 172.21.15.27:0/3386243983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754073970 0x7fb7541552f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb7440097a0 tx=0x7fb74402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb75b3bd640 1 -- 172.21.15.27:0/3386243983 shutdown_connections 2024-04-20T10:56:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb75b3bd640 1 --2- 172.21.15.27:0/3386243983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754073970 0x7fb7541552f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb75b3bd640 1 -- 172.21.15.27:0/3386243983 >> 172.21.15.27:0/3386243983 conn(0x7fb7540ca8f0 msgr2=0x7fb7540cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb75b3bd640 1 -- 172.21.15.27:0/3386243983 shutdown_connections 2024-04-20T10:56:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.743+0000 7fb75b3bd640 1 -- 172.21.15.27:0/3386243983 wait complete. 2024-04-20T10:56:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.744+0000 7fb75b3bd640 1 Processor -- start 2024-04-20T10:56:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.744+0000 7fb75b3bd640 1 -- start start 2024-04-20T10:56:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb75b3bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754072130 0x7fb75414c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb75b3bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb754073970 0x7fb75406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb75b3bd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb75415ed10 0x7fb7541e5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb75b3bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7540c3570 con 0x7fb754072130 2024-04-20T10:56:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb75b3bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7540c4050 con 0x7fb754073970 2024-04-20T10:56:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb75b3bd640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fb7540c3850 con 0x7fb75415ed10 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb759159640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754072130 0x7fb75414c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb759159640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754072130 0x7fb75414c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35630/0 (socket says 172.21.15.27:35630) 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb759159640 1 -- 172.21.15.27:0/1012812959 learned_addr learned my addr 172.21.15.27:0/1012812959 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb759159640 1 -- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb75415ed10 msgr2=0x7fb7541e5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb759159640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb75415ed10 0x7fb7541e5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb759159640 1 -- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb754073970 msgr2=0x7fb75406faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb758958640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb754073970 0x7fb75406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb759159640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb754073970 0x7fb75406faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb759159640 1 -- 172.21.15.27:0/1012812959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75414c9a0 con 0x7fb754072130 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.745+0000 7fb758958640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb754073970 0x7fb75406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.746+0000 7fb759159640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754072130 0x7fb75414c460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00ca70 tx=0x7fb74c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.746+0000 7fb751ffb640 1 -- 172.21.15.27:0/1012812959 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c017070 con 0x7fb754072130 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.746+0000 7fb751ffb640 1 -- 172.21.15.27:0/1012812959 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c00abd0 con 0x7fb754072130 2024-04-20T10:56:23.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.746+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75414fbd0 con 0x7fb754072130 2024-04-20T10:56:23.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.746+0000 7fb751ffb640 1 -- 172.21.15.27:0/1012812959 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c012400 con 0x7fb754072130 2024-04-20T10:56:23.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.746+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7541f1750 con 0x7fb754072130 2024-04-20T10:56:23.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.747+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75404fcf0 con 0x7fb754072130 2024-04-20T10:56:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.747+0000 7fb751ffb640 1 -- 172.21.15.27:0/1012812959 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb74c01a070 con 0x7fb754072130 2024-04-20T10:56:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.748+0000 7fb751ffb640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb74006c8f0 0x7fb74006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.748+0000 7fb751ffb640 1 -- 172.21.15.27:0/1012812959 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb74c08b9a0 con 0x7fb754072130 2024-04-20T10:56:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.748+0000 7fb758958640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb74006c8f0 0x7fb74006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.748+0000 7fb758958640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb74006c8f0 0x7fb74006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb7540c84c0 tx=0x7fb748009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.752+0000 7fb751ffb640 1 -- 172.21.15.27:0/1012812959 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb74c059cc0 con 0x7fb754072130 2024-04-20T10:56:23.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.939+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb754158e30 con 0x7fb754072130 2024-04-20T10:56:23.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.940+0000 7fb751ffb640 1 -- 172.21.15.27:0/1012812959 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16819 (secure 0 0 0) 0x7fb74c0595c0 con 0x7fb754072130 2024-04-20T10:56:23.941 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:23.941 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:56:22.896869+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:56:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb74006c8f0 msgr2=0x7fb74006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb74006c8f0 0x7fb74006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb7540c84c0 tx=0x7fb748009040 comp rx=0 tx=0).stop 2024-04-20T10:56:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754072130 msgr2=0x7fb75414c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754072130 0x7fb75414c460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00ca70 tx=0x7fb74c00cf30 comp rx=0 tx=0).stop 2024-04-20T10:56:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 shutdown_connections 2024-04-20T10:56:23.943 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb74006c8f0 0x7fb74006edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb75415ed10 0x7fb7541e5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb754073970 0x7fb75406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 --2- 172.21.15.27:0/1012812959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb754072130 0x7fb75414c460 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:23.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 >> 172.21.15.27:0/1012812959 conn(0x7fb7540ca8f0 msgr2=0x7fb75414b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:23.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 shutdown_connections 2024-04-20T10:56:23.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:23.942+0000 7fb75b3bd640 1 -- 172.21.15.27:0/1012812959 wait complete. 2024-04-20T10:56:23.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:55:29.717620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:55:39.832461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:55:41.103975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:56:23.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:56:24.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.057+0000 7fe9beb4b640 1 Processor -- start 2024-04-20T10:56:24.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.057+0000 7fe9beb4b640 1 -- start start 2024-04-20T10:56:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9beb4b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe9b80c8840 0x7fe9b806e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9beb4b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 0x7fe9b80c6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9beb4b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b80c90f0 con 0x7fe9b81536b0 2024-04-20T10:56:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9beb4b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b80c8490 con 0x7fe9b80c8840 2024-04-20T10:56:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9beb4b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b80c76d0 con 0x7fe9b806eb40 2024-04-20T10:56:24.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9b7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe9b80c8840 0x7fe9b806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9bc8e7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe9b81536b0 0x7fe9b8159e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37436/0 (socket says 172.21.15.27:37436) 2024-04-20T10:56:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9bc8e7640 1 -- 172.21.15.27:0/2520246641 learned_addr learned my addr 172.21.15.27:0/2520246641 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9bd0e8640 1 --2- 172.21.15.27:0/2520246641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 0x7fe9b80c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9b7fff640 1 -- 172.21.15.27:0/2520246641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 msgr2=0x7fe9b80c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9b7fff640 1 --2- 172.21.15.27:0/2520246641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 0x7fe9b80c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9b7fff640 1 -- 172.21.15.27:0/2520246641 >> v1:172.21.15.27:6789/0 conn(0x7fe9b81536b0 legacy=0x7fe9b8159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9b7fff640 1 -- 172.21.15.27:0/2520246641 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b80c79b0 con 0x7fe9b80c8840 2024-04-20T10:56:24.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.058+0000 7fe9b77fe640 1 -- 172.21.15.27:0/2520246641 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2221935633 0 0) 0x7fe9b80c90f0 con 0x7fe9b81536b0 2024-04-20T10:56:24.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.059+0000 7fe9b7fff640 1 --2- 172.21.15.27:0/2520246641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 0x7fe9b806e600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0097a0 tx=0x7fe9ac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3e36d7760ff4a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.059+0000 7fe9b77fe640 1 -- 172.21.15.27:0/2520246641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac034070 con 0x7fe9b80c8840 2024-04-20T10:56:24.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.059+0000 7fe9b77fe640 1 -- 172.21.15.27:0/2520246641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac030610 con 0x7fe9b80c8840 2024-04-20T10:56:24.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.059+0000 7fe9b77fe640 1 -- 172.21.15.27:0/2520246641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0309e0 con 0x7fe9b80c8840 2024-04-20T10:56:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.060+0000 7fe9beb4b640 1 -- 172.21.15.27:0/2520246641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 msgr2=0x7fe9b806e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.060+0000 7fe9beb4b640 1 --2- 172.21.15.27:0/2520246641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 0x7fe9b806e600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0097a0 tx=0x7fe9ac02b960 comp rx=0 tx=0).stop 2024-04-20T10:56:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.060+0000 7fe9beb4b640 1 -- 172.21.15.27:0/2520246641 shutdown_connections 2024-04-20T10:56:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.060+0000 7fe9beb4b640 1 --2- 172.21.15.27:0/2520246641 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 0x7fe9b80c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.060+0000 7fe9beb4b640 1 --2- 172.21.15.27:0/2520246641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 0x7fe9b806e600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.060+0000 7fe9beb4b640 1 -- 172.21.15.27:0/2520246641 >> 172.21.15.27:0/2520246641 conn(0x7fe9b80c9630 msgr2=0x7fe9b80c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.060+0000 7fe9beb4b640 1 -- 172.21.15.27:0/2520246641 shutdown_connections 2024-04-20T10:56:24.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.060+0000 7fe9beb4b640 1 -- 172.21.15.27:0/2520246641 wait complete. 2024-04-20T10:56:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.061+0000 7fe9beb4b640 1 Processor -- start 2024-04-20T10:56:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.061+0000 7fe9beb4b640 1 -- start start 2024-04-20T10:56:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9beb4b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 0x7fe9b8147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9beb4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 0x7fe9b814ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9beb4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9b81536b0 0x7fe9b80714e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9beb4b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b8153a90 con 0x7fe9b80c8840 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9beb4b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b80c7f70 con 0x7fe9b81536b0 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9beb4b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b80c8c90 con 0x7fe9b806eb40 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 0x7fe9b814ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 0x7fe9b814ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35652/0 (socket says 172.21.15.27:35652) 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9b7fff640 1 -- 172.21.15.27:0/3851508378 learned_addr learned my addr 172.21.15.27:0/3851508378 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9bd0e8640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9b81536b0 0x7fe9b80714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9b7fff640 1 -- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 msgr2=0x7fe9b8147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9bc8e7640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 0x7fe9b8147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9b7fff640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 0x7fe9b8147520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9b7fff640 1 -- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9b81536b0 msgr2=0x7fe9b80714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9b7fff640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9b81536b0 0x7fe9b80714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.062+0000 7fe9b7fff640 1 -- 172.21.15.27:0/3851508378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b81efd30 con 0x7fe9b80c8840 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.063+0000 7fe9b7fff640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 0x7fe9b814ba30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac000c00 tx=0x7fe9ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.063+0000 7fe9b57fa640 1 -- 172.21.15.27:0/3851508378 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac034040 con 0x7fe9b80c8840 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.063+0000 7fe9b57fa640 1 -- 172.21.15.27:0/3851508378 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac030410 con 0x7fe9b80c8840 2024-04-20T10:56:24.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.063+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b8147d60 con 0x7fe9b80c8840 2024-04-20T10:56:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.063+0000 7fe9b57fa640 1 -- 172.21.15.27:0/3851508378 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac004250 con 0x7fe9b80c8840 2024-04-20T10:56:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.063+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b81ef250 con 0x7fe9b80c8840 2024-04-20T10:56:24.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.064+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b804fcf0 con 0x7fe9b80c8840 2024-04-20T10:56:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.064+0000 7fe9b57fa640 1 -- 172.21.15.27:0/3851508378 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9ac04c060 con 0x7fe9b80c8840 2024-04-20T10:56:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.065+0000 7fe9b57fa640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe9a406c920 0x7fe9a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.065+0000 7fe9b57fa640 1 -- 172.21.15.27:0/3851508378 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe9ac0ad960 con 0x7fe9b80c8840 2024-04-20T10:56:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.065+0000 7fe9bc8e7640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe9a406c920 0x7fe9a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.065+0000 7fe9bc8e7640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe9a406c920 0x7fe9a406ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00012e0 tx=0x7fe9b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.069+0000 7fe9b57fa640 1 -- 172.21.15.27:0/3851508378 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe9ac07bda0 con 0x7fe9b80c8840 2024-04-20T10:56:24.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.251+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe9b80b2fa0 con 0x7fe9b80c8840 2024-04-20T10:56:24.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.252+0000 7fe9b57fa640 1 -- 172.21.15.27:0/3851508378 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fe9ac07b6a0 con 0x7fe9b80c8840 2024-04-20T10:56:24.253 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe9a406c920 msgr2=0x7fe9a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe9a406c920 0x7fe9a406ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00012e0 tx=0x7fe9b0009040 comp rx=0 tx=0).stop 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 msgr2=0x7fe9b814ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 0x7fe9b814ba30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac000c00 tx=0x7fe9ac0046c0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 shutdown_connections 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe9a406c920 0x7fe9a406ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9b81536b0 0x7fe9b80714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b80c8840 0x7fe9b814ba30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 --2- 172.21.15.27:0/3851508378 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe9b806eb40 0x7fe9b8147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 >> 172.21.15.27:0/3851508378 conn(0x7fe9b80c9630 msgr2=0x7fe9b80cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:24.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 shutdown_connections 2024-04-20T10:56:24.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.254+0000 7fe9beb4b640 1 -- 172.21.15.27:0/3851508378 wait complete. 2024-04-20T10:56:24.265 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:56:24.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.369+0000 7f9271806640 1 Processor -- start 2024-04-20T10:56:24.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.369+0000 7f9271806640 1 -- start start 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f9271806640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f926c155950 0x7f926c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f9271806640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f926c15b9c0 con 0x7f926c0b5c00 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f9271806640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f926c15bca0 con 0x7f926c0cc330 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f9271806640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f926c15bf80 con 0x7f926c155950 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f926affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f926c155950 0x7f926c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f926b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f926c0b5c00 0x7f926c0d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37438/0 (socket says 172.21.15.27:37438) 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f926b7fe640 1 -- 172.21.15.27:0/291155440 learned_addr learned my addr 172.21.15.27:0/291155440 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f926a7fc640 1 -- 172.21.15.27:0/291155440 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1070151463 0 0) 0x7f926c15b9c0 con 0x7f926c0b5c00 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f926a7fc640 1 -- 172.21.15.27:0/291155440 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9258003520 con 0x7f926c0b5c00 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f926affd640 1 -- 172.21.15.27:0/291155440 >> v1:172.21.15.137:6789/0 conn(0x7f926c0cc330 legacy=0x7f926c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f926affd640 1 -- 172.21.15.27:0/291155440 >> v1:172.21.15.27:6789/0 conn(0x7f926c0b5c00 legacy=0x7f926c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.370+0000 7f926affd640 1 -- 172.21.15.27:0/291155440 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c15c260 con 0x7f926c155950 2024-04-20T10:56:24.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.371+0000 7f926affd640 1 --2- 172.21.15.27:0/291155440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 0x7f926c0cbdf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f925c0097a0 tx=0x7f925c02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18720280683cd55f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.371+0000 7f926a7fc640 1 -- 172.21.15.27:0/291155440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c02f810 con 0x7f926c155950 2024-04-20T10:56:24.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.371+0000 7f926a7fc640 1 -- 172.21.15.27:0/291155440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f925c02f9b0 con 0x7f926c155950 2024-04-20T10:56:24.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.371+0000 7f926a7fc640 1 -- 172.21.15.27:0/291155440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c02fd80 con 0x7f926c155950 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.372+0000 7f9271806640 1 -- 172.21.15.27:0/291155440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 msgr2=0x7f926c0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.372+0000 7f9271806640 1 --2- 172.21.15.27:0/291155440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 0x7f926c0cbdf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f925c0097a0 tx=0x7f925c02b9f0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.372+0000 7f9271806640 1 -- 172.21.15.27:0/291155440 shutdown_connections 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.372+0000 7f9271806640 1 --2- 172.21.15.27:0/291155440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 0x7f926c0cbdf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.372+0000 7f9271806640 1 -- 172.21.15.27:0/291155440 >> 172.21.15.27:0/291155440 conn(0x7f926c0676c0 msgr2=0x7f926c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.372+0000 7f9271806640 1 -- 172.21.15.27:0/291155440 shutdown_connections 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.372+0000 7f9271806640 1 -- 172.21.15.27:0/291155440 wait complete. 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.373+0000 7f9271806640 1 Processor -- start 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.373+0000 7f9271806640 1 -- start start 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9271806640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f926c0b5c00 0x7f926c0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9271806640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f926c0cc330 0x7f926c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9271806640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 0x7f926c0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9271806640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f926c15fd10 con 0x7f926c155950 2024-04-20T10:56:24.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9271806640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f926c15fb90 con 0x7f926c0cc330 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9271806640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f926c155d30 con 0x7f926c0b5c00 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 0x7f926c0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 0x7f926c0ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35656/0 (socket says 172.21.15.27:35656) 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926b7fe640 1 -- 172.21.15.27:0/3644180824 learned_addr learned my addr 172.21.15.27:0/3644180824 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9263fff640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f926c0cc330 0x7f926c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926affd640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f926c0b5c00 0x7f926c0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926b7fe640 1 -- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f926c0b5c00 msgr2=0x7f926c0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926b7fe640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f926c0b5c00 0x7f926c0c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926b7fe640 1 -- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f926c0cc330 msgr2=0x7f926c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926b7fe640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f926c0cc330 0x7f926c0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926b7fe640 1 -- 172.21.15.27:0/3644180824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c1f4ea0 con 0x7f926c155950 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9263fff640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f926c0cc330 0x7f926c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926affd640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f926c0b5c00 0x7f926c0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:24.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f926b7fe640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 0x7f926c0ca640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f926400c9f0 tx=0x7f926400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9268ff9640 1 -- 172.21.15.27:0/3644180824 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264017070 con 0x7f926c155950 2024-04-20T10:56:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9268ff9640 1 -- 172.21.15.27:0/3644180824 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f926400ab70 con 0x7f926c155950 2024-04-20T10:56:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c1f5180 con 0x7f926c155950 2024-04-20T10:56:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9268ff9640 1 -- 172.21.15.27:0/3644180824 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264012400 con 0x7f926c155950 2024-04-20T10:56:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.374+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c1f5740 con 0x7f926c155950 2024-04-20T10:56:24.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.375+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f926c0c5e80 con 0x7f926c155950 2024-04-20T10:56:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.376+0000 7f9268ff9640 1 -- 172.21.15.27:0/3644180824 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f926401a070 con 0x7f926c155950 2024-04-20T10:56:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.376+0000 7f9268ff9640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f924c06c8f0 0x7f924c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.376+0000 7f9268ff9640 1 -- 172.21.15.27:0/3644180824 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f926408b820 con 0x7f926c155950 2024-04-20T10:56:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.377+0000 7f926affd640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f924c06c8f0 0x7f924c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.377+0000 7f926affd640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f924c06c8f0 0x7f924c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f925c036000 tx=0x7f925c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.381+0000 7f9268ff9640 1 -- 172.21.15.27:0/3644180824 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f926c0c5e80 con 0x7f926c155950 2024-04-20T10:56:24.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.565+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f926c04fcf0 con 0x7f926c155950 2024-04-20T10:56:24.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.565+0000 7f9268ff9640 1 -- 172.21.15.27:0/3644180824 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f92640594f0 con 0x7f926c155950 2024-04-20T10:56:24.566 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-20T10:56:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f924c06c8f0 msgr2=0x7f924c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f924c06c8f0 0x7f924c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f925c036000 tx=0x7f925c008030 comp rx=0 tx=0).stop 2024-04-20T10:56:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 msgr2=0x7f926c0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 0x7f926c0ca640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f926400c9f0 tx=0x7f926400ceb0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 shutdown_connections 2024-04-20T10:56:24.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f924c06c8f0 0x7f924c06edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f926c155950 0x7f926c0ca640 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f926c0cc330 0x7f926c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 --2- 172.21.15.27:0/3644180824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f926c0b5c00 0x7f926c0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 >> 172.21.15.27:0/3644180824 conn(0x7f926c0676c0 msgr2=0x7f926c0c60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:24.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 shutdown_connections 2024-04-20T10:56:24.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.567+0000 7f9271806640 1 -- 172.21.15.27:0/3644180824 wait complete. 2024-04-20T10:56:24.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:56:24.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.682+0000 7fe59d2d2640 1 Processor -- start 2024-04-20T10:56:24.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.682+0000 7fe59d2d2640 1 -- start start 2024-04-20T10:56:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe59d2d2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe598155950 0x7fe5980cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe59d2d2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5980cc330 0x7fe5980b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe59d2d2640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe5980b5a30 0x7fe5980d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe59d2d2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe5980b5ea0 con 0x7fe598155950 2024-04-20T10:56:24.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe59d2d2640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fe59815b9c0 con 0x7fe5980b5a30 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe59d2d2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe59815bca0 con 0x7fe5980cc330 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe596ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe598155950 0x7fe5980cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe596ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe598155950 0x7fe5980cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52706/0 (socket says 172.21.15.27:52706) 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe596ffd640 1 -- 172.21.15.27:0/3121844127 learned_addr learned my addr 172.21.15.27:0/3121844127 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe5967fc640 1 --2- 172.21.15.27:0/3121844127 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5980cc330 0x7fe5980b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe5977fe640 1 --2- 172.21.15.27:0/3121844127 >> v2:172.21.15.137:3300/0 conn(0x7fe5980b5a30 0x7fe5980d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe596ffd640 1 -- 172.21.15.27:0/3121844127 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5980cc330 msgr2=0x7fe5980b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe596ffd640 1 --2- 172.21.15.27:0/3121844127 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5980cc330 0x7fe5980b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe596ffd640 1 -- 172.21.15.27:0/3121844127 >> v2:172.21.15.137:3300/0 conn(0x7fe5980b5a30 msgr2=0x7fe5980d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe596ffd640 1 --2- 172.21.15.27:0/3121844127 >> v2:172.21.15.137:3300/0 conn(0x7fe5980b5a30 0x7fe5980d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe596ffd640 1 -- 172.21.15.27:0/3121844127 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59815bf80 con 0x7fe598155950 2024-04-20T10:56:24.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.683+0000 7fe5967fc640 1 --2- 172.21.15.27:0/3121844127 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5980cc330 0x7fe5980b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.684+0000 7fe596ffd640 1 --2- 172.21.15.27:0/3121844127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe598155950 0x7fe5980cbdf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe59808e4b0 tx=0x7fe59002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317c2d7b592c0e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.684+0000 7fe595ffb640 1 -- 172.21.15.27:0/3121844127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590034070 con 0x7fe598155950 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.684+0000 7fe595ffb640 1 -- 172.21.15.27:0/3121844127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe590030860 con 0x7fe598155950 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.684+0000 7fe595ffb640 1 -- 172.21.15.27:0/3121844127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590030c30 con 0x7fe598155950 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.685+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3121844127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe598155950 msgr2=0x7fe5980cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.685+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3121844127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe598155950 0x7fe5980cbdf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe59808e4b0 tx=0x7fe59002b9a0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.685+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3121844127 shutdown_connections 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.685+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3121844127 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe5980cc330 0x7fe5980b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.685+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3121844127 >> v2:172.21.15.137:3300/0 conn(0x7fe5980b5a30 0x7fe5980d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.685+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3121844127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe598155950 0x7fe5980cbdf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.685+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3121844127 >> 172.21.15.27:0/3121844127 conn(0x7fe5980676c0 msgr2=0x7fe598067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:24.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.685+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3121844127 shutdown_connections 2024-04-20T10:56:24.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.685+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3121844127 wait complete. 2024-04-20T10:56:24.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.686+0000 7fe59d2d2640 1 Processor -- start 2024-04-20T10:56:24.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.686+0000 7fe59d2d2640 1 -- start start 2024-04-20T10:56:24.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe59d2d2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5980b5a30 0x7fe5981e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe59d2d2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5980cc330 0x7fe5981f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe59d2d2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe598155950 0x7fe5981f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe59d2d2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe59815fb90 con 0x7fe5980cc330 2024-04-20T10:56:24.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe59d2d2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe59815c540 con 0x7fe5980b5a30 2024-04-20T10:56:24.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe59d2d2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe59815fd10 con 0x7fe598155950 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5967fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5980cc330 0x7fe5981f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5967fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5980cc330 0x7fe5981f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52722/0 (socket says 172.21.15.27:52722) 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5967fc640 1 -- 172.21.15.27:0/3010956634 learned_addr learned my addr 172.21.15.27:0/3010956634 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe596ffd640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5980b5a30 0x7fe5981e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5977fe640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe598155950 0x7fe5981f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5967fc640 1 -- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe598155950 msgr2=0x7fe5981f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5967fc640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe598155950 0x7fe5981f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5967fc640 1 -- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5980b5a30 msgr2=0x7fe5981e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5967fc640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5980b5a30 0x7fe5981e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5967fc640 1 -- 172.21.15.27:0/3010956634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59815e8b0 con 0x7fe5980cc330 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe5967fc640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5980cc330 0x7fe5981f4cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00d800 tx=0x7fe58c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.687+0000 7fe57f7fe640 1 -- 172.21.15.27:0/3010956634 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c004030 con 0x7fe5980cc330 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.688+0000 7fe57f7fe640 1 -- 172.21.15.27:0/3010956634 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58c0041d0 con 0x7fe5980cc330 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.688+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59815eb90 con 0x7fe5980cc330 2024-04-20T10:56:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.688+0000 7fe57f7fe640 1 -- 172.21.15.27:0/3010956634 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0096a0 con 0x7fe5980cc330 2024-04-20T10:56:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.688+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5980c7370 con 0x7fe5980cc330 2024-04-20T10:56:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.688+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59804fcf0 con 0x7fe5980cc330 2024-04-20T10:56:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.689+0000 7fe57f7fe640 1 -- 172.21.15.27:0/3010956634 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe58c009840 con 0x7fe5980cc330 2024-04-20T10:56:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.689+0000 7fe57f7fe640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe58406c8f0 0x7fe58406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.690+0000 7fe57f7fe640 1 -- 172.21.15.27:0/3010956634 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe58c08bde0 con 0x7fe5980cc330 2024-04-20T10:56:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.690+0000 7fe596ffd640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe58406c8f0 0x7fe58406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.690+0000 7fe596ffd640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe58406c8f0 0x7fe58406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe590036000 tx=0x7fe59003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.694+0000 7fe57f7fe640 1 -- 172.21.15.27:0/3010956634 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe58c059ab0 con 0x7fe5980cc330 2024-04-20T10:56:24.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.880+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe5980cd0a0 con 0x7fe5980cc330 2024-04-20T10:56:24.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.880+0000 7fe57f7fe640 1 -- 172.21.15.27:0/3010956634 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fe58c05ddb0 con 0x7fe5980cc330 2024-04-20T10:56:24.881 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-20T10:56:24.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe58406c8f0 msgr2=0x7fe58406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe58406c8f0 0x7fe58406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe590036000 tx=0x7fe59003b040 comp rx=0 tx=0).stop 2024-04-20T10:56:24.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5980cc330 msgr2=0x7fe5981f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5980cc330 0x7fe5981f4cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00d800 tx=0x7fe58c00dcc0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 shutdown_connections 2024-04-20T10:56:24.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe58406c8f0 0x7fe58406edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe598155950 0x7fe5981f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5980cc330 0x7fe5981f4cf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 --2- 172.21.15.27:0/3010956634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5980b5a30 0x7fe5981e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 >> 172.21.15.27:0/3010956634 conn(0x7fe5980676c0 msgr2=0x7fe5980cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:24.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 shutdown_connections 2024-04-20T10:56:24.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.882+0000 7fe59d2d2640 1 -- 172.21.15.27:0/3010956634 wait complete. 2024-04-20T10:56:24.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:56:24.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.994+0000 7fe6bf4d9640 1 Processor -- start 2024-04-20T10:56:24.995 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.994+0000 7fe6bf4d9640 1 -- start start 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.995+0000 7fe6bf4d9640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe6b8071f60 0x7fe6b80730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.995+0000 7fe6bf4d9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b815c180 con 0x7fe6b8073620 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.995+0000 7fe6bf4d9640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b806c920 con 0x7fe6b8155720 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.995+0000 7fe6bf4d9640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fe6b806b930 con 0x7fe6b8071f60 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.995+0000 7fe6bda76640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe6b8073620 0x7fe6b8165c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41086/0 (socket says 172.21.15.27:41086) 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.995+0000 7fe6bda76640 1 -- 172.21.15.27:0/112545027 learned_addr learned my addr 172.21.15.27:0/112545027 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.996+0000 7fe6bca74640 1 --2- 172.21.15.27:0/112545027 >> v2:172.21.15.137:3300/0 conn(0x7fe6b8071f60 0x7fe6b80730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.996+0000 7fe6affff640 1 -- 172.21.15.27:0/112545027 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3887412060 0 0) 0x7fe6b815c180 con 0x7fe6b8073620 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.996+0000 7fe6affff640 1 -- 172.21.15.27:0/112545027 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6a4003520 con 0x7fe6b8073620 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.996+0000 7fe6affff640 1 -- 172.21.15.27:0/112545027 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3189745909 0 0) 0x7fe6b806c920 con 0x7fe6b8155720 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.996+0000 7fe6affff640 1 -- 172.21.15.27:0/112545027 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6b815c180 con 0x7fe6b8155720 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.996+0000 7fe6affff640 1 -- 172.21.15.27:0/112545027 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1457124516 0 0) 0x7fe6a4003520 con 0x7fe6b8073620 2024-04-20T10:56:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.996+0000 7fe6bca74640 1 -- 172.21.15.27:0/112545027 >> v1:172.21.15.137:6789/0 conn(0x7fe6b8155720 legacy=0x7fe6b80c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.996+0000 7fe6bca74640 1 -- 172.21.15.27:0/112545027 >> v1:172.21.15.27:6789/0 conn(0x7fe6b8073620 legacy=0x7fe6b8165c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.996+0000 7fe6bca74640 1 -- 172.21.15.27:0/112545027 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b806bb60 con 0x7fe6b8071f60 2024-04-20T10:56:24.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.997+0000 7fe6bca74640 1 --2- 172.21.15.27:0/112545027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6b8071f60 0x7fe6b80730e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80097a0 tx=0x7fe6a802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8097c9bcd81e156f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:24.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.997+0000 7fe6affff640 1 -- 172.21.15.27:0/112545027 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4001670 con 0x7fe6b8071f60 2024-04-20T10:56:24.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.997+0000 7fe6affff640 1 -- 172.21.15.27:0/112545027 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a8037070 con 0x7fe6b8071f60 2024-04-20T10:56:24.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.997+0000 7fe6affff640 1 -- 172.21.15.27:0/112545027 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80317b0 con 0x7fe6b8071f60 2024-04-20T10:56:24.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.998+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/112545027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6b8071f60 msgr2=0x7fe6b80730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:24.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.998+0000 7fe6bf4d9640 1 --2- 172.21.15.27:0/112545027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6b8071f60 0x7fe6b80730e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80097a0 tx=0x7fe6a802b8b0 comp rx=0 tx=0).stop 2024-04-20T10:56:24.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.998+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/112545027 shutdown_connections 2024-04-20T10:56:25.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.998+0000 7fe6bf4d9640 1 --2- 172.21.15.27:0/112545027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6b8071f60 0x7fe6b80730e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.998+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/112545027 >> 172.21.15.27:0/112545027 conn(0x7fe6b806f950 msgr2=0x7fe6b806fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:25.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.998+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/112545027 shutdown_connections 2024-04-20T10:56:25.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.998+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/112545027 wait complete. 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.999+0000 7fe6bf4d9640 1 Processor -- start 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:24.999+0000 7fe6bf4d9640 1 -- start start 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bf4d9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6b8071f60 0x7fe6b8152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bf4d9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b8073620 0x7fe6b815d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bf4d9640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe6b8155720 0x7fe6b815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bf4d9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b806c120 con 0x7fe6b8073620 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bf4d9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b806c490 con 0x7fe6b8071f60 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bf4d9640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b8072370 con 0x7fe6b8155720 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bca74640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b8073620 0x7fe6b815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bca74640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b8073620 0x7fe6b815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52728/0 (socket says 172.21.15.27:52728) 2024-04-20T10:56:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bca74640 1 -- 172.21.15.27:0/3408777710 learned_addr learned my addr 172.21.15.27:0/3408777710 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bd275640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6b8071f60 0x7fe6b8152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bda76640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe6b8155720 0x7fe6b815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bca74640 1 -- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe6b8155720 msgr2=0x7fe6b815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bca74640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe6b8155720 0x7fe6b815d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bca74640 1 -- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6b8071f60 msgr2=0x7fe6b8152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bca74640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6b8071f60 0x7fe6b8152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bca74640 1 -- 172.21.15.27:0/3408777710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b815f250 con 0x7fe6b8073620 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bca74640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b8073620 0x7fe6b815d040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6a802bec0 tx=0x7fe6a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6adffb640 1 -- 172.21.15.27:0/3408777710 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a804a070 con 0x7fe6b8073620 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6adffb640 1 -- 172.21.15.27:0/3408777710 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a8031ec0 con 0x7fe6b8073620 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b815b870 con 0x7fe6b8073620 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.000+0000 7fe6adffb640 1 -- 172.21.15.27:0/3408777710 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8045640 con 0x7fe6b8073620 2024-04-20T10:56:25.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.001+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b815a620 con 0x7fe6b8073620 2024-04-20T10:56:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.002+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b80cd790 con 0x7fe6b8073620 2024-04-20T10:56:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.002+0000 7fe6adffb640 1 -- 172.21.15.27:0/3408777710 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6a804d070 con 0x7fe6b8073620 2024-04-20T10:56:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.003+0000 7fe6adffb640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe69c06c920 0x7fe69c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.003+0000 7fe6adffb640 1 -- 172.21.15.27:0/3408777710 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe6a80ad250 con 0x7fe6b8073620 2024-04-20T10:56:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.003+0000 7fe6bd275640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe69c06c920 0x7fe69c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.003+0000 7fe6bd275640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe69c06c920 0x7fe69c06ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0001560 tx=0x7fe6b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.007+0000 7fe6adffb640 1 -- 172.21.15.27:0/3408777710 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe6b80cd790 con 0x7fe6b8073620 2024-04-20T10:56:25.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.231+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6b804fcf0 con 0x7fe6b8073620 2024-04-20T10:56:25.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.232+0000 7fe6adffb640 1 -- 172.21.15.27:0/3408777710 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7fe6a8040070 con 0x7fe6b8073620 2024-04-20T10:56:25.233 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:25.233 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:56:25.233 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe69c06c920 msgr2=0x7fe69c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe69c06c920 0x7fe69c06ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0001560 tx=0x7fe6b0009040 comp rx=0 tx=0).stop 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b8073620 msgr2=0x7fe6b815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b8073620 0x7fe6b815d040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6a802bec0 tx=0x7fe6a80046c0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 shutdown_connections 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe69c06c920 0x7fe69c06ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe6b8155720 0x7fe6b815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b8073620 0x7fe6b815d040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 --2- 172.21.15.27:0/3408777710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6b8071f60 0x7fe6b8152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 >> 172.21.15.27:0/3408777710 conn(0x7fe6b806f950 msgr2=0x7fe6b80c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:25.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 shutdown_connections 2024-04-20T10:56:25.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.234+0000 7fe6bf4d9640 1 -- 172.21.15.27:0/3408777710 wait complete. 2024-04-20T10:56:25.245 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:56:25.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.345+0000 7f47f2f3d640 1 Processor -- start 2024-04-20T10:56:25.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.345+0000 7f47f2f3d640 1 -- start start 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.346+0000 7f47f2f3d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f47ec0d0c90 0x7f47ec0d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.346+0000 7f47f2f3d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47ec15b9c0 con 0x7f47ec0b54f0 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.346+0000 7f47f2f3d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47ec15bca0 con 0x7f47ec155950 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.346+0000 7f47f2f3d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f47ec15bf80 con 0x7f47ec0d0c90 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.346+0000 7f47ebfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f47ec0b54f0 0x7f47ec0b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41092/0 (socket says 172.21.15.27:41092) 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.346+0000 7f47ebfff640 1 -- 172.21.15.27:0/601308239 learned_addr learned my addr 172.21.15.27:0/601308239 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47f14da640 1 --2- 172.21.15.27:0/601308239 >> v2:172.21.15.137:3300/0 conn(0x7f47ec0d0c90 0x7f47ec0d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064770794 0 0) 0x7f47ec15b9c0 con 0x7f47ec0b54f0 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47d8003520 con 0x7f47ec0b54f0 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 179412322 0 0) 0x7f47d8003520 con 0x7f47ec0b54f0 2024-04-20T10:56:25.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 >> v2:172.21.15.137:3300/0 conn(0x7f47ec0d0c90 msgr2=0x7f47ec0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 --2- 172.21.15.27:0/601308239 >> v2:172.21.15.137:3300/0 conn(0x7f47ec0d0c90 0x7f47ec0d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 >> v1:172.21.15.137:6789/0 conn(0x7f47ec155950 legacy=0x7f47ec0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec15c260 con 0x7f47ec0b54f0 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f47dc0019f0 con 0x7f47ec0b54f0 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 >> v1:172.21.15.27:6789/0 conn(0x7f47ec0b54f0 legacy=0x7f47ec0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47d80039b0 0x7f47d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47d8004800 0x7f47d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80051c0 0x7f47d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec15bf80 con 0x7f47d80039b0 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47d800c070 con 0x7f47d80051c0 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f47d800c270 con 0x7f47d8004800 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47f0cd9640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47d80039b0 0x7f47d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47f14da640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80051c0 0x7f47d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47ebfff640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47d8004800 0x7f47d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47f0cd9640 1 -- 172.21.15.27:0/601308239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47d8004800 msgr2=0x7f47d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47f0cd9640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47d8004800 0x7f47d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47f0cd9640 1 -- 172.21.15.27:0/601308239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80051c0 msgr2=0x7f47d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47f0cd9640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80051c0 0x7f47d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47f0cd9640 1 -- 172.21.15.27:0/601308239 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d8003520 con 0x7f47d80039b0 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47f14da640 1 -- 172.21.15.27:0/601308239 reap_dead start 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.347+0000 7f47ebfff640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47d8004800 0x7f47d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.348+0000 7f47f0cd9640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47d80039b0 0x7f47d8003dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47e4001750 tx=0x7f47d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d8cd4b0fb96e15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.348+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4001370 con 0x7f47d80039b0 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.348+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e40052d0 con 0x7f47d80039b0 2024-04-20T10:56:25.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.348+0000 7f47eb7fe640 1 -- 172.21.15.27:0/601308239 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4003830 con 0x7f47d80039b0 2024-04-20T10:56:25.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.349+0000 7f47f2f3d640 1 -- 172.21.15.27:0/601308239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47d80039b0 msgr2=0x7f47d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.349+0000 7f47f2f3d640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47d80039b0 0x7f47d8003dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f47e4001750 tx=0x7f47d8005830 comp rx=0 tx=0).stop 2024-04-20T10:56:25.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.349+0000 7f47f2f3d640 1 -- 172.21.15.27:0/601308239 shutdown_connections 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.349+0000 7f47f2f3d640 1 --2- 172.21.15.27:0/601308239 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47d80039b0 0x7f47d8003dd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.349+0000 7f47f2f3d640 1 -- 172.21.15.27:0/601308239 >> 172.21.15.27:0/601308239 conn(0x7f47ec0676c0 msgr2=0x7f47ec067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.349+0000 7f47f2f3d640 1 -- 172.21.15.27:0/601308239 shutdown_connections 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.349+0000 7f47f2f3d640 1 -- 172.21.15.27:0/601308239 wait complete. 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.350+0000 7f47f2f3d640 1 Processor -- start 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.350+0000 7f47f2f3d640 1 -- start start 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47f2f3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80039b0 0x7f47ec1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47f2f3d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec1f4de0 0x7f47ec1f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47f2f3d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47ec1e8490 0x7f47ec15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47f2f3d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec15fd10 con 0x7f47ec1f4de0 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47f2f3d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec15fb90 con 0x7f47d80039b0 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47f2f3d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f47ec155d30 con 0x7f47ec1e8490 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47ebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec1f4de0 0x7f47ec1f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47ebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec1f4de0 0x7f47ec1f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52740/0 (socket says 172.21.15.27:52740) 2024-04-20T10:56:25.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47ebfff640 1 -- 172.21.15.27:0/4010801842 learned_addr learned my addr 172.21.15.27:0/4010801842 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47f0cd9640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80039b0 0x7f47ec1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47f14da640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47ec1e8490 0x7f47ec15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47ebfff640 1 -- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47ec1e8490 msgr2=0x7f47ec15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47ebfff640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47ec1e8490 0x7f47ec15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47ebfff640 1 -- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80039b0 msgr2=0x7f47ec1589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47ebfff640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80039b0 0x7f47ec1589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47ebfff640 1 -- 172.21.15.27:0/4010801842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec1f5390 con 0x7f47ec1f4de0 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.351+0000 7f47f0cd9640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80039b0 0x7f47ec1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.352+0000 7f47ebfff640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec1f4de0 0x7f47ec1f2180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f47dc007a10 tx=0x7f47dc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.352+0000 7f47e97fa640 1 -- 172.21.15.27:0/4010801842 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc00edf0 con 0x7f47ec1f4de0 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.352+0000 7f47e97fa640 1 -- 172.21.15.27:0/4010801842 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc018070 con 0x7f47ec1f4de0 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.352+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec1f5670 con 0x7f47ec1f4de0 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.352+0000 7f47e97fa640 1 -- 172.21.15.27:0/4010801842 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc013640 con 0x7f47ec1f4de0 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.352+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec1f5c30 con 0x7f47ec1f4de0 2024-04-20T10:56:25.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.353+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ec04fcf0 con 0x7f47ec1f4de0 2024-04-20T10:56:25.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.353+0000 7f47e97fa640 1 -- 172.21.15.27:0/4010801842 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47dc0137e0 con 0x7f47ec1f4de0 2024-04-20T10:56:25.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.354+0000 7f47e97fa640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47cc06c8f0 0x7f47cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:56:25.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.354+0000 7f47e97fa640 1 -- 172.21.15.27:0/4010801842 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f47dc08c0a0 con 0x7f47ec1f4de0 2024-04-20T10:56:25.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.354+0000 7f47f0cd9640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47cc06c8f0 0x7f47cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:56:25.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.354+0000 7f47f0cd9640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47cc06c8f0 0x7f47cc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f47e4033000 tx=0x7f47e4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:56:25.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.358+0000 7f47e97fa640 1 -- 172.21.15.27:0/4010801842 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f47dc05a450 con 0x7f47ec1f4de0 2024-04-20T10:56:25.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.546+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47ec0cfb40 con 0x7f47ec1f4de0 2024-04-20T10:56:25.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.547+0000 7f47e97fa640 1 -- 172.21.15.27:0/4010801842 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16819 (secure 0 0 0) 0x7f47dc059d50 con 0x7f47ec1f4de0 2024-04-20T10:56:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:56:25.548 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T10:56:22.896869+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:56:25.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47cc06c8f0 msgr2=0x7f47cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47cc06c8f0 0x7f47cc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f47e4033000 tx=0x7f47e4009030 comp rx=0 tx=0).stop 2024-04-20T10:56:25.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec1f4de0 msgr2=0x7f47ec1f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:56:25.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec1f4de0 0x7f47ec1f2180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f47dc007a10 tx=0x7f47dc00e530 comp rx=0 tx=0).stop 2024-04-20T10:56:25.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 shutdown_connections 2024-04-20T10:56:25.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f47cc06c8f0 0x7f47cc06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47ec1e8490 0x7f47ec15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47ec1f4de0 0x7f47ec1f2180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 --2- 172.21.15.27:0/4010801842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d80039b0 0x7f47ec1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:56:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 >> 172.21.15.27:0/4010801842 conn(0x7f47ec0676c0 msgr2=0x7f47ec151f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:56:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 shutdown_connections 2024-04-20T10:56:25.551 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:56:25.549+0000 7f47f2f3d640 1 -- 172.21.15.27:0/4010801842 wait complete. 2024-04-20T10:56:25.561 DEBUG:tasks.mds_thrash:Yielding 2024-04-20T10:56:25.561 INFO:teuthology.run_tasks:Running task workunit... 2024-04-20T10:56:25.573 INFO:tasks.workunit:Pulling workunits from ref a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:56:25.573 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-20T10:56:25.573 INFO:tasks.workunit:timeout=6h 2024-04-20T10:56:25.573 INFO:tasks.workunit:cleanup=True 2024-04-20T10:56:25.574 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-20T10:56:25.574 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2024-04-20T10:56:25.574 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 45 secs before thrashing 2024-04-20T10:56:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-20T10:56:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-20T10:56:25.589 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-20T10:56:25.590 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:56:25.590 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-20T10:56:25.590 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-20T10:56:25.590 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-04-20 10:55:47.779470087 +0000 2024-04-20T10:56:25.590 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-04-20 10:56:19.320086895 +0000 2024-04-20T10:56:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-04-20 10:55:47.779470087 +0000 2024-04-20T10:56:25.590 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-20T10:56:25.590 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-20T10:56:25.658 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:56:25.715 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-20T10:56:41.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:56:41.735 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:57:10.575 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.680+0000 7f084473b640 1 Processor -- start 2024-04-20T10:57:10.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.680+0000 7f084473b640 1 -- start start 2024-04-20T10:57:10.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.681+0000 7f084473b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f083c153410 0x7f083c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.681+0000 7f084473b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f083c1537f0 0x7f083c0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.681+0000 7f084473b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f083c15d6b0 con 0x7f083c159260 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.681+0000 7f084473b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f083c06cad0 con 0x7f083c153410 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.681+0000 7f084473b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0c7cd0 con 0x7f083c1537f0 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.681+0000 7f08424d7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f083c153410 0x7f083c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.681+0000 7f0842cd8640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f083c159260 0x7f083c06b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53286/0 (socket says 172.21.15.27:53286) 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.681+0000 7f0842cd8640 1 -- 172.21.15.27:0/3313378912 learned_addr learned my addr 172.21.15.27:0/3313378912 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.682+0000 7f0841cd6640 1 --2- 172.21.15.27:0/3313378912 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f083c1537f0 0x7f083c0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.682+0000 7f08424d7640 1 -- 172.21.15.27:0/3313378912 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f083c1537f0 msgr2=0x7f083c0c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:10.683 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.682+0000 7f08424d7640 1 --2- 172.21.15.27:0/3313378912 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f083c1537f0 0x7f083c0c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.682+0000 7f08424d7640 1 -- 172.21.15.27:0/3313378912 >> v1:172.21.15.27:6789/0 conn(0x7f083c159260 legacy=0x7f083c06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.682+0000 7f08414d5640 1 -- 172.21.15.27:0/3313378912 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056224357 0 0) 0x7f083c15d6b0 con 0x7f083c159260 2024-04-20T10:57:10.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.682+0000 7f08424d7640 1 -- 172.21.15.27:0/3313378912 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c0c7fb0 con 0x7f083c153410 2024-04-20T10:57:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f08424d7640 1 --2- 172.21.15.27:0/3313378912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 0x7f083c15d170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08380097a0 tx=0x7f083802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b48b1ba961ef33 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f08414d5640 1 -- 172.21.15.27:0/3313378912 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838034070 con 0x7f083c153410 2024-04-20T10:57:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f08414d5640 1 -- 172.21.15.27:0/3313378912 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0838030610 con 0x7f083c153410 2024-04-20T10:57:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f08414d5640 1 -- 172.21.15.27:0/3313378912 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08380309e0 con 0x7f083c153410 2024-04-20T10:57:10.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f084473b640 1 -- 172.21.15.27:0/3313378912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 msgr2=0x7f083c15d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f084473b640 1 --2- 172.21.15.27:0/3313378912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 0x7f083c15d170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08380097a0 tx=0x7f083802b960 comp rx=0 tx=0).stop 2024-04-20T10:57:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f084473b640 1 -- 172.21.15.27:0/3313378912 shutdown_connections 2024-04-20T10:57:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f084473b640 1 --2- 172.21.15.27:0/3313378912 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f083c1537f0 0x7f083c0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f084473b640 1 --2- 172.21.15.27:0/3313378912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 0x7f083c15d170 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f084473b640 1 -- 172.21.15.27:0/3313378912 >> 172.21.15.27:0/3313378912 conn(0x7f083c0ccbd0 msgr2=0x7f083c0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f084473b640 1 -- 172.21.15.27:0/3313378912 shutdown_connections 2024-04-20T10:57:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.683+0000 7f084473b640 1 -- 172.21.15.27:0/3313378912 wait complete. 2024-04-20T10:57:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f084473b640 1 Processor -- start 2024-04-20T10:57:10.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f084473b640 1 -- start start 2024-04-20T10:57:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f084473b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 0x7f083c15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f084473b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f083c1537f0 0x7f083c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f084473b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f083c159260 0x7f083c14f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:10.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f084473b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f083c15cc50 con 0x7f083c153410 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f084473b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f083c15c970 con 0x7f083c1537f0 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f084473b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0c75d0 con 0x7f083c159260 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f08424d7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 0x7f083c15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f08424d7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 0x7f083c15b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47534/0 (socket says 172.21.15.27:47534) 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.685+0000 7f08424d7640 1 -- 172.21.15.27:0/47048823 learned_addr learned my addr 172.21.15.27:0/47048823 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f08424d7640 1 -- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f083c159260 msgr2=0x7f083c14f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f08424d7640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f083c159260 0x7f083c14f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f08424d7640 1 -- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f083c1537f0 msgr2=0x7f083c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f08424d7640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f083c1537f0 0x7f083c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f08424d7640 1 -- 172.21.15.27:0/47048823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c15b1f0 con 0x7f083c153410 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f08424d7640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 0x7f083c15b7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f08380097a0 tx=0x7f08380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f0832ffd640 1 -- 172.21.15.27:0/47048823 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838034040 con 0x7f083c153410 2024-04-20T10:57:10.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f0832ffd640 1 -- 172.21.15.27:0/47048823 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0838004030 con 0x7f083c153410 2024-04-20T10:57:10.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f0832ffd640 1 -- 172.21.15.27:0/47048823 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838004420 con 0x7f083c153410 2024-04-20T10:57:10.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c1508c0 con 0x7f083c153410 2024-04-20T10:57:10.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.686+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c152340 con 0x7f083c153410 2024-04-20T10:57:10.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.687+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c152170 con 0x7f083c153410 2024-04-20T10:57:10.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.688+0000 7f0832ffd640 1 -- 172.21.15.27:0/47048823 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0838037070 con 0x7f083c153410 2024-04-20T10:57:10.690 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.688+0000 7f0832ffd640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f082006c8f0 0x7f082006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.688+0000 7f0832ffd640 1 -- 172.21.15.27:0/47048823 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f08380ad9b0 con 0x7f083c153410 2024-04-20T10:57:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.688+0000 7f0841cd6640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f082006c8f0 0x7f082006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:10.691 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.689+0000 7f0841cd6640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f082006c8f0 0x7f082006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:10.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.693+0000 7f0832ffd640 1 -- 172.21.15.27:0/47048823 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f083807bdf0 con 0x7f083c153410 2024-04-20T10:57:10.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.932+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f083c150de0 con 0x7f083c153410 2024-04-20T10:57:10.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.932+0000 7f0832ffd640 1 -- 172.21.15.27:0/47048823 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7f083c150de0 con 0x7f083c153410 2024-04-20T10:57:10.933 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:10.933 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:57:10.933 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:57:10.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f082006c8f0 msgr2=0x7f082006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:10.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f082006c8f0 0x7f082006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c008040 comp rx=0 tx=0).stop 2024-04-20T10:57:10.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 msgr2=0x7f083c15b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:10.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 0x7f083c15b7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f08380097a0 tx=0x7f08380046c0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 shutdown_connections 2024-04-20T10:57:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f082006c8f0 0x7f082006edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f083c159260 0x7f083c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f083c1537f0 0x7f083c15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 --2- 172.21.15.27:0/47048823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f083c153410 0x7f083c15b7b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 >> 172.21.15.27:0/47048823 conn(0x7f083c0ccbd0 msgr2=0x7f083c159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 shutdown_connections 2024-04-20T10:57:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:10.934+0000 7f084473b640 1 -- 172.21.15.27:0/47048823 wait complete. 2024-04-20T10:57:10.947 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=4) 2024-04-20T10:57:10.947 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T10:57:10.947 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:57:10.960 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T10:57:10.960 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.065+0000 7f24d9bb6640 1 Processor -- start 2024-04-20T10:57:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.065+0000 7f24d9bb6640 1 -- start start 2024-04-20T10:57:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d9bb6640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f24d415c860 0x7f24d415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d9bb6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f24d413e330 0x7f24d413e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d9bb6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24d413f2c0 con 0x7f24d413bab0 2024-04-20T10:57:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d9bb6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f24d40cc160 con 0x7f24d413e330 2024-04-20T10:57:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d9bb6640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f24d415d1e0 con 0x7f24d415c860 2024-04-20T10:57:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f24d413e330 0x7f24d413e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f24d413bab0 0x7f24d4070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53302/0 (socket says 172.21.15.27:53302) 2024-04-20T10:57:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d37fe640 1 -- 172.21.15.27:0/152442743 learned_addr learned my addr 172.21.15.27:0/152442743 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d2ffd640 1 --2- 172.21.15.27:0/152442743 >> v2:172.21.15.137:3300/0 conn(0x7f24d415c860 0x7f24d415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d3fff640 1 -- 172.21.15.27:0/152442743 >> v2:172.21.15.137:3300/0 conn(0x7f24d415c860 msgr2=0x7f24d415cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d3fff640 1 --2- 172.21.15.27:0/152442743 >> v2:172.21.15.137:3300/0 conn(0x7f24d415c860 0x7f24d415cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d3fff640 1 -- 172.21.15.27:0/152442743 >> v1:172.21.15.27:6789/0 conn(0x7f24d413bab0 legacy=0x7f24d4070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d27fc640 1 -- 172.21.15.27:0/152442743 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759384479 0 0) 0x7f24d413f2c0 con 0x7f24d413bab0 2024-04-20T10:57:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.066+0000 7f24d3fff640 1 -- 172.21.15.27:0/152442743 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d4159360 con 0x7f24d413e330 2024-04-20T10:57:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.067+0000 7f24d3fff640 1 --2- 172.21.15.27:0/152442743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d413e330 0x7f24d413e7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f24d4153aa0 tx=0x7f24cc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48c6a60cb09cd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.067+0000 7f24d27fc640 1 -- 172.21.15.27:0/152442743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc039070 con 0x7f24d413e330 2024-04-20T10:57:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.067+0000 7f24d27fc640 1 -- 172.21.15.27:0/152442743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24cc02cc80 con 0x7f24d413e330 2024-04-20T10:57:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.067+0000 7f24d27fc640 1 -- 172.21.15.27:0/152442743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc034660 con 0x7f24d413e330 2024-04-20T10:57:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.068+0000 7f24d9bb6640 1 -- 172.21.15.27:0/152442743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d413e330 msgr2=0x7f24d413e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.068+0000 7f24d9bb6640 1 --2- 172.21.15.27:0/152442743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d413e330 0x7f24d413e7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f24d4153aa0 tx=0x7f24cc02bed0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.068+0000 7f24d9bb6640 1 -- 172.21.15.27:0/152442743 shutdown_connections 2024-04-20T10:57:11.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.068+0000 7f24d9bb6640 1 --2- 172.21.15.27:0/152442743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d413e330 0x7f24d413e7c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.068+0000 7f24d9bb6640 1 --2- 172.21.15.27:0/152442743 >> v2:172.21.15.137:3300/0 conn(0x7f24d415c860 0x7f24d415cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.068+0000 7f24d9bb6640 1 -- 172.21.15.27:0/152442743 >> 172.21.15.27:0/152442743 conn(0x7f24d406c070 msgr2=0x7f24d40676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:11.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.068+0000 7f24d9bb6640 1 -- 172.21.15.27:0/152442743 shutdown_connections 2024-04-20T10:57:11.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.068+0000 7f24d9bb6640 1 -- 172.21.15.27:0/152442743 wait complete. 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.069+0000 7f24d9bb6640 1 Processor -- start 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.069+0000 7f24d9bb6640 1 -- start start 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d9bb6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d413bab0 0x7f24d4155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d9bb6640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24d413e330 0x7f24d4155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d9bb6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d415c860 0x7f24d40ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d9bb6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4159c00 con 0x7f24d415c860 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d9bb6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4159920 con 0x7f24d413bab0 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d9bb6640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f24d413be90 con 0x7f24d413e330 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d415c860 0x7f24d40ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d415c860 0x7f24d40ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47556/0 (socket says 172.21.15.27:47556) 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d3fff640 1 -- 172.21.15.27:0/269872585 learned_addr learned my addr 172.21.15.27:0/269872585 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d2ffd640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24d413e330 0x7f24d4155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d37fe640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d413bab0 0x7f24d4155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d3fff640 1 -- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24d413e330 msgr2=0x7f24d4155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d3fff640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24d413e330 0x7f24d4155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d3fff640 1 -- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d413bab0 msgr2=0x7f24d4155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.070+0000 7f24d3fff640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d413bab0 0x7f24d4155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.071+0000 7f24d3fff640 1 -- 172.21.15.27:0/269872585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d4154350 con 0x7f24d415c860 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.071+0000 7f24d2ffd640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24d413e330 0x7f24d4155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.071+0000 7f24d3fff640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d415c860 0x7f24d40ca5c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f24cc02c4e0 tx=0x7f24cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.071+0000 7f24bbfff640 1 -- 172.21.15.27:0/269872585 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc039050 con 0x7f24d415c860 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.071+0000 7f24bbfff640 1 -- 172.21.15.27:0/269872585 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24cc045070 con 0x7f24d415c860 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.071+0000 7f24bbfff640 1 -- 172.21.15.27:0/269872585 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc03e830 con 0x7f24d415c860 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.071+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d4154630 con 0x7f24d415c860 2024-04-20T10:57:11.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.071+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d4154bf0 con 0x7f24d415c860 2024-04-20T10:57:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.072+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d404fcf0 con 0x7f24d415c860 2024-04-20T10:57:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.072+0000 7f24bbfff640 1 -- 172.21.15.27:0/269872585 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24cc03e9d0 con 0x7f24d415c860 2024-04-20T10:57:11.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.073+0000 7f24bbfff640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f24c006c8f0 0x7f24c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:11.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.073+0000 7f24bbfff640 1 -- 172.21.15.27:0/269872585 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f24cc0ae960 con 0x7f24d415c860 2024-04-20T10:57:11.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.073+0000 7f24d37fe640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f24c006c8f0 0x7f24c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:11.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.074+0000 7f24d37fe640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f24c006c8f0 0x7f24c006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24c4000f60 tx=0x7f24c4007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:11.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.078+0000 7f24bbfff640 1 -- 172.21.15.27:0/269872585 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f24cc07ce20 con 0x7f24d415c860 2024-04-20T10:57:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.322+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24d40c72d0 con 0x7f24d415c860 2024-04-20T10:57:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.322+0000 7f24bbfff640 1 -- 172.21.15.27:0/269872585 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7f24cc07c720 con 0x7f24d415c860 2024-04-20T10:57:11.323 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:11.323 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:57:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:57:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f24c006c8f0 msgr2=0x7f24c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f24c006c8f0 0x7f24c006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24c4000f60 tx=0x7f24c4007b50 comp rx=0 tx=0).stop 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d415c860 msgr2=0x7f24d40ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d415c860 0x7f24d40ca5c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f24cc02c4e0 tx=0x7f24cc004a40 comp rx=0 tx=0).stop 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 shutdown_connections 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f24c006c8f0 0x7f24c006edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24d415c860 0x7f24d40ca5c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f24d413e330 0x7f24d4155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 --2- 172.21.15.27:0/269872585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d413bab0 0x7f24d4155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.324+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 >> 172.21.15.27:0/269872585 conn(0x7f24d406c070 msgr2=0x7f24d413a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.325+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 shutdown_connections 2024-04-20T10:57:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:11.325+0000 7f24d9bb6640 1 -- 172.21.15.27:0/269872585 wait complete. 2024-04-20T10:57:11.336 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-20T10:57:11.853 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:57:11.856 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:57:13.337 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:13.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.439+0000 7f883f3ed640 1 Processor -- start 2024-04-20T10:57:13.440 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.439+0000 7f883f3ed640 1 -- start start 2024-04-20T10:57:13.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883f3ed640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8838155950 0x7f88380cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:13.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883f3ed640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f88380cc330 0x7f88380b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:13.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883f3ed640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f88380b5a30 0x7f88380d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:13.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883f3ed640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f88380b5ea0 con 0x7f88380b5a30 2024-04-20T10:57:13.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883f3ed640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f883815b9c0 con 0x7f88380cc330 2024-04-20T10:57:13.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883f3ed640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f883815bca0 con 0x7f8838155950 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883d98a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f88380b5a30 0x7f88380d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883d98a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f88380b5a30 0x7f88380d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47566/0 (socket says 172.21.15.27:47566) 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883d98a640 1 -- 172.21.15.27:0/1240497139 learned_addr learned my addr 172.21.15.27:0/1240497139 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883d189640 1 --2- 172.21.15.27:0/1240497139 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8838155950 0x7f88380cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883c988640 1 --2- 172.21.15.27:0/1240497139 >> v2:172.21.15.137:3300/0 conn(0x7f88380cc330 0x7f88380b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883d98a640 1 -- 172.21.15.27:0/1240497139 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8838155950 msgr2=0x7f88380cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883d98a640 1 --2- 172.21.15.27:0/1240497139 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8838155950 0x7f88380cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883d98a640 1 -- 172.21.15.27:0/1240497139 >> v2:172.21.15.137:3300/0 conn(0x7f88380cc330 msgr2=0x7f88380b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883d98a640 1 --2- 172.21.15.27:0/1240497139 >> v2:172.21.15.137:3300/0 conn(0x7f88380cc330 0x7f88380b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.440+0000 7f883d98a640 1 -- 172.21.15.27:0/1240497139 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883815bf80 con 0x7f88380b5a30 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.441+0000 7f883d98a640 1 --2- 172.21.15.27:0/1240497139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 0x7f88380d0c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f88380ce840 tx=0x7f883002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87e40a04f944edd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.441+0000 7f882ffff640 1 -- 172.21.15.27:0/1240497139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883002cb00 con 0x7f88380b5a30 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.441+0000 7f882ffff640 1 -- 172.21.15.27:0/1240497139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f883002cca0 con 0x7f88380b5a30 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.441+0000 7f882ffff640 1 -- 172.21.15.27:0/1240497139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830034610 con 0x7f88380b5a30 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.442+0000 7f883f3ed640 1 -- 172.21.15.27:0/1240497139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 msgr2=0x7f88380d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.442+0000 7f883f3ed640 1 --2- 172.21.15.27:0/1240497139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 0x7f88380d0c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f88380ce840 tx=0x7f883002bf60 comp rx=0 tx=0).stop 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.442+0000 7f883f3ed640 1 -- 172.21.15.27:0/1240497139 shutdown_connections 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.442+0000 7f883f3ed640 1 --2- 172.21.15.27:0/1240497139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 0x7f88380d0c90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.442+0000 7f883f3ed640 1 --2- 172.21.15.27:0/1240497139 >> v2:172.21.15.137:3300/0 conn(0x7f88380cc330 0x7f88380b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.442+0000 7f883f3ed640 1 --2- 172.21.15.27:0/1240497139 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8838155950 0x7f88380cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.442+0000 7f883f3ed640 1 -- 172.21.15.27:0/1240497139 >> 172.21.15.27:0/1240497139 conn(0x7f88380676c0 msgr2=0x7f8838067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:13.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.442+0000 7f883f3ed640 1 -- 172.21.15.27:0/1240497139 shutdown_connections 2024-04-20T10:57:13.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.442+0000 7f883f3ed640 1 -- 172.21.15.27:0/1240497139 wait complete. 2024-04-20T10:57:13.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.443+0000 7f883f3ed640 1 Processor -- start 2024-04-20T10:57:13.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.443+0000 7f883f3ed640 1 -- start start 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883f3ed640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 0x7f8838151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883f3ed640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f88380cc330 0x7f88381f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883f3ed640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8838155950 0x7f88381f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883f3ed640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f883815fb90 con 0x7f88380b5a30 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883f3ed640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f883815c540 con 0x7f8838155950 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883f3ed640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f883815fd10 con 0x7f88380cc330 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d189640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 0x7f8838151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d189640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 0x7f8838151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47572/0 (socket says 172.21.15.27:47572) 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d189640 1 -- 172.21.15.27:0/2641324251 learned_addr learned my addr 172.21.15.27:0/2641324251 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883c988640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f88380cc330 0x7f88381f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d98a640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8838155950 0x7f88381f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d189640 1 -- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f88380cc330 msgr2=0x7f88381f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d189640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f88380cc330 0x7f88381f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d189640 1 -- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8838155950 msgr2=0x7f88381f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d189640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8838155950 0x7f88381f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d189640 1 -- 172.21.15.27:0/2641324251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883815caa0 con 0x7f88380b5a30 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883c988640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f88380cc330 0x7f88381f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d98a640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8838155950 0x7f88381f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.444+0000 7f883d189640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 0x7f8838151a80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f883400d620 tx=0x7f883400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.445+0000 7f882dffb640 1 -- 172.21.15.27:0/2641324251 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883400b620 con 0x7f88380b5a30 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.445+0000 7f882dffb640 1 -- 172.21.15.27:0/2641324251 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8834019070 con 0x7f88380b5a30 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.445+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883815cd80 con 0x7f88380b5a30 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.445+0000 7f882dffb640 1 -- 172.21.15.27:0/2641324251 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883400ba00 con 0x7f88380b5a30 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.445+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883815d340 con 0x7f88380b5a30 2024-04-20T10:57:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.446+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88380c7bb0 con 0x7f88380b5a30 2024-04-20T10:57:13.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.446+0000 7f882dffb640 1 -- 172.21.15.27:0/2641324251 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f883400bba0 con 0x7f88380b5a30 2024-04-20T10:57:13.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.447+0000 7f882dffb640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f881c06c8f0 0x7f881c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:13.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.447+0000 7f882dffb640 1 -- 172.21.15.27:0/2641324251 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f883408bf00 con 0x7f88380b5a30 2024-04-20T10:57:13.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.447+0000 7f883c988640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f881c06c8f0 0x7f881c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.448+0000 7f883c988640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f881c06c8f0 0x7f881c06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f88381592d0 tx=0x7f8828008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:13.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.451+0000 7f882dffb640 1 -- 172.21.15.27:0/2641324251 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f88380c7bb0 con 0x7f88380b5a30 2024-04-20T10:57:13.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.674+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f883804fcf0 con 0x7f88380b5a30 2024-04-20T10:57:13.675 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.674+0000 7f882dffb640 1 -- 172.21.15.27:0/2641324251 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7f883405a340 con 0x7f88380b5a30 2024-04-20T10:57:13.675 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:13.675 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:57:13.676 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:57:13.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f881c06c8f0 msgr2=0x7f881c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:13.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f881c06c8f0 0x7f881c06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f88381592d0 tx=0x7f8828008040 comp rx=0 tx=0).stop 2024-04-20T10:57:13.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 msgr2=0x7f8838151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:13.678 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 0x7f8838151a80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f883400d620 tx=0x7f883400dae0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 shutdown_connections 2024-04-20T10:57:13.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f881c06c8f0 0x7f881c06edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8838155950 0x7f88381f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f88380cc330 0x7f88381f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 --2- 172.21.15.27:0/2641324251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380b5a30 0x7f8838151a80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:13.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 >> 172.21.15.27:0/2641324251 conn(0x7f88380676c0 msgr2=0x7f88380cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:13.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.677+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 shutdown_connections 2024-04-20T10:57:13.679 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:13.678+0000 7f883f3ed640 1 -- 172.21.15.27:0/2641324251 wait complete. 2024-04-20T10:57:13.693 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-20T10:57:15.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.796+0000 7fd49e88c640 1 Processor -- start 2024-04-20T10:57:15.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.796+0000 7fd49e88c640 1 -- start start 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.797+0000 7fd49e88c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd49815c860 0x7fd49815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.797+0000 7fd49e88c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fd49813e330 0x7fd49813e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.797+0000 7fd49e88c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd49813f2c0 con 0x7fd49813bab0 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.797+0000 7fd49e88c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd4980cc160 con 0x7fd49815c860 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.797+0000 7fd49e88c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fd49815d1e0 con 0x7fd49813e330 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.797+0000 7fd4977fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd49815c860 0x7fd49815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.797+0000 7fd497fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd49813bab0 0x7fd498070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40100/0 (socket says 172.21.15.27:40100) 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.797+0000 7fd497fff640 1 -- 172.21.15.27:0/889502349 learned_addr learned my addr 172.21.15.27:0/889502349 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.798+0000 7fd49ce29640 1 --2- 172.21.15.27:0/889502349 >> v2:172.21.15.137:3300/0 conn(0x7fd49813e330 0x7fd49813e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.798+0000 7fd4977fe640 1 -- 172.21.15.27:0/889502349 >> v2:172.21.15.137:3300/0 conn(0x7fd49813e330 msgr2=0x7fd49813e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.798+0000 7fd4977fe640 1 --2- 172.21.15.27:0/889502349 >> v2:172.21.15.137:3300/0 conn(0x7fd49813e330 0x7fd49813e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.798+0000 7fd4977fe640 1 -- 172.21.15.27:0/889502349 >> v1:172.21.15.27:6789/0 conn(0x7fd49813bab0 legacy=0x7fd498070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:15.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.798+0000 7fd4977fe640 1 -- 172.21.15.27:0/889502349 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd498159360 con 0x7fd49815c860 2024-04-20T10:57:15.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.798+0000 7fd496ffd640 1 -- 172.21.15.27:0/889502349 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463711580 0 0) 0x7fd49813f2c0 con 0x7fd49813bab0 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd4977fe640 1 --2- 172.21.15.27:0/889502349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49815c860 0x7fd49815cca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd4980cbb60 tx=0x7fd48c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4d8d7b27d48bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd496ffd640 1 -- 172.21.15.27:0/889502349 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c035020 con 0x7fd49815c860 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd496ffd640 1 -- 172.21.15.27:0/889502349 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd48c031830 con 0x7fd49815c860 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd496ffd640 1 -- 172.21.15.27:0/889502349 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c031c00 con 0x7fd49815c860 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd49e88c640 1 -- 172.21.15.27:0/889502349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49815c860 msgr2=0x7fd49815cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd49e88c640 1 --2- 172.21.15.27:0/889502349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49815c860 0x7fd49815cca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd4980cbb60 tx=0x7fd48c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd49e88c640 1 -- 172.21.15.27:0/889502349 shutdown_connections 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd49e88c640 1 --2- 172.21.15.27:0/889502349 >> v2:172.21.15.137:3300/0 conn(0x7fd49813e330 0x7fd49813e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd49e88c640 1 --2- 172.21.15.27:0/889502349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49815c860 0x7fd49815cca0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd49e88c640 1 -- 172.21.15.27:0/889502349 >> 172.21.15.27:0/889502349 conn(0x7fd49806c070 msgr2=0x7fd4980676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd49e88c640 1 -- 172.21.15.27:0/889502349 shutdown_connections 2024-04-20T10:57:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.799+0000 7fd49e88c640 1 -- 172.21.15.27:0/889502349 wait complete. 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.801+0000 7fd49e88c640 1 Processor -- start 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.801+0000 7fd49e88c640 1 -- start start 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.801+0000 7fd49e88c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49813bab0 0x7fd49815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.801+0000 7fd49e88c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd49813e330 0x7fd498136d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.801+0000 7fd49e88c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd49815c860 0x7fd498151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.801+0000 7fd49e88c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd498159c00 con 0x7fd49813bab0 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.801+0000 7fd49e88c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd498159920 con 0x7fd49815c860 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.801+0000 7fd49e88c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd49813be90 con 0x7fd49813e330 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd497fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49813bab0 0x7fd49815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd497fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49813bab0 0x7fd49815b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52818/0 (socket says 172.21.15.27:52818) 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd497fff640 1 -- 172.21.15.27:0/925370860 learned_addr learned my addr 172.21.15.27:0/925370860 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd49ce29640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd49815c860 0x7fd498151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:15.804 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd4977fe640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd49813e330 0x7fd498136d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd497fff640 1 -- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd49813e330 msgr2=0x7fd498136d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd497fff640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd49813e330 0x7fd498136d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd497fff640 1 -- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd49815c860 msgr2=0x7fd498151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd497fff640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd49815c860 0x7fd498151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd497fff640 1 -- 172.21.15.27:0/925370860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd498157fe0 con 0x7fd49813bab0 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd497fff640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49813bab0 0x7fd49815b8c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd48800c4d0 tx=0x7fd48800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd494ff9640 1 -- 172.21.15.27:0/925370860 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488018070 con 0x7fd49813bab0 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd494ff9640 1 -- 172.21.15.27:0/925370860 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4880074e0 con 0x7fd49813bab0 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd494ff9640 1 -- 172.21.15.27:0/925370860 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488004a20 con 0x7fd49813bab0 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49813de30 con 0x7fd49813bab0 2024-04-20T10:57:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.802+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49813d070 con 0x7fd49813bab0 2024-04-20T10:57:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.803+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49804fcf0 con 0x7fd49813bab0 2024-04-20T10:57:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.804+0000 7fd494ff9640 1 -- 172.21.15.27:0/925370860 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd488004bc0 con 0x7fd49813bab0 2024-04-20T10:57:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.805+0000 7fd494ff9640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c920 0x7fd48406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.805+0000 7fd494ff9640 1 -- 172.21.15.27:0/925370860 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd48808ade0 con 0x7fd49813bab0 2024-04-20T10:57:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.805+0000 7fd4977fe640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c920 0x7fd48406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.806+0000 7fd4977fe640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c920 0x7fd48406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd48c036000 tx=0x7fd48c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:15.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:15.810+0000 7fd494ff9640 1 -- 172.21.15.27:0/925370860 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd488058b50 con 0x7fd49813bab0 2024-04-20T10:57:16.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.059+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4980cc500 con 0x7fd49813bab0 2024-04-20T10:57:16.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.059+0000 7fd494ff9640 1 -- 172.21.15.27:0/925370860 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7fd48805ce50 con 0x7fd49813bab0 2024-04-20T10:57:16.060 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:16.060 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:57:16.060 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:57:16.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c920 msgr2=0x7fd48406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:16.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c920 0x7fd48406ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd48c036000 tx=0x7fd48c03c040 comp rx=0 tx=0).stop 2024-04-20T10:57:16.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49813bab0 msgr2=0x7fd49815b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49813bab0 0x7fd49815b8c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd48800c4d0 tx=0x7fd48800c990 comp rx=0 tx=0).stop 2024-04-20T10:57:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 shutdown_connections 2024-04-20T10:57:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd48406c920 0x7fd48406ede0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd49815c860 0x7fd498151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd49813e330 0x7fd498136d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 --2- 172.21.15.27:0/925370860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd49813bab0 0x7fd49815b8c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.061+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 >> 172.21.15.27:0/925370860 conn(0x7fd49806c070 msgr2=0x7fd4980c5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.062+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 shutdown_connections 2024-04-20T10:57:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:16.062+0000 7fd49e88c640 1 -- 172.21.15.27:0/925370860 wait complete. 2024-04-20T10:57:16.074 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-20T10:57:18.075 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:18.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.177+0000 7f60fc643640 1 Processor -- start 2024-04-20T10:57:18.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.177+0000 7f60fc643640 1 -- start start 2024-04-20T10:57:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.178+0000 7f60fc643640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f4155600 0x7f60f415b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fc643640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f60f40cb010 0x7f60f4070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fc643640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f40cf4f0 con 0x7f60f406d6c0 2024-04-20T10:57:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fc643640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f60f406d230 con 0x7f60f40cb010 2024-04-20T10:57:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fc643640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f60f40cc1a0 con 0x7f60f4155600 2024-04-20T10:57:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fabe0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f60f40cb010 0x7f60f4070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fabe0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f60f40cb010 0x7f60f4070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52830/0 (socket says 172.21.15.27:52830) 2024-04-20T10:57:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fabe0640 1 -- 172.21.15.27:0/3708256031 learned_addr learned my addr 172.21.15.27:0/3708256031 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fa3df640 1 --2- 172.21.15.27:0/3708256031 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f4155600 0x7f60f415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fabe0640 1 -- 172.21.15.27:0/3708256031 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f4155600 msgr2=0x7f60f415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fabe0640 1 --2- 172.21.15.27:0/3708256031 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f4155600 0x7f60f415b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fabe0640 1 -- 172.21.15.27:0/3708256031 >> v1:172.21.15.137:6789/0 conn(0x7f60f406d6c0 legacy=0x7f60f40ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.179+0000 7f60fabe0640 1 -- 172.21.15.27:0/3708256031 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f40cf250 con 0x7f60f40cb010 2024-04-20T10:57:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.180+0000 7f60fabe0640 1 --2- 172.21.15.27:0/3708256031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 0x7f60f4070c00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009e50 tx=0x7f60ec02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2243420053ed37 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.180+0000 7f60f93dd640 1 -- 172.21.15.27:0/3708256031 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec02cb90 con 0x7f60f40cb010 2024-04-20T10:57:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.180+0000 7f60f93dd640 1 -- 172.21.15.27:0/3708256031 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec02cd30 con 0x7f60f40cb010 2024-04-20T10:57:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.180+0000 7f60f93dd640 1 -- 172.21.15.27:0/3708256031 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec034610 con 0x7f60f40cb010 2024-04-20T10:57:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.181+0000 7f60fc643640 1 -- 172.21.15.27:0/3708256031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 msgr2=0x7f60f4070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.181+0000 7f60fc643640 1 --2- 172.21.15.27:0/3708256031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 0x7f60f4070c00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009e50 tx=0x7f60ec02bff0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.181+0000 7f60fc643640 1 -- 172.21.15.27:0/3708256031 shutdown_connections 2024-04-20T10:57:18.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.181+0000 7f60fc643640 1 --2- 172.21.15.27:0/3708256031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 0x7f60f4070c00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.181+0000 7f60fc643640 1 --2- 172.21.15.27:0/3708256031 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f4155600 0x7f60f415b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.181+0000 7f60fc643640 1 -- 172.21.15.27:0/3708256031 >> 172.21.15.27:0/3708256031 conn(0x7f60f406c070 msgr2=0x7f60f40676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:18.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.181+0000 7f60fc643640 1 -- 172.21.15.27:0/3708256031 shutdown_connections 2024-04-20T10:57:18.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.181+0000 7f60fc643640 1 -- 172.21.15.27:0/3708256031 wait complete. 2024-04-20T10:57:18.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.182+0000 7f60fc643640 1 Processor -- start 2024-04-20T10:57:18.183 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.182+0000 7f60fc643640 1 -- start start 2024-04-20T10:57:18.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60fc643640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f406d6c0 0x7f60f414d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:18.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60fc643640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 0x7f60f4153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:18.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60fc643640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60f4155600 0x7f60f41f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:18.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60fc643640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f60f406cec0 con 0x7f60f40cb010 2024-04-20T10:57:18.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60fc643640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60f406cbe0 con 0x7f60f4155600 2024-04-20T10:57:18.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60fc643640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f60f415b960 con 0x7f60f406d6c0 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60f9bde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 0x7f60f4153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60f9bde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 0x7f60f4153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52832/0 (socket says 172.21.15.27:52832) 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60f9bde640 1 -- 172.21.15.27:0/842770800 learned_addr learned my addr 172.21.15.27:0/842770800 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60fabe0640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60f4155600 0x7f60f41f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.183+0000 7f60fa3df640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f406d6c0 0x7f60f414d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60f9bde640 1 -- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f406d6c0 msgr2=0x7f60f414d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60f9bde640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f406d6c0 0x7f60f414d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60f9bde640 1 -- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60f4155600 msgr2=0x7f60f41f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60f9bde640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60f4155600 0x7f60f41f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60f9bde640 1 -- 172.21.15.27:0/842770800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f4158200 con 0x7f60f40cb010 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60fabe0640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60f4155600 0x7f60f41f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60f9bde640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 0x7f60f4153db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f60e400c430 tx=0x7f60e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60eaffd640 1 -- 172.21.15.27:0/842770800 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4017070 con 0x7f60f40cb010 2024-04-20T10:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60eaffd640 1 -- 172.21.15.27:0/842770800 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e40052d0 con 0x7f60f40cb010 2024-04-20T10:57:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f41542f0 con 0x7f60f40cb010 2024-04-20T10:57:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60eaffd640 1 -- 172.21.15.27:0/842770800 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4003850 con 0x7f60f40cb010 2024-04-20T10:57:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.184+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f4158470 con 0x7f60f40cb010 2024-04-20T10:57:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.185+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f40c7960 con 0x7f60f40cb010 2024-04-20T10:57:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.186+0000 7f60eaffd640 1 -- 172.21.15.27:0/842770800 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60e401a070 con 0x7f60f40cb010 2024-04-20T10:57:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.186+0000 7f60eaffd640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f60d806c8f0 0x7f60d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.186+0000 7f60eaffd640 1 -- 172.21.15.27:0/842770800 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f60e408aee0 con 0x7f60f40cb010 2024-04-20T10:57:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.186+0000 7f60fa3df640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f60d806c8f0 0x7f60d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.187+0000 7f60fa3df640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f60d806c8f0 0x7f60d806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60f00079a0 tx=0x7f60f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:18.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.191+0000 7f60eaffd640 1 -- 172.21.15.27:0/842770800 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f60f40c7960 con 0x7f60f40cb010 2024-04-20T10:57:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.419+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60f40635c0 con 0x7f60f40cb010 2024-04-20T10:57:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.420+0000 7f60eaffd640 1 -- 172.21.15.27:0/842770800 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7f60f40635c0 con 0x7f60f40cb010 2024-04-20T10:57:18.421 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:18.421 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:57:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:57:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f60d806c8f0 msgr2=0x7f60d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f60d806c8f0 0x7f60d806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60f00079a0 tx=0x7f60f0008040 comp rx=0 tx=0).stop 2024-04-20T10:57:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 msgr2=0x7f60f4153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 0x7f60f4153db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f60e400c430 tx=0x7f60e400c8f0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 shutdown_connections 2024-04-20T10:57:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f60d806c8f0 0x7f60d806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60f4155600 0x7f60f41f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60f40cb010 0x7f60f4153db0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 --2- 172.21.15.27:0/842770800 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f60f406d6c0 0x7f60f414d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 >> 172.21.15.27:0/842770800 conn(0x7f60f406c070 msgr2=0x7f60f40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 shutdown_connections 2024-04-20T10:57:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:18.422+0000 7f60fc643640 1 -- 172.21.15.27:0/842770800 wait complete. 2024-04-20T10:57:18.434 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-20T10:57:20.435 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:20.536 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.536+0000 7f49fbc69640 1 Processor -- start 2024-04-20T10:57:20.537 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.536+0000 7f49fbc69640 1 -- start start 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.537+0000 7f49fbc69640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f40caa80 0x7f49f40c8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.537+0000 7f49fbc69640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f49f4072640 0x7f49f40c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.537+0000 7f49fbc69640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f4067ae0 con 0x7f49f4152110 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.537+0000 7f49fbc69640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f49f40721b0 con 0x7f49f4072640 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.537+0000 7f49fbc69640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f49f4071880 con 0x7f49f40caa80 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49fa206640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f49f4072640 0x7f49f40c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49fa206640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f49f4072640 0x7f49f40c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:35046/0 (socket says 172.21.15.27:35046) 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49fa206640 1 -- 172.21.15.27:0/2256446764 learned_addr learned my addr 172.21.15.27:0/2256446764 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49f9a05640 1 --1- 172.21.15.27:0/2256446764 >> v1:172.21.15.137:6789/0 conn(0x7f49f4152110 0x7f49f4158be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:36598/0 (socket says 172.21.15.27:36598) 2024-04-20T10:57:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49f9204640 1 --2- 172.21.15.27:0/2256446764 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f40caa80 0x7f49f40c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49fa206640 1 -- 172.21.15.27:0/2256446764 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f40caa80 msgr2=0x7f49f40c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49fa206640 1 --2- 172.21.15.27:0/2256446764 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f40caa80 0x7f49f40c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49fa206640 1 -- 172.21.15.27:0/2256446764 >> v1:172.21.15.137:6789/0 conn(0x7f49f4152110 legacy=0x7f49f4158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49f8a03640 1 -- 172.21.15.27:0/2256446764 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258513362 0 0) 0x7f49f4067ae0 con 0x7f49f4152110 2024-04-20T10:57:20.540 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.538+0000 7f49fa206640 1 -- 172.21.15.27:0/2256446764 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f4071b60 con 0x7f49f4072640 2024-04-20T10:57:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.539+0000 7f49fa206640 1 --2- 172.21.15.27:0/2256446764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 0x7f49f40c9dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00d150 tx=0x7f49ec02e160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a070e944280a130 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.539+0000 7f49f8a03640 1 -- 172.21.15.27:0/2256446764 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00b0d0 con 0x7f49f4072640 2024-04-20T10:57:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.539+0000 7f49f8a03640 1 -- 172.21.15.27:0/2256446764 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ec00b270 con 0x7f49f4072640 2024-04-20T10:57:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.539+0000 7f49f8a03640 1 -- 172.21.15.27:0/2256446764 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec009600 con 0x7f49f4072640 2024-04-20T10:57:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.540+0000 7f49fbc69640 1 -- 172.21.15.27:0/2256446764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 msgr2=0x7f49f40c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.540+0000 7f49fbc69640 1 --2- 172.21.15.27:0/2256446764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 0x7f49f40c9dd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00d150 tx=0x7f49ec02e160 comp rx=0 tx=0).stop 2024-04-20T10:57:20.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.540+0000 7f49fbc69640 1 -- 172.21.15.27:0/2256446764 shutdown_connections 2024-04-20T10:57:20.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.540+0000 7f49fbc69640 1 --2- 172.21.15.27:0/2256446764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 0x7f49f40c9dd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.540+0000 7f49fbc69640 1 --2- 172.21.15.27:0/2256446764 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f40caa80 0x7f49f40c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.540+0000 7f49fbc69640 1 -- 172.21.15.27:0/2256446764 >> 172.21.15.27:0/2256446764 conn(0x7f49f406b930 msgr2=0x7f49f406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:20.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.540+0000 7f49fbc69640 1 -- 172.21.15.27:0/2256446764 shutdown_connections 2024-04-20T10:57:20.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.540+0000 7f49fbc69640 1 -- 172.21.15.27:0/2256446764 wait complete. 2024-04-20T10:57:20.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.542+0000 7f49fbc69640 1 Processor -- start 2024-04-20T10:57:20.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.542+0000 7f49fbc69640 1 -- start start 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49fbc69640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 0x7f49f41efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49fbc69640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f40caa80 0x7f49f41f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49fbc69640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f4152110 0x7f49f4154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49fbc69640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4152670 con 0x7f49f40caa80 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49fbc69640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41524f0 con 0x7f49f4072640 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49fbc69640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f49f40676c0 con 0x7f49f4152110 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9a05640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 0x7f49f41efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9204640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f40caa80 0x7f49f41f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9204640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f40caa80 0x7f49f41f0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52842/0 (socket says 172.21.15.27:52842) 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9204640 1 -- 172.21.15.27:0/3949056753 learned_addr learned my addr 172.21.15.27:0/3949056753 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:20.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49fa206640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f4152110 0x7f49f4154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9204640 1 -- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f4152110 msgr2=0x7f49f4154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9204640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f4152110 0x7f49f4154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9204640 1 -- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 msgr2=0x7f49f41efd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9204640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 0x7f49f41efd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9204640 1 -- 172.21.15.27:0/3949056753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f4153390 con 0x7f49f40caa80 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.544+0000 7f49f9a05640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 0x7f49f41efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.545+0000 7f49f9204640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f40caa80 0x7f49f41f0280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009f60 tx=0x7f49e4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.545+0000 7f49ea7fc640 1 -- 172.21.15.27:0/3949056753 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4020070 con 0x7f49f40caa80 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.545+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f41f08c0 con 0x7f49f40caa80 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.545+0000 7f49ea7fc640 1 -- 172.21.15.27:0/3949056753 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e400ca00 con 0x7f49f40caa80 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.545+0000 7f49ea7fc640 1 -- 172.21.15.27:0/3949056753 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400cdd0 con 0x7f49f40caa80 2024-04-20T10:57:20.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.545+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f4152bd0 con 0x7f49f40caa80 2024-04-20T10:57:20.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.546+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f40ce200 con 0x7f49f40caa80 2024-04-20T10:57:20.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.547+0000 7f49ea7fc640 1 -- 172.21.15.27:0/3949056753 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49e4023070 con 0x7f49f40caa80 2024-04-20T10:57:20.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.547+0000 7f49ea7fc640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49d806c8f0 0x7f49d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:20.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.547+0000 7f49ea7fc640 1 -- 172.21.15.27:0/3949056753 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f49e408fac0 con 0x7f49f40caa80 2024-04-20T10:57:20.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.547+0000 7f49f9a05640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49d806c8f0 0x7f49d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:20.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.548+0000 7f49f9a05640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49d806c8f0 0x7f49d806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007fd0 tx=0x7f49f0007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:20.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.555+0000 7f49ea7fc640 1 -- 172.21.15.27:0/3949056753 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f49e405d710 con 0x7f49f40caa80 2024-04-20T10:57:20.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.905+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49f404fcf0 con 0x7f49f40caa80 2024-04-20T10:57:20.906 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.906+0000 7f49ea7fc640 1 -- 172.21.15.27:0/3949056753 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7f49e4061a10 con 0x7f49f40caa80 2024-04-20T10:57:20.906 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:20.907 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:57:20.907 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.907+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49d806c8f0 msgr2=0x7f49d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.907+0000 7f49fbc69640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49d806c8f0 0x7f49d806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007fd0 tx=0x7f49f0007de0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f40caa80 msgr2=0x7f49f41f0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f40caa80 0x7f49f41f0280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009f60 tx=0x7f49e4009f90 comp rx=0 tx=0).stop 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 shutdown_connections 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49d806c8f0 0x7f49d806edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49f4152110 0x7f49f4154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f40caa80 0x7f49f41f0280 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 --2- 172.21.15.27:0/3949056753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4072640 0x7f49f41efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:20.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 >> 172.21.15.27:0/3949056753 conn(0x7f49f406b930 msgr2=0x7f49f414fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:20.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 shutdown_connections 2024-04-20T10:57:20.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:20.908+0000 7f49fbc69640 1 -- 172.21.15.27:0/3949056753 wait complete. 2024-04-20T10:57:20.919 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-20T10:57:22.921 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.028+0000 7f133e58e640 1 Processor -- start 2024-04-20T10:57:23.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.028+0000 7f133e58e640 1 -- start start 2024-04-20T10:57:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f133e58e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f13380c8dd0 0x7f133815c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f133e58e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f1338071200 0x7f1338071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f133e58e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1338155b20 con 0x7f13380ceb10 2024-04-20T10:57:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f133e58e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f133815edb0 con 0x7f13380c8dd0 2024-04-20T10:57:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f133e58e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f133815f050 con 0x7f1338071200 2024-04-20T10:57:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f133cb2b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f13380ceb10 0x7f133815cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40104/0 (socket says 172.21.15.27:40104) 2024-04-20T10:57:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f133cb2b640 1 -- 172.21.15.27:0/661216728 learned_addr learned my addr 172.21.15.27:0/661216728 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f13377fe640 1 --2- 172.21.15.27:0/661216728 >> v2:172.21.15.137:3300/0 conn(0x7f1338071200 0x7f1338071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f1337fff640 1 --2- 172.21.15.27:0/661216728 >> v2:172.21.15.27:3300/0 conn(0x7f13380c8dd0 0x7f133815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f1336ffd640 1 -- 172.21.15.27:0/661216728 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066239991 0 0) 0x7f1330001490 con 0x7f13380ceb10 2024-04-20T10:57:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f1336ffd640 1 -- 172.21.15.27:0/661216728 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1320003520 con 0x7f13380ceb10 2024-04-20T10:57:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f1337fff640 1 -- 172.21.15.27:0/661216728 >> v2:172.21.15.137:3300/0 conn(0x7f1338071200 msgr2=0x7f1338071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f1337fff640 1 --2- 172.21.15.27:0/661216728 >> v2:172.21.15.137:3300/0 conn(0x7f1338071200 0x7f1338071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f1337fff640 1 -- 172.21.15.27:0/661216728 >> v1:172.21.15.27:6789/0 conn(0x7f13380ceb10 legacy=0x7f133815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.029+0000 7f1337fff640 1 -- 172.21.15.27:0/661216728 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133815f330 con 0x7f13380c8dd0 2024-04-20T10:57:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f1337fff640 1 --2- 172.21.15.27:0/661216728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13380c8dd0 0x7f133815c250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f133806ec20 tx=0x7f132802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03e97fb79e0632c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f1336ffd640 1 -- 172.21.15.27:0/661216728 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280315a0 con 0x7f13380c8dd0 2024-04-20T10:57:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f1336ffd640 1 -- 172.21.15.27:0/661216728 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1328037070 con 0x7f13380c8dd0 2024-04-20T10:57:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f1336ffd640 1 -- 172.21.15.27:0/661216728 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328031960 con 0x7f13380c8dd0 2024-04-20T10:57:23.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f133e58e640 1 -- 172.21.15.27:0/661216728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13380c8dd0 msgr2=0x7f133815c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f133e58e640 1 --2- 172.21.15.27:0/661216728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13380c8dd0 0x7f133815c250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f133806ec20 tx=0x7f132802b840 comp rx=0 tx=0).stop 2024-04-20T10:57:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f133e58e640 1 -- 172.21.15.27:0/661216728 shutdown_connections 2024-04-20T10:57:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f133e58e640 1 --2- 172.21.15.27:0/661216728 >> v2:172.21.15.137:3300/0 conn(0x7f1338071200 0x7f1338071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f133e58e640 1 --2- 172.21.15.27:0/661216728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13380c8dd0 0x7f133815c250 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.031+0000 7f133e58e640 1 -- 172.21.15.27:0/661216728 >> 172.21.15.27:0/661216728 conn(0x7f13380676c0 msgr2=0x7f1338067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.032+0000 7f133e58e640 1 -- 172.21.15.27:0/661216728 shutdown_connections 2024-04-20T10:57:23.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.032+0000 7f133e58e640 1 -- 172.21.15.27:0/661216728 wait complete. 2024-04-20T10:57:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.033+0000 7f133e58e640 1 Processor -- start 2024-04-20T10:57:23.033 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.034+0000 7f133e58e640 1 -- start start 2024-04-20T10:57:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133e58e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1338071200 0x7f13380cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133e58e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13380c8dd0 0x7f13381f1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133e58e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13380ceb10 0x7f13381f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133e58e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f133815d5b0 con 0x7f1338071200 2024-04-20T10:57:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133e58e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f133815f8f0 con 0x7f13380ceb10 2024-04-20T10:57:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133e58e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f133815d730 con 0x7f13380c8dd0 2024-04-20T10:57:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f1337fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1338071200 0x7f13380cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f1337fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1338071200 0x7f13380cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52872/0 (socket says 172.21.15.27:52872) 2024-04-20T10:57:23.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f1337fff640 1 -- 172.21.15.27:0/49733663 learned_addr learned my addr 172.21.15.27:0/49733663 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133cb2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13380ceb10 0x7f13381f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f13377fe640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13380c8dd0 0x7f13381f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133cb2b640 1 -- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13380c8dd0 msgr2=0x7f13381f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133cb2b640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13380c8dd0 0x7f13381f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133cb2b640 1 -- 172.21.15.27:0/49733663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1338071200 msgr2=0x7f13380cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133cb2b640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1338071200 0x7f13380cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f133cb2b640 1 -- 172.21.15.27:0/49733663 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13381e7410 con 0x7f13380ceb10 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.035+0000 7f13377fe640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13380c8dd0 0x7f13381f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.036+0000 7f133cb2b640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13380ceb10 0x7f13381f1570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f133000caa0 tx=0x7f133000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.036+0000 7f1334ff9640 1 -- 172.21.15.27:0/49733663 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330017070 con 0x7f13380ceb10 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.036+0000 7f1334ff9640 1 -- 172.21.15.27:0/49733663 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133000ac00 con 0x7f13380ceb10 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.036+0000 7f1334ff9640 1 -- 172.21.15.27:0/49733663 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330012400 con 0x7f13380ceb10 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.036+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13381e76f0 con 0x7f13380ceb10 2024-04-20T10:57:23.037 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.036+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13380b41f0 con 0x7f13380ceb10 2024-04-20T10:57:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.037+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133804fcf0 con 0x7f13380ceb10 2024-04-20T10:57:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.038+0000 7f1334ff9640 1 -- 172.21.15.27:0/49733663 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1330007070 con 0x7f13380ceb10 2024-04-20T10:57:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.038+0000 7f1334ff9640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f132406c920 0x7f132406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.038+0000 7f1334ff9640 1 -- 172.21.15.27:0/49733663 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f133008b7d0 con 0x7f13380ceb10 2024-04-20T10:57:23.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.039+0000 7f1337fff640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f132406c920 0x7f132406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:23.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.039+0000 7f1337fff640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f132406c920 0x7f132406ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1328009560 tx=0x7f132803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:23.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.043+0000 7f1334ff9640 1 -- 172.21.15.27:0/49733663 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1338155b20 con 0x7f13380ceb10 2024-04-20T10:57:23.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.276+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13380c96e0 con 0x7f13380ceb10 2024-04-20T10:57:23.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.277+0000 7f1334ff9640 1 -- 172.21.15.27:0/49733663 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7f1330059530 con 0x7f13380ceb10 2024-04-20T10:57:23.276 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:23.277 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:57:23.277 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f132406c920 msgr2=0x7f132406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f132406c920 0x7f132406ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1328009560 tx=0x7f132803d040 comp rx=0 tx=0).stop 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13380ceb10 msgr2=0x7f13381f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13380ceb10 0x7f13381f1570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f133000caa0 tx=0x7f133000cf60 comp rx=0 tx=0).stop 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 shutdown_connections 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f132406c920 0x7f132406ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13380ceb10 0x7f13381f1570 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f13380c8dd0 0x7f13381f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 --2- 172.21.15.27:0/49733663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1338071200 0x7f13380cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 >> 172.21.15.27:0/49733663 conn(0x7f13380676c0 msgr2=0x7f13380cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 shutdown_connections 2024-04-20T10:57:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:23.280+0000 7f133e58e640 1 -- 172.21.15.27:0/49733663 wait complete. 2024-04-20T10:57:23.295 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-20T10:57:25.295 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.412+0000 7f92b5c55640 1 Processor -- start 2024-04-20T10:57:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.412+0000 7f92b5c55640 1 -- start start 2024-04-20T10:57:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.413+0000 7f92b5c55640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f92b00b54f0 0x7f92b00b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.413+0000 7f92b5c55640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f92b00d0c90 0x7f92b00d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.413+0000 7f92b5c55640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b00b5e70 con 0x7f92b0155950 2024-04-20T10:57:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.413+0000 7f92b5c55640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f92b015b9c0 con 0x7f92b00b54f0 2024-04-20T10:57:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.413+0000 7f92b5c55640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f92b015bca0 con 0x7f92b00d0c90 2024-04-20T10:57:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.413+0000 7f92af7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f92b0155950 0x7f92b00cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39978/0 (socket says 172.21.15.27:39978) 2024-04-20T10:57:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.413+0000 7f92af7fe640 1 -- 172.21.15.27:0/1898686982 learned_addr learned my addr 172.21.15.27:0/1898686982 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:25.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.413+0000 7f92aeffd640 1 --2- 172.21.15.27:0/1898686982 >> v2:172.21.15.27:3300/0 conn(0x7f92b00b54f0 0x7f92b00b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:25.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.414+0000 7f92affff640 1 --2- 172.21.15.27:0/1898686982 >> v2:172.21.15.137:3300/0 conn(0x7f92b00d0c90 0x7f92b00d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:25.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.414+0000 7f92ae7fc640 1 -- 172.21.15.27:0/1898686982 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325322439 0 0) 0x7f92b00b5e70 con 0x7f92b0155950 2024-04-20T10:57:25.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.414+0000 7f92aeffd640 1 -- 172.21.15.27:0/1898686982 >> v2:172.21.15.137:3300/0 conn(0x7f92b00d0c90 msgr2=0x7f92b00d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:25.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.414+0000 7f92aeffd640 1 --2- 172.21.15.27:0/1898686982 >> v2:172.21.15.137:3300/0 conn(0x7f92b00d0c90 0x7f92b00d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:25.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.414+0000 7f92aeffd640 1 -- 172.21.15.27:0/1898686982 >> v1:172.21.15.27:6789/0 conn(0x7f92b0155950 legacy=0x7f92b00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:25.414 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.414+0000 7f92aeffd640 1 -- 172.21.15.27:0/1898686982 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b015bf80 con 0x7f92b00b54f0 2024-04-20T10:57:25.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.415+0000 7f92aeffd640 1 --2- 172.21.15.27:0/1898686982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 0x7f92b00b5930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f92b00cb2e0 tx=0x7f92a402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41bcc0c2976db96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:25.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.415+0000 7f92ae7fc640 1 -- 172.21.15.27:0/1898686982 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4034070 con 0x7f92b00b54f0 2024-04-20T10:57:25.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.415+0000 7f92ae7fc640 1 -- 172.21.15.27:0/1898686982 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a4030840 con 0x7f92b00b54f0 2024-04-20T10:57:25.415 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.415+0000 7f92ae7fc640 1 -- 172.21.15.27:0/1898686982 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a4030c10 con 0x7f92b00b54f0 2024-04-20T10:57:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.416+0000 7f92b5c55640 1 -- 172.21.15.27:0/1898686982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 msgr2=0x7f92b00b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.416+0000 7f92b5c55640 1 --2- 172.21.15.27:0/1898686982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 0x7f92b00b5930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f92b00cb2e0 tx=0x7f92a402b980 comp rx=0 tx=0).stop 2024-04-20T10:57:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.416+0000 7f92b5c55640 1 -- 172.21.15.27:0/1898686982 shutdown_connections 2024-04-20T10:57:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.416+0000 7f92b5c55640 1 --2- 172.21.15.27:0/1898686982 >> v2:172.21.15.137:3300/0 conn(0x7f92b00d0c90 0x7f92b00d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.416+0000 7f92b5c55640 1 --2- 172.21.15.27:0/1898686982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 0x7f92b00b5930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.416+0000 7f92b5c55640 1 -- 172.21.15.27:0/1898686982 >> 172.21.15.27:0/1898686982 conn(0x7f92b00676c0 msgr2=0x7f92b0067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.416+0000 7f92b5c55640 1 -- 172.21.15.27:0/1898686982 shutdown_connections 2024-04-20T10:57:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.416+0000 7f92b5c55640 1 -- 172.21.15.27:0/1898686982 wait complete. 2024-04-20T10:57:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.418+0000 7f92b5c55640 1 Processor -- start 2024-04-20T10:57:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.418+0000 7f92b5c55640 1 -- start start 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92b5c55640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 0x7f92b00b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92b5c55640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f92b00d0c90 0x7f92b01f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92b5c55640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b0155950 0x7f92b01f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92b5c55640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92b00d15e0 con 0x7f92b00b54f0 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92b5c55640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f92b015c540 con 0x7f92b0155950 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92b5c55640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f92b015fb90 con 0x7f92b00d0c90 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 0x7f92b00b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 0x7f92b00b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46354/0 (socket says 172.21.15.27:46354) 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92af7fe640 1 -- 172.21.15.27:0/1057395183 learned_addr learned my addr 172.21.15.27:0/1057395183 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92affff640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b0155950 0x7f92b01f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92aeffd640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f92b00d0c90 0x7f92b01f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92af7fe640 1 -- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f92b00d0c90 msgr2=0x7f92b01f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:25.421 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92af7fe640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f92b00d0c90 0x7f92b01f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92af7fe640 1 -- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b0155950 msgr2=0x7f92b01f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92af7fe640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b0155950 0x7f92b01f54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92af7fe640 1 -- 172.21.15.27:0/1057395183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b015b080 con 0x7f92b00b54f0 2024-04-20T10:57:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.420+0000 7f92af7fe640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 0x7f92b00b6300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f92a000c4d0 tx=0x7f92a000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.421+0000 7f9297fff640 1 -- 172.21.15.27:0/1057395183 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0018070 con 0x7f92b00b54f0 2024-04-20T10:57:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.421+0000 7f9297fff640 1 -- 172.21.15.27:0/1057395183 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a0007500 con 0x7f92b00b54f0 2024-04-20T10:57:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.421+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b015b360 con 0x7f92b00b54f0 2024-04-20T10:57:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.421+0000 7f9297fff640 1 -- 172.21.15.27:0/1057395183 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0004a20 con 0x7f92b00b54f0 2024-04-20T10:57:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.421+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b015cea0 con 0x7f92b00b54f0 2024-04-20T10:57:25.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.422+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b00cc690 con 0x7f92b00b54f0 2024-04-20T10:57:25.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.423+0000 7f9297fff640 1 -- 172.21.15.27:0/1057395183 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92a0004bc0 con 0x7f92b00b54f0 2024-04-20T10:57:25.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.423+0000 7f9297fff640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f929c06c8f0 0x7f929c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:25.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.423+0000 7f9297fff640 1 -- 172.21.15.27:0/1057395183 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f92a008ae80 con 0x7f92b00b54f0 2024-04-20T10:57:25.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.424+0000 7f92aeffd640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f929c06c8f0 0x7f929c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:25.424 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.424+0000 7f92aeffd640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f929c06c8f0 0x7f929c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f92a4036000 tx=0x7f92a403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:25.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.432+0000 7f9297fff640 1 -- 172.21.15.27:0/1057395183 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f92b00cc690 con 0x7f92b00b54f0 2024-04-20T10:57:25.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.656+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92b0155d30 con 0x7f92b00b54f0 2024-04-20T10:57:25.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.657+0000 7f9297fff640 1 -- 172.21.15.27:0/1057395183 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6931 (secure 0 0 0) 0x7f92b0155d30 con 0x7f92b00b54f0 2024-04-20T10:57:25.657 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:25.657 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:56:11.871426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4262":{"gid":4262,"name":"b","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.137:6838/1990627502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1990627502},{"type":"v1","addr":"172.21.15.137:6838","nonce":1990627502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4262,4275,4272,4278,4293]},"id":1}]} 2024-04-20T10:57:25.657 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f929c06c8f0 msgr2=0x7f929c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f929c06c8f0 0x7f929c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f92a4036000 tx=0x7f92a403b040 comp rx=0 tx=0).stop 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 msgr2=0x7f92b00b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 0x7f92b00b6300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f92a000c4d0 tx=0x7f92a000c990 comp rx=0 tx=0).stop 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 shutdown_connections 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f929c06c8f0 0x7f929c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b0155950 0x7f92b01f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f92b00d0c90 0x7f92b01f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 --2- 172.21.15.27:0/1057395183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92b00b54f0 0x7f92b00b6300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:25.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 >> 172.21.15.27:0/1057395183 conn(0x7f92b00676c0 msgr2=0x7f92b014b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:25.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 shutdown_connections 2024-04-20T10:57:25.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:25.659+0000 7f92b5c55640 1 -- 172.21.15.27:0/1057395183 wait complete. 2024-04-20T10:57:25.670 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-04-20T10:57:27.671 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:27.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.819+0000 7f1b8f6f2640 1 Processor -- start 2024-04-20T10:57:27.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.820+0000 7f1b8f6f2640 1 -- start start 2024-04-20T10:57:27.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.820+0000 7f1b8f6f2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b880c4390 0x7f1b880c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:27.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b8f6f2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b8815b7b0 con 0x7f1b880c4770 2024-04-20T10:57:27.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b8f6f2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b8815ba90 con 0x7f1b880cac70 2024-04-20T10:57:27.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b8f6f2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1b8815b580 con 0x7f1b880c4390 2024-04-20T10:57:27.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b8d48e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b880c4390 0x7f1b880c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:27.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b8d48e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b880c4390 0x7f1b880c2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46370/0 (socket says 172.21.15.27:46370) 2024-04-20T10:57:27.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b8d48e640 1 -- 172.21.15.27:0/1359896103 learned_addr learned my addr 172.21.15.27:0/1359896103 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:27.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b8d48e640 1 -- 172.21.15.27:0/1359896103 >> v1:172.21.15.137:6789/0 conn(0x7f1b880cac70 legacy=0x7f1b881551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:27.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b8d48e640 1 -- 172.21.15.27:0/1359896103 >> v1:172.21.15.27:6789/0 conn(0x7f1b880c4770 legacy=0x7f1b8815ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:27.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b7ffff640 1 -- 172.21.15.27:0/1359896103 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 715590744 0 0) 0x7f1b8815b7b0 con 0x7f1b880c4770 2024-04-20T10:57:27.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.821+0000 7f1b8d48e640 1 -- 172.21.15.27:0/1359896103 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8815bd70 con 0x7f1b880c4390 2024-04-20T10:57:27.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.822+0000 7f1b8d48e640 1 --2- 172.21.15.27:0/1359896103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4390 0x7f1b880c2ea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1b840098d0 tx=0x7f1b84034260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f38a70a83db4bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:27.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.822+0000 7f1b7ffff640 1 -- 172.21.15.27:0/1359896103 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84041070 con 0x7f1b880c4390 2024-04-20T10:57:27.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.822+0000 7f1b7ffff640 1 -- 172.21.15.27:0/1359896103 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8403c640 con 0x7f1b880c4390 2024-04-20T10:57:27.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.822+0000 7f1b7ffff640 1 -- 172.21.15.27:0/1359896103 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8403ca10 con 0x7f1b880c4390 2024-04-20T10:57:27.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.823+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1359896103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4390 msgr2=0x7f1b880c2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:27.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.823+0000 7f1b8f6f2640 1 --2- 172.21.15.27:0/1359896103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4390 0x7f1b880c2ea0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1b840098d0 tx=0x7f1b84034260 comp rx=0 tx=0).stop 2024-04-20T10:57:27.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.823+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1359896103 shutdown_connections 2024-04-20T10:57:27.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.823+0000 7f1b8f6f2640 1 --2- 172.21.15.27:0/1359896103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4390 0x7f1b880c2ea0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:27.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.823+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1359896103 >> 172.21.15.27:0/1359896103 conn(0x7f1b8806fb50 msgr2=0x7f1b8806ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:27.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.823+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1359896103 shutdown_connections 2024-04-20T10:57:27.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.823+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1359896103 wait complete. 2024-04-20T10:57:27.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.824+0000 7f1b8f6f2640 1 Processor -- start 2024-04-20T10:57:27.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.824+0000 7f1b8f6f2640 1 -- start start 2024-04-20T10:57:27.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.825+0000 7f1b8f6f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b880c4390 0x7f1b880cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:27.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.825+0000 7f1b8f6f2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4770 0x7f1b880ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:27.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.825+0000 7f1b8f6f2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b880cac70 0x7f1b881f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:27.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.825+0000 7f1b8f6f2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8815f6a0 con 0x7f1b880c4770 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.825+0000 7f1b8f6f2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8815c330 con 0x7f1b880c4390 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.825+0000 7f1b8f6f2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8815f820 con 0x7f1b880cac70 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.825+0000 7f1b8cc8d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4770 0x7f1b880ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.825+0000 7f1b8cc8d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4770 0x7f1b880ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46374/0 (socket says 172.21.15.27:46374) 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8cc8d640 1 -- 172.21.15.27:0/1605086294 learned_addr learned my addr 172.21.15.27:0/1605086294 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8cc8d640 1 -- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b880cac70 msgr2=0x7f1b881f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8cc8d640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b880cac70 0x7f1b881f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8cc8d640 1 -- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b880c4390 msgr2=0x7f1b880cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8d48e640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b880c4390 0x7f1b880cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8cc8d640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b880c4390 0x7f1b880cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8cc8d640 1 -- 172.21.15.27:0/1605086294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b88151580 con 0x7f1b880c4770 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8d48e640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b880c4390 0x7f1b880cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8cc8d640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4770 0x7f1b880ccb80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c550 tx=0x7f1b7800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b7dffb640 1 -- 172.21.15.27:0/1605086294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78017070 con 0x7f1b880c4770 2024-04-20T10:57:27.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b7dffb640 1 -- 172.21.15.27:0/1605086294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b780035f0 con 0x7f1b880c4770 2024-04-20T10:57:27.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b88159f90 con 0x7f1b880c4770 2024-04-20T10:57:27.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.826+0000 7f1b7dffb640 1 -- 172.21.15.27:0/1605086294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b780039e0 con 0x7f1b880c4770 2024-04-20T10:57:27.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.827+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b88151ae0 con 0x7f1b880c4770 2024-04-20T10:57:27.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.828+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8804fcf0 con 0x7f1b880c4770 2024-04-20T10:57:27.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.828+0000 7f1b7dffb640 1 -- 172.21.15.27:0/1605086294 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b7801a070 con 0x7f1b880c4770 2024-04-20T10:57:27.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.829+0000 7f1b7dffb640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1b6c06c8f0 0x7f1b6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:27.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.829+0000 7f1b7dffb640 1 -- 172.21.15.27:0/1605086294 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f1b7808b0f0 con 0x7f1b880c4770 2024-04-20T10:57:27.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.829+0000 7f1b8d48e640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1b6c06c8f0 0x7f1b6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:27.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.830+0000 7f1b8d48e640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1b6c06c8f0 0x7f1b6c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b8403e000 tx=0x7f1b84038000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:27.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:27.833+0000 7f1b7dffb640 1 -- 172.21.15.27:0/1605086294 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1b88151580 con 0x7f1b880c4770 2024-04-20T10:57:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.063+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b880c93d0 con 0x7f1b880c4770 2024-04-20T10:57:28.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.064+0000 7f1b7dffb640 1 -- 172.21.15.27:0/1605086294 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6100 (secure 0 0 0) 0x7f1b780593c0 con 0x7f1b880c4770 2024-04-20T10:57:28.064 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:28.064 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:27.321952+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:57:28.064 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2024-04-20T10:57:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1b6c06c8f0 msgr2=0x7f1b6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1b6c06c8f0 0x7f1b6c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1b8403e000 tx=0x7f1b84038000 comp rx=0 tx=0).stop 2024-04-20T10:57:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4770 msgr2=0x7f1b880ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4770 0x7f1b880ccb80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c550 tx=0x7f1b7800ca10 comp rx=0 tx=0).stop 2024-04-20T10:57:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 shutdown_connections 2024-04-20T10:57:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1b6c06c8f0 0x7f1b6c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b880cac70 0x7f1b881f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b880c4770 0x7f1b880ccb80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 --2- 172.21.15.27:0/1605086294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b880c4390 0x7f1b880cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 >> 172.21.15.27:0/1605086294 conn(0x7f1b8806fb50 msgr2=0x7f1b88153800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 shutdown_connections 2024-04-20T10:57:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.066+0000 7f1b8f6f2640 1 -- 172.21.15.27:0/1605086294 wait complete. 2024-04-20T10:57:28.077 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2024-04-20T10:57:28.078 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T10:57:28.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.185+0000 7f726863a640 1 Processor -- start 2024-04-20T10:57:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.186+0000 7f726863a640 1 -- start start 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.186+0000 7f726863a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7260155950 0x7f72600cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.186+0000 7f726863a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f72600b5c00 0x7f72600d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f726863a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f726015b9c0 con 0x7f72600cc330 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f726863a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f726015bca0 con 0x7f7260155950 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f726863a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f726015bf80 con 0x7f72600b5c00 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f72663d6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7260155950 0x7f72600cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f72663d6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7260155950 0x7f72600cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46386/0 (socket says 172.21.15.27:46386) 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f72663d6640 1 -- 172.21.15.27:0/2794514627 learned_addr learned my addr 172.21.15.27:0/2794514627 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f7266bd7640 1 --2- 172.21.15.27:0/2794514627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f72600b5c00 0x7f72600d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f72663d6640 1 -- 172.21.15.27:0/2794514627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f72600b5c00 msgr2=0x7f72600d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f72663d6640 1 --2- 172.21.15.27:0/2794514627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f72600b5c00 0x7f72600d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f72663d6640 1 -- 172.21.15.27:0/2794514627 >> v1:172.21.15.137:6789/0 conn(0x7f72600cc330 legacy=0x7f72600b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f72653d4640 1 -- 172.21.15.27:0/2794514627 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 370290350 0 0) 0x7f726015b9c0 con 0x7f72600cc330 2024-04-20T10:57:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.187+0000 7f72663d6640 1 -- 172.21.15.27:0/2794514627 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726015c260 con 0x7f7260155950 2024-04-20T10:57:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.188+0000 7f72663d6640 1 --2- 172.21.15.27:0/2794514627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7260155950 0x7f72600cbdf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f725c009870 tx=0x7f725c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec0f9b7e394280b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.189+0000 7f72653d4640 1 -- 172.21.15.27:0/2794514627 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c02f840 con 0x7f7260155950 2024-04-20T10:57:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.189+0000 7f72653d4640 1 -- 172.21.15.27:0/2794514627 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725c02f9e0 con 0x7f7260155950 2024-04-20T10:57:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.189+0000 7f72653d4640 1 -- 172.21.15.27:0/2794514627 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c02f840 con 0x7f7260155950 2024-04-20T10:57:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.189+0000 7f726863a640 1 -- 172.21.15.27:0/2794514627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7260155950 msgr2=0x7f72600cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.189+0000 7f726863a640 1 --2- 172.21.15.27:0/2794514627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7260155950 0x7f72600cbdf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f725c009870 tx=0x7f725c02ba20 comp rx=0 tx=0).stop 2024-04-20T10:57:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.189+0000 7f726863a640 1 -- 172.21.15.27:0/2794514627 shutdown_connections 2024-04-20T10:57:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.189+0000 7f726863a640 1 --2- 172.21.15.27:0/2794514627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f72600b5c00 0x7f72600d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.189+0000 7f726863a640 1 --2- 172.21.15.27:0/2794514627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7260155950 0x7f72600cbdf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.189+0000 7f726863a640 1 -- 172.21.15.27:0/2794514627 >> 172.21.15.27:0/2794514627 conn(0x7f72600676c0 msgr2=0x7f7260067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.190+0000 7f726863a640 1 -- 172.21.15.27:0/2794514627 shutdown_connections 2024-04-20T10:57:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.190+0000 7f726863a640 1 -- 172.21.15.27:0/2794514627 wait complete. 2024-04-20T10:57:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.191+0000 7f726863a640 1 Processor -- start 2024-04-20T10:57:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.191+0000 7f726863a640 1 -- start start 2024-04-20T10:57:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.191+0000 7f726863a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72600b5c00 0x7f72600c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:28.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.191+0000 7f726863a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72600cc330 0x7f726015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f726863a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7260155950 0x7f726015d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f726863a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f726015fd10 con 0x7f72600cc330 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f726863a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f726015fb90 con 0x7f72600b5c00 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f726863a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7260155d30 con 0x7f7260155950 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7265bd5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72600cc330 0x7f726015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7265bd5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72600cc330 0x7f726015ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46390/0 (socket says 172.21.15.27:46390) 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7265bd5640 1 -- 172.21.15.27:0/1721013926 learned_addr learned my addr 172.21.15.27:0/1721013926 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f72663d6640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72600b5c00 0x7f72600c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7266bd7640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7260155950 0x7f726015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7265bd5640 1 -- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7260155950 msgr2=0x7f726015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7265bd5640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7260155950 0x7f726015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7265bd5640 1 -- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72600b5c00 msgr2=0x7f72600c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7265bd5640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72600b5c00 0x7f72600c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7265bd5640 1 -- 172.21.15.27:0/1721013926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72601f17e0 con 0x7f72600cc330 2024-04-20T10:57:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7266bd7640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7260155950 0x7f726015d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:57:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.192+0000 7f7265bd5640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72600cc330 0x7f726015ce20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f725000c410 tx=0x7f725000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.193+0000 7f7256ffd640 1 -- 172.21.15.27:0/1721013926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250017070 con 0x7f72600cc330 2024-04-20T10:57:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.193+0000 7f7256ffd640 1 -- 172.21.15.27:0/1721013926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72500035f0 con 0x7f72600cc330 2024-04-20T10:57:28.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.193+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72601f1ac0 con 0x7f72600cc330 2024-04-20T10:57:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.194+0000 7f7256ffd640 1 -- 172.21.15.27:0/1721013926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72500039e0 con 0x7f72600cc330 2024-04-20T10:57:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.194+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72601f2080 con 0x7f72600cc330 2024-04-20T10:57:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.194+0000 7f7256ffd640 1 -- 172.21.15.27:0/1721013926 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f725001a070 con 0x7f72600cc330 2024-04-20T10:57:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.195+0000 7f7256ffd640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f724406c8f0 0x7f724406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.195+0000 7f7256ffd640 1 -- 172.21.15.27:0/1721013926 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f725008f170 con 0x7f72600cc330 2024-04-20T10:57:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.195+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f726004fcf0 con 0x7f72600cc330 2024-04-20T10:57:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.195+0000 7f72663d6640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f724406c8f0 0x7f724406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.196+0000 7f72663d6640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f724406c8f0 0x7f724406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f725c036000 tx=0x7f725c008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:28.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.204+0000 7f7256ffd640 1 -- 172.21.15.27:0/1721013926 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7250059210 con 0x7f72600cc330 2024-04-20T10:57:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.664+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72600b6b20 con 0x7f72600cc330 2024-04-20T10:57:28.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.665+0000 7f7256ffd640 1 -- 172.21.15.27:0/1721013926 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6100 (secure 0 0 0) 0x7f7250058b10 con 0x7f72600cc330 2024-04-20T10:57:28.664 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:28.665 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:27.321952+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:57:28.665 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2024-04-20T10:57:28.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.667+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f724406c8f0 msgr2=0x7f724406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.667+0000 7f726863a640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f724406c8f0 0x7f724406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f725c036000 tx=0x7f725c008030 comp rx=0 tx=0).stop 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.667+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72600cc330 msgr2=0x7f726015ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.667+0000 7f726863a640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72600cc330 0x7f726015ce20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f725000c410 tx=0x7f725000c8d0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.668+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 shutdown_connections 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.668+0000 7f726863a640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f724406c8f0 0x7f724406edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.668+0000 7f726863a640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7260155950 0x7f726015d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.668+0000 7f726863a640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72600cc330 0x7f726015ce20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.668+0000 7f726863a640 1 --2- 172.21.15.27:0/1721013926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72600b5c00 0x7f72600c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.668+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 >> 172.21.15.27:0/1721013926 conn(0x7f72600676c0 msgr2=0x7f7260154960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.668+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 shutdown_connections 2024-04-20T10:57:28.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:28.668+0000 7f726863a640 1 -- 172.21.15.27:0/1721013926 wait complete. 2024-04-20T10:57:28.683 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 18, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 18, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4259": { "addr": "172.21.15.137:6835/1164978945", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6834", "nonce": 1164978945, "type": "v2" }, { "addr": "172.21.15.137:6835", "nonce": 1164978945, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4259, "incarnation": 18, "join_fscid": -1, "name": "f", "rank": 4, "state": "up:replay", "state_seq": 1 }, "gid_4272": { "addr": "172.21.15.27:6835/859496751", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6832", "nonce": 859496751, "type": "v2" }, { "addr": "172.21.15.27:6835", "nonce": 859496751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4272, "incarnation": 14, "join_fscid": -1, "name": "c", "rank": 3, "state": "up:active", "state_seq": 9 }, "gid_4275": { "addr": "172.21.15.27:6837/1680990583", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6834", "nonce": 1680990583, "type": "v2" }, { "addr": "172.21.15.27:6837", "nonce": 1680990583, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4275, "incarnation": 12, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4278": { "addr": "172.21.15.27:6836/4195543922", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6833", "nonce": 4195543922, "type": "v2" }, { "addr": "172.21.15.27:6836", "nonce": 4195543922, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4278, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 23, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T10:57:27.321952+0000", "qdb_cluster": [ 4272, 4275, 4278, 4293 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 4278, "mds_2": 4275, "mds_3": 4272, "mds_4": 4259 } } } ], "standbys": [] } 2024-04-20T10:57:30.684 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:30.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.840+0000 7f9bcc7a4640 1 Processor -- start 2024-04-20T10:57:30.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.840+0000 7f9bcc7a4640 1 -- start start 2024-04-20T10:57:30.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.841+0000 7f9bcc7a4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9bc4152070 0x7f9bc4158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:30.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.841+0000 7f9bcc7a4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc406dad0 0x7f9bc4162a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:30.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.841+0000 7f9bcc7a4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc4072280 con 0x7f9bc406d660 2024-04-20T10:57:30.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.841+0000 7f9bcc7a4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc4072b20 con 0x7f9bc4152070 2024-04-20T10:57:30.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.841+0000 7f9bcc7a4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc4072560 con 0x7f9bc406dad0 2024-04-20T10:57:30.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.842+0000 7f9bca540640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9bc4152070 0x7f9bc4158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:30.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.842+0000 7f9bca540640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9bc4152070 0x7f9bc4158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:49028/0 (socket says 172.21.15.27:49028) 2024-04-20T10:57:30.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.842+0000 7f9bca540640 1 -- 172.21.15.27:0/1201279921 learned_addr learned my addr 172.21.15.27:0/1201279921 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:30.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.842+0000 7f9bcad41640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc406dad0 0x7f9bc4162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:30.842 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.842+0000 7f9bca540640 1 -- 172.21.15.27:0/1201279921 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc406dad0 msgr2=0x7f9bc4162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:30.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.842+0000 7f9bca540640 1 --2- 172.21.15.27:0/1201279921 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc406dad0 0x7f9bc4162a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:30.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.842+0000 7f9bca540640 1 -- 172.21.15.27:0/1201279921 >> v1:172.21.15.137:6789/0 conn(0x7f9bc406d660 legacy=0x7f9bc406ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:30.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.842+0000 7f9bca540640 1 -- 172.21.15.27:0/1201279921 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc4072840 con 0x7f9bc4152070 2024-04-20T10:57:30.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.843+0000 7f9bca540640 1 --2- 172.21.15.27:0/1201279921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc4152070 0x7f9bc4158a40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000bb10 tx=0x7f9bc002e1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aaf07274742ab89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:30.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.843+0000 7f9bc953e640 1 -- 172.21.15.27:0/1201279921 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc003b070 con 0x7f9bc4152070 2024-04-20T10:57:30.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.843+0000 7f9bc953e640 1 -- 172.21.15.27:0/1201279921 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc0036440 con 0x7f9bc4152070 2024-04-20T10:57:30.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.843+0000 7f9bc953e640 1 -- 172.21.15.27:0/1201279921 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0036810 con 0x7f9bc4152070 2024-04-20T10:57:30.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.844+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/1201279921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc4152070 msgr2=0x7f9bc4158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:30.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.844+0000 7f9bcc7a4640 1 --2- 172.21.15.27:0/1201279921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc4152070 0x7f9bc4158a40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000bb10 tx=0x7f9bc002e1f0 comp rx=0 tx=0).stop 2024-04-20T10:57:30.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.845+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/1201279921 shutdown_connections 2024-04-20T10:57:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.845+0000 7f9bcc7a4640 1 --2- 172.21.15.27:0/1201279921 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc406dad0 0x7f9bc4162a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.845+0000 7f9bcc7a4640 1 --2- 172.21.15.27:0/1201279921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc4152070 0x7f9bc4158a40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.845+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/1201279921 >> 172.21.15.27:0/1201279921 conn(0x7f9bc406f950 msgr2=0x7f9bc406fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.845+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/1201279921 shutdown_connections 2024-04-20T10:57:30.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.845+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/1201279921 wait complete. 2024-04-20T10:57:30.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.846+0000 7f9bcc7a4640 1 Processor -- start 2024-04-20T10:57:30.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.846+0000 7f9bcc7a4640 1 -- start start 2024-04-20T10:57:30.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bcc7a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc406d660 0x7f9bc41ea770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bcc7a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc406dad0 0x7f9bc41eff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bcc7a4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc4152070 0x7f9bc41f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bcc7a4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4071f60 con 0x7f9bc406dad0 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bcc7a4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4158520 con 0x7f9bc406d660 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bcc7a4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc4158f80 con 0x7f9bc4152070 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bca540640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc406d660 0x7f9bc41ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bca540640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc406d660 0x7f9bc41ea770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:49036/0 (socket says 172.21.15.27:49036) 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bca540640 1 -- 172.21.15.27:0/376141535 learned_addr learned my addr 172.21.15.27:0/376141535 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bc9d3f640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc406dad0 0x7f9bc41eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.848+0000 7f9bcad41640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc4152070 0x7f9bc41f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bca540640 1 -- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc4152070 msgr2=0x7f9bc41f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bca540640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc4152070 0x7f9bc41f04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bca540640 1 -- 172.21.15.27:0/376141535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc406dad0 msgr2=0x7f9bc41eff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bca540640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc406dad0 0x7f9bc41eff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bca540640 1 -- 172.21.15.27:0/376141535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc41eacb0 con 0x7f9bc406d660 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bc9d3f640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc406dad0 0x7f9bc41eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bca540640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc406d660 0x7f9bc41ea770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000b2c0 tx=0x7f9bc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bbaffd640 1 -- 172.21.15.27:0/376141535 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc003b040 con 0x7f9bc406d660 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bbaffd640 1 -- 172.21.15.27:0/376141535 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc0046070 con 0x7f9bc406d660 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bbaffd640 1 -- 172.21.15.27:0/376141535 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0034bd0 con 0x7f9bc406d660 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.849+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc4154260 con 0x7f9bc406d660 2024-04-20T10:57:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.850+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc4154530 con 0x7f9bc406d660 2024-04-20T10:57:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.851+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc404fcf0 con 0x7f9bc406d660 2024-04-20T10:57:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.852+0000 7f9bbaffd640 1 -- 172.21.15.27:0/376141535 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bc004b460 con 0x7f9bc406d660 2024-04-20T10:57:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.852+0000 7f9bbaffd640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9ba806c920 0x7f9ba806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.853+0000 7f9bc9d3f640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9ba806c920 0x7f9ba806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.853+0000 7f9bbaffd640 1 -- 172.21.15.27:0/376141535 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f9bc00b09a0 con 0x7f9bc406d660 2024-04-20T10:57:30.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.853+0000 7f9bc9d3f640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9ba806c920 0x7f9ba806ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9bc406f380 tx=0x7f9bb4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:30.857+0000 7f9bbaffd640 1 -- 172.21.15.27:0/376141535 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9bc007e620 con 0x7f9bc406d660 2024-04-20T10:57:31.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.315+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bc4158cd0 con 0x7f9bc406d660 2024-04-20T10:57:31.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.316+0000 7f9bbaffd640 1 -- 172.21.15.27:0/376141535 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+6100 (secure 0 0 0) 0x7f9bc0082920 con 0x7f9bc406d660 2024-04-20T10:57:31.316 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:31.316 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:27.321952+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:57:31.317 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 18 2024-04-20T10:57:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9ba806c920 msgr2=0x7f9ba806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9ba806c920 0x7f9ba806ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9bc406f380 tx=0x7f9bb4009040 comp rx=0 tx=0).stop 2024-04-20T10:57:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc406d660 msgr2=0x7f9bc41ea770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc406d660 0x7f9bc41ea770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000b2c0 tx=0x7f9bc00046c0 comp rx=0 tx=0).stop 2024-04-20T10:57:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 shutdown_connections 2024-04-20T10:57:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9ba806c920 0x7f9ba806ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9bc4152070 0x7f9bc41f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:31.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc406dad0 0x7f9bc41eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:31.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 --2- 172.21.15.27:0/376141535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9bc406d660 0x7f9bc41ea770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:31.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 >> 172.21.15.27:0/376141535 conn(0x7f9bc406f950 msgr2=0x7f9bc40c5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:31.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 shutdown_connections 2024-04-20T10:57:31.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:31.319+0000 7f9bcc7a4640 1 -- 172.21.15.27:0/376141535 wait complete. 2024-04-20T10:57:31.335 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:57:33.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.441+0000 7f49d4f87640 1 Processor -- start 2024-04-20T10:57:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.441+0000 7f49d4f87640 1 -- start start 2024-04-20T10:57:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49d4f87640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d0153730 0x7f49d00c8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49d4f87640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f49d00b48d0 0x7f49d01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49d4f87640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f49d01598f0 0x7f49d00ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49d4f87640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f49d0153b10 con 0x7f49d01598f0 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49d4f87640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f49d015d900 con 0x7f49d00b48d0 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49d4f87640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f49d00c8e50 con 0x7f49d0153730 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d0153730 0x7f49d00c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49ced9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f49d01598f0 0x7f49d00ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d0153730 0x7f49d00c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:53290/0 (socket says 172.21.15.27:53290) 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49ce59c640 1 -- 172.21.15.27:0/3093628770 learned_addr learned my addr 172.21.15.27:0/3093628770 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49cdd9b640 1 --2- 172.21.15.27:0/3093628770 >> v2:172.21.15.137:3300/0 conn(0x7f49d00b48d0 0x7f49d01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49ce59c640 1 -- 172.21.15.27:0/3093628770 >> v2:172.21.15.137:3300/0 conn(0x7f49d00b48d0 msgr2=0x7f49d01593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49ce59c640 1 --2- 172.21.15.27:0/3093628770 >> v2:172.21.15.137:3300/0 conn(0x7f49d00b48d0 0x7f49d01593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49ce59c640 1 -- 172.21.15.27:0/3093628770 >> v2:172.21.15.27:3300/0 conn(0x7f49d01598f0 msgr2=0x7f49d00ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.442+0000 7f49ce59c640 1 --2- 172.21.15.27:0/3093628770 >> v2:172.21.15.27:3300/0 conn(0x7f49d01598f0 0x7f49d00ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.443+0000 7f49ce59c640 1 -- 172.21.15.27:0/3093628770 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d015cb40 con 0x7f49d0153730 2024-04-20T10:57:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.443+0000 7f49ced9d640 1 --2- 172.21.15.27:0/3093628770 >> v2:172.21.15.27:3300/0 conn(0x7f49d01598f0 0x7f49d00ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.444+0000 7f49ce59c640 1 --2- 172.21.15.27:0/3093628770 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d0153730 0x7f49d00c8530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f49d0068680 tx=0x7f49c002b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3600105fa6d09c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.444+0000 7f49cd59a640 1 -- 172.21.15.27:0/3093628770 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0035020 con 0x7f49d0153730 2024-04-20T10:57:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.444+0000 7f49cd59a640 1 -- 172.21.15.27:0/3093628770 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c0031870 con 0x7f49d0153730 2024-04-20T10:57:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.444+0000 7f49cd59a640 1 -- 172.21.15.27:0/3093628770 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0031c40 con 0x7f49d0153730 2024-04-20T10:57:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.445+0000 7f49d4f87640 1 -- 172.21.15.27:0/3093628770 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d0153730 msgr2=0x7f49d00c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.445+0000 7f49d4f87640 1 --2- 172.21.15.27:0/3093628770 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d0153730 0x7f49d00c8530 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f49d0068680 tx=0x7f49c002b910 comp rx=0 tx=0).stop 2024-04-20T10:57:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.445+0000 7f49d4f87640 1 -- 172.21.15.27:0/3093628770 shutdown_connections 2024-04-20T10:57:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.445+0000 7f49d4f87640 1 --2- 172.21.15.27:0/3093628770 >> v2:172.21.15.27:3300/0 conn(0x7f49d01598f0 0x7f49d00ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.445+0000 7f49d4f87640 1 --2- 172.21.15.27:0/3093628770 >> v2:172.21.15.137:3300/0 conn(0x7f49d00b48d0 0x7f49d01593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.445+0000 7f49d4f87640 1 --2- 172.21.15.27:0/3093628770 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d0153730 0x7f49d00c8530 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.445+0000 7f49d4f87640 1 -- 172.21.15.27:0/3093628770 >> 172.21.15.27:0/3093628770 conn(0x7f49d00676c0 msgr2=0x7f49d0067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:33.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.445+0000 7f49d4f87640 1 -- 172.21.15.27:0/3093628770 shutdown_connections 2024-04-20T10:57:33.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.445+0000 7f49d4f87640 1 -- 172.21.15.27:0/3093628770 wait complete. 2024-04-20T10:57:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.446+0000 7f49d4f87640 1 Processor -- start 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.446+0000 7f49d4f87640 1 -- start start 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49d4f87640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d00b48d0 0x7f49d0150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49d4f87640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d0153730 0x7f49d015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49d4f87640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d01598f0 0x7f49d015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49d4f87640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f49d015d3e0 con 0x7f49d0153730 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49d4f87640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49d015d100 con 0x7f49d00b48d0 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49d4f87640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f49d00c9b70 con 0x7f49d01598f0 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49cdd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d0153730 0x7f49d015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d00b48d0 0x7f49d0150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d00b48d0 0x7f49d0150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:49056/0 (socket says 172.21.15.27:49056) 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49ce59c640 1 -- 172.21.15.27:0/469090086 learned_addr learned my addr 172.21.15.27:0/469090086 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.447+0000 7f49ced9d640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d01598f0 0x7f49d015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49ce59c640 1 -- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d01598f0 msgr2=0x7f49d015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49ce59c640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d01598f0 0x7f49d015bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49ce59c640 1 -- 172.21.15.27:0/469090086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d0153730 msgr2=0x7f49d015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49ce59c640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d0153730 0x7f49d015b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49ce59c640 1 -- 172.21.15.27:0/469090086 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d014fbe0 con 0x7f49d00b48d0 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49cdd9b640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d0153730 0x7f49d015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49ced9d640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d01598f0 0x7f49d015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49ce59c640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d00b48d0 0x7f49d0150c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49c0000c00 tx=0x7f49c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49beffd640 1 -- 172.21.15.27:0/469090086 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0035040 con 0x7f49d00b48d0 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49beffd640 1 -- 172.21.15.27:0/469090086 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c0030d00 con 0x7f49d00b48d0 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49beffd640 1 -- 172.21.15.27:0/469090086 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00456a0 con 0x7f49d00b48d0 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d014fec0 con 0x7f49d00b48d0 2024-04-20T10:57:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.448+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d0150480 con 0x7f49d00b48d0 2024-04-20T10:57:33.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.450+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d004fcf0 con 0x7f49d00b48d0 2024-04-20T10:57:33.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.450+0000 7f49beffd640 1 -- 172.21.15.27:0/469090086 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49c0038020 con 0x7f49d00b48d0 2024-04-20T10:57:33.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.450+0000 7f49beffd640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49b006c8f0 0x7f49b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:33.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.450+0000 7f49beffd640 1 -- 172.21.15.27:0/469090086 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f49c00adec0 con 0x7f49d00b48d0 2024-04-20T10:57:33.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.450+0000 7f49cdd9b640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49b006c8f0 0x7f49b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:33.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.451+0000 7f49cdd9b640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49b006c8f0 0x7f49b006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49d00b5100 tx=0x7f49c4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.455+0000 7f49beffd640 1 -- 172.21.15.27:0/469090086 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f49c007c180 con 0x7f49d00b48d0 2024-04-20T10:57:33.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.749+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49d014f710 con 0x7f49d00b48d0 2024-04-20T10:57:33.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.750+0000 7f49beffd640 1 -- 172.21.15.27:0/469090086 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+6102 (secure 0 0 0) 0x7f49c007ba80 con 0x7f49d00b48d0 2024-04-20T10:57:33.750 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:33.750 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:32.332342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:resolve","state_seq":31,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:57:33.750 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 19 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49b006c8f0 msgr2=0x7f49b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49b006c8f0 0x7f49b006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49d00b5100 tx=0x7f49c4009190 comp rx=0 tx=0).stop 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d00b48d0 msgr2=0x7f49d0150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d00b48d0 0x7f49d0150c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49c0000c00 tx=0x7f49c00046c0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 shutdown_connections 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f49b006c8f0 0x7f49b006edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f49d01598f0 0x7f49d015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49d0153730 0x7f49d015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 --2- 172.21.15.27:0/469090086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d00b48d0 0x7f49d0150c80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 >> 172.21.15.27:0/469090086 conn(0x7f49d00676c0 msgr2=0x7f49d00cb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:33.753 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 shutdown_connections 2024-04-20T10:57:33.754 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:33.753+0000 7f49d4f87640 1 -- 172.21.15.27:0/469090086 wait complete. 2024-04-20T10:57:33.767 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:57:35.769 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:35.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.924+0000 7f841f8a7640 1 Processor -- start 2024-04-20T10:57:35.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.925+0000 7f841f8a7640 1 -- start start 2024-04-20T10:57:35.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.926+0000 7f841f8a7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84180b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:35.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.926+0000 7f841f8a7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84180b5e70 con 0x7f84180d0c90 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.926+0000 7f841f8a7640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841815b9c0 con 0x7f8418155950 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.926+0000 7f841f8a7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f841815bca0 con 0x7f84180b54f0 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.926+0000 7f841de44640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f84180d0c90 0x7f84180d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60564/0 (socket says 172.21.15.27:60564) 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.926+0000 7f841de44640 1 -- 172.21.15.27:0/2074045279 learned_addr learned my addr 172.21.15.27:0/2074045279 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.926+0000 7f841ce42640 1 --2- 172.21.15.27:0/2074045279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84180b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.926+0000 7f840ffff640 1 -- 172.21.15.27:0/2074045279 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3461526542 0 0) 0x7f84180b5e70 con 0x7f84180d0c90 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.926+0000 7f840ffff640 1 -- 172.21.15.27:0/2074045279 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8404003520 con 0x7f84180d0c90 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.927+0000 7f840ffff640 1 -- 172.21.15.27:0/2074045279 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2778698106 0 0) 0x7f841815b9c0 con 0x7f8418155950 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.927+0000 7f841ce42640 1 -- 172.21.15.27:0/2074045279 >> v1:172.21.15.137:6789/0 conn(0x7f8418155950 legacy=0x7f84180cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.927+0000 7f841ce42640 1 -- 172.21.15.27:0/2074045279 >> v1:172.21.15.27:6789/0 conn(0x7f84180d0c90 legacy=0x7f84180d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:35.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.927+0000 7f841ce42640 1 -- 172.21.15.27:0/2074045279 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841815bf80 con 0x7f84180b54f0 2024-04-20T10:57:35.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.928+0000 7f841ce42640 1 --2- 172.21.15.27:0/2074045279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84180b5930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84080097d0 tx=0x7f840802b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d95ec86a565a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:35.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.928+0000 7f840ffff640 1 -- 172.21.15.27:0/2074045279 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408037070 con 0x7f84180b54f0 2024-04-20T10:57:35.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.928+0000 7f840ffff640 1 -- 172.21.15.27:0/2074045279 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8408030510 con 0x7f84180b54f0 2024-04-20T10:57:35.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.928+0000 7f840ffff640 1 -- 172.21.15.27:0/2074045279 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84100019f0 con 0x7f84180b54f0 2024-04-20T10:57:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.929+0000 7f841f8a7640 1 -- 172.21.15.27:0/2074045279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 msgr2=0x7f84180b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.929+0000 7f841f8a7640 1 --2- 172.21.15.27:0/2074045279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84180b5930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84080097d0 tx=0x7f840802b7b0 comp rx=0 tx=0).stop 2024-04-20T10:57:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.929+0000 7f841f8a7640 1 -- 172.21.15.27:0/2074045279 shutdown_connections 2024-04-20T10:57:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.929+0000 7f841f8a7640 1 --2- 172.21.15.27:0/2074045279 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84180b5930 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.929+0000 7f841f8a7640 1 -- 172.21.15.27:0/2074045279 >> 172.21.15.27:0/2074045279 conn(0x7f84180676c0 msgr2=0x7f8418067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.929+0000 7f841f8a7640 1 -- 172.21.15.27:0/2074045279 shutdown_connections 2024-04-20T10:57:35.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.929+0000 7f841f8a7640 1 -- 172.21.15.27:0/2074045279 wait complete. 2024-04-20T10:57:35.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.931+0000 7f841f8a7640 1 Processor -- start 2024-04-20T10:57:35.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.931+0000 7f841f8a7640 1 -- start start 2024-04-20T10:57:35.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.932+0000 7f841f8a7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84181f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:35.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.932+0000 7f841f8a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84180d0c90 0x7f84180b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:35.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841f8a7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418155950 0x7f841814e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:35.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841ce42640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84180d0c90 0x7f84180b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:35.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841f8a7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f841815fb90 con 0x7f84180d0c90 2024-04-20T10:57:35.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841f8a7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f841815c540 con 0x7f8418155950 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841f8a7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f841815fd10 con 0x7f84180b54f0 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841d643640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84181f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841de44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418155950 0x7f841814e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841d643640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84181f58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:43226/0 (socket says 172.21.15.27:43226) 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841d643640 1 -- 172.21.15.27:0/3762929621 learned_addr learned my addr 172.21.15.27:0/3762929621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841de44640 1 -- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 msgr2=0x7f84181f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841de44640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84181f58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841de44640 1 -- 172.21.15.27:0/3762929621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84180d0c90 msgr2=0x7f84180b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841de44640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84180d0c90 0x7f84180b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841de44640 1 -- 172.21.15.27:0/3762929621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841815a7d0 con 0x7f8418155950 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.933+0000 7f841de44640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418155950 0x7f841814e600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f841000caf0 tx=0x7f8410007f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.934+0000 7f840dffb640 1 -- 172.21.15.27:0/3762929621 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410018070 con 0x7f8418155950 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.934+0000 7f840dffb640 1 -- 172.21.15.27:0/3762929621 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841000a760 con 0x7f8418155950 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.934+0000 7f840dffb640 1 -- 172.21.15.27:0/3762929621 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841000ab50 con 0x7f8418155950 2024-04-20T10:57:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.934+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8418159900 con 0x7f8418155950 2024-04-20T10:57:35.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.934+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8418159ea0 con 0x7f8418155950 2024-04-20T10:57:35.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.935+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84180cf490 con 0x7f8418155950 2024-04-20T10:57:35.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.936+0000 7f840dffb640 1 -- 172.21.15.27:0/3762929621 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f841000acf0 con 0x7f8418155950 2024-04-20T10:57:35.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.937+0000 7f840dffb640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f83fc06c8f0 0x7f83fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:35.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.937+0000 7f840dffb640 1 -- 172.21.15.27:0/3762929621 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f841008b4a0 con 0x7f8418155950 2024-04-20T10:57:35.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.937+0000 7f841d643640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f83fc06c8f0 0x7f83fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:35.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.938+0000 7f841d643640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f83fc06c8f0 0x7f83fc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8414000f60 tx=0x7f8414009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:35.941 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:35.942+0000 7f840dffb640 1 -- 172.21.15.27:0/3762929621 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f84180cf490 con 0x7f8418155950 2024-04-20T10:57:36.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.397+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84180686d0 con 0x7f8418155950 2024-04-20T10:57:36.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.398+0000 7f840dffb640 1 -- 172.21.15.27:0/3762929621 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+6102 (secure 0 0 0) 0x7f84180686d0 con 0x7f8418155950 2024-04-20T10:57:36.398 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:36.398 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:32.332342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:resolve","state_seq":31,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:57:36.398 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 19 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f83fc06c8f0 msgr2=0x7f83fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f83fc06c8f0 0x7f83fc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8414000f60 tx=0x7f8414009040 comp rx=0 tx=0).stop 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418155950 msgr2=0x7f841814e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418155950 0x7f841814e600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f841000caf0 tx=0x7f8410007f30 comp rx=0 tx=0).stop 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 shutdown_connections 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f83fc06c8f0 0x7f83fc06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8418155950 0x7f841814e600 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84180d0c90 0x7f84180b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 --2- 172.21.15.27:0/3762929621 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f84180b54f0 0x7f84181f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 >> 172.21.15.27:0/3762929621 conn(0x7f84180676c0 msgr2=0x7f8418151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:36.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 shutdown_connections 2024-04-20T10:57:36.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:36.401+0000 7f841f8a7640 1 -- 172.21.15.27:0/3762929621 wait complete. 2024-04-20T10:57:36.415 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:57:38.417 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:38.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.573+0000 7f8f20596640 1 Processor -- start 2024-04-20T10:57:38.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.573+0000 7f8f20596640 1 -- start start 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f20596640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8f180b54f0 0x7f8f180b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f20596640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f180b5e70 con 0x7f8f18155950 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f1e332640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8f18155950 0x7f8f180cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60576/0 (socket says 172.21.15.27:60576) 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f1e332640 1 -- 172.21.15.27:0/1312300397 learned_addr learned my addr 172.21.15.27:0/1312300397 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f1db31640 1 --2- 172.21.15.27:0/1312300397 >> v2:172.21.15.137:3300/0 conn(0x7f8f180b54f0 0x7f8f180b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f20596640 1 -- 172.21.15.27:0/1312300397 --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f1815b9c0 con 0x7f8f180d0c90 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f20596640 1 -- 172.21.15.27:0/1312300397 --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f8f1815bca0 con 0x7f8f180b54f0 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407861392 0 0) 0x7f8f180b5e70 con 0x7f8f18155950 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f04003520 con 0x7f8f18155950 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733291612 0 0) 0x7f8f1815b9c0 con 0x7f8f180d0c90 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.575+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f180b5e70 con 0x7f8f180d0c90 2024-04-20T10:57:38.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1906696163 0 0) 0x7f8f04003520 con 0x7f8f18155950 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 >> v2:172.21.15.137:3300/0 conn(0x7f8f180b54f0 msgr2=0x7f8f180b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 --2- 172.21.15.27:0/1312300397 >> v2:172.21.15.137:3300/0 conn(0x7f8f180b54f0 0x7f8f180b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 >> v1:172.21.15.137:6789/0 conn(0x7f8f180d0c90 legacy=0x7f8f180d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1815bf80 con 0x7f8f18155950 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f8f14001670 con 0x7f8f18155950 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 >> v1:172.21.15.27:6789/0 conn(0x7f8f18155950 legacy=0x7f8f180cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f04004bd0 0x7f8f04004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f04003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f04004160 0x7f8f0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1815bca0 con 0x7f8f04004160 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0400bf30 con 0x7f8f04004bd0 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0400c130 con 0x7f8f040037a0 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1db31640 1 --2- 172.21.15.27:0/1312300397 >> v2:172.21.15.137:3300/0 conn(0x7f8f180b54f0 0x7f8f180b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1eb33640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f04004160 0x7f8f0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:38.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1e332640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f04004bd0 0x7f8f04004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1db31640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f04003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1db31640 1 -- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f04004bd0 msgr2=0x7f8f04004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1db31640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f04004bd0 0x7f8f04004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1db31640 1 -- 172.21.15.27:0/1312300397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f04004160 msgr2=0x7f8f0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1db31640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f04004160 0x7f8f0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1db31640 1 -- 172.21.15.27:0/1312300397 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1815bf80 con 0x7f8f040037a0 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1e332640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f04004bd0 0x7f8f04004fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1eb33640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f04004160 0x7f8f0400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.576+0000 7f8f1eb33640 1 -- 172.21.15.27:0/1312300397 reap_dead start 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.578+0000 7f8f1db31640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f04003c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009820 tx=0x7f8f04005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29d53462319a8a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.578+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0804a070 con 0x7f8f040037a0 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.578+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f0803d870 con 0x7f8f040037a0 2024-04-20T10:57:38.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.578+0000 7f8f1d330640 1 -- 172.21.15.27:0/1312300397 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08009860 con 0x7f8f040037a0 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.579+0000 7f8f20596640 1 -- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 msgr2=0x7f8f04003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.579+0000 7f8f20596640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f04003c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009820 tx=0x7f8f04005830 comp rx=0 tx=0).stop 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.579+0000 7f8f20596640 1 -- 172.21.15.27:0/1312300397 shutdown_connections 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.579+0000 7f8f20596640 1 --2- 172.21.15.27:0/1312300397 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f04003c20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.579+0000 7f8f20596640 1 -- 172.21.15.27:0/1312300397 >> 172.21.15.27:0/1312300397 conn(0x7f8f180676c0 msgr2=0x7f8f18067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.579+0000 7f8f20596640 1 -- 172.21.15.27:0/1312300397 shutdown_connections 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.579+0000 7f8f20596640 1 -- 172.21.15.27:0/1312300397 wait complete. 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.581+0000 7f8f20596640 1 Processor -- start 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.581+0000 7f8f20596640 1 -- start start 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f20596640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f1814fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f20596640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f180c8a10 0x7f8f1814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:38.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f20596640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1815a850 0x7f8f180ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f20596640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1815fb90 con 0x7f8f1815a850 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f20596640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1815c540 con 0x7f8f180c8a10 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f1eb33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1815a850 0x7f8f180ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f1eb33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1815a850 0x7f8f180ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37634/0 (socket says 172.21.15.27:37634) 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f1eb33640 1 -- 172.21.15.27:0/512408572 learned_addr learned my addr 172.21.15.27:0/512408572 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8f1815fd10 con 0x7f8f040037a0 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f1e332640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f1814fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.582+0000 7f8f1db31640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f180c8a10 0x7f8f1814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f1e332640 1 -- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f180c8a10 msgr2=0x7f8f1814e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f1e332640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f180c8a10 0x7f8f1814e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f1e332640 1 -- 172.21.15.27:0/512408572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1815a850 msgr2=0x7f8f180ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f1e332640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1815a850 0x7f8f180ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f1e332640 1 -- 172.21.15.27:0/512408572 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f181521f0 con 0x7f8f040037a0 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f1eb33640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1815a850 0x7f8f180ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:38.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f1db31640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f180c8a10 0x7f8f1814e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:38.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f1e332640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f1814fdd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8f140017e0 tx=0x7f8f140077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:38.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f0effd640 1 -- 172.21.15.27:0/512408572 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400fae0 con 0x7f8f040037a0 2024-04-20T10:57:38.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f0effd640 1 -- 172.21.15.27:0/512408572 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f14018070 con 0x7f8f040037a0 2024-04-20T10:57:38.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f181503f0 con 0x7f8f040037a0 2024-04-20T10:57:38.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f0effd640 1 -- 172.21.15.27:0/512408572 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400fe80 con 0x7f8f040037a0 2024-04-20T10:57:38.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.583+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f18150690 con 0x7f8f040037a0 2024-04-20T10:57:38.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.584+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f180c7bb0 con 0x7f8f040037a0 2024-04-20T10:57:38.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.585+0000 7f8f0effd640 1 -- 172.21.15.27:0/512408572 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f1401d460 con 0x7f8f040037a0 2024-04-20T10:57:38.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.586+0000 7f8f0effd640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8efc06c8f0 0x7f8efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:38.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.586+0000 7f8f0effd640 1 -- 172.21.15.27:0/512408572 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f8f1408bd00 con 0x7f8f040037a0 2024-04-20T10:57:38.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.586+0000 7f8f1db31640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8efc06c8f0 0x7f8efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:38.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.587+0000 7f8f1db31640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8efc06c8f0 0x7f8efc06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f08047000 tx=0x7f8f08041000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:38.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:38.591+0000 7f8f0effd640 1 -- 172.21.15.27:0/512408572 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8f180c7bb0 con 0x7f8f040037a0 2024-04-20T10:57:39.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.051+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f1815b9c0 con 0x7f8f040037a0 2024-04-20T10:57:39.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.052+0000 7f8f0effd640 1 -- 172.21.15.27:0/512408572 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+6104 (secure 0 0 0) 0x7f8f1815b9c0 con 0x7f8f040037a0 2024-04-20T10:57:39.051 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:39.052 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:38.042616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:57:39.052 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 20 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8efc06c8f0 msgr2=0x7f8efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8efc06c8f0 0x7f8efc06edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8f08047000 tx=0x7f8f08041000 comp rx=0 tx=0).stop 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 msgr2=0x7f8f1814fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f1814fdd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8f140017e0 tx=0x7f8f140077f0 comp rx=0 tx=0).stop 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 shutdown_connections 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8efc06c8f0 0x7f8efc06edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f1815a850 0x7f8f180ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f180c8a10 0x7f8f1814e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 --2- 172.21.15.27:0/512408572 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8f040037a0 0x7f8f1814fdd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 >> 172.21.15.27:0/512408572 conn(0x7f8f180676c0 msgr2=0x7f8f180cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 shutdown_connections 2024-04-20T10:57:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:39.054+0000 7f8f20596640 1 -- 172.21.15.27:0/512408572 wait complete. 2024-04-20T10:57:39.065 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:57:41.066 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.221+0000 7fbd25831640 1 Processor -- start 2024-04-20T10:57:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.221+0000 7fbd25831640 1 -- start start 2024-04-20T10:57:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.222+0000 7fbd25831640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2015d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.222+0000 7fbd25831640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbd201537f0 0x7fbd200c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.222+0000 7fbd25831640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd2015d6b0 con 0x7fbd20159260 2024-04-20T10:57:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.222+0000 7fbd25831640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbd2006cad0 con 0x7fbd201537f0 2024-04-20T10:57:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.222+0000 7fbd25831640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbd200c7cd0 con 0x7fbd20153410 2024-04-20T10:57:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.222+0000 7fbd17fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbd201537f0 0x7fbd200c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.222+0000 7fbd17fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbd201537f0 0x7fbd200c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37648/0 (socket says 172.21.15.27:37648) 2024-04-20T10:57:41.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.222+0000 7fbd17fff640 1 -- 172.21.15.27:0/3780343623 learned_addr learned my addr 172.21.15.27:0/3780343623 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:41.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.223+0000 7fbd1effd640 1 --2- 172.21.15.27:0/3780343623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:41.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.223+0000 7fbd17fff640 1 -- 172.21.15.27:0/3780343623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 msgr2=0x7fbd2015d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:41.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.223+0000 7fbd17fff640 1 --2- 172.21.15.27:0/3780343623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2015d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.223+0000 7fbd17fff640 1 -- 172.21.15.27:0/3780343623 >> v1:172.21.15.27:6789/0 conn(0x7fbd20159260 legacy=0x7fbd2006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.223+0000 7fbd17fff640 1 -- 172.21.15.27:0/3780343623 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd200c7fb0 con 0x7fbd201537f0 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.224+0000 7fbd17fff640 1 --2- 172.21.15.27:0/3780343623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201537f0 0x7fbd200c6730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009870 tx=0x7fbd0802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b5dde3a320e780 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.224+0000 7fbd1e7fc640 1 -- 172.21.15.27:0/3780343623 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0802f830 con 0x7fbd201537f0 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.224+0000 7fbd1e7fc640 1 -- 172.21.15.27:0/3780343623 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd0802f9d0 con 0x7fbd201537f0 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.224+0000 7fbd1e7fc640 1 -- 172.21.15.27:0/3780343623 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0802fda0 con 0x7fbd201537f0 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.225+0000 7fbd25831640 1 -- 172.21.15.27:0/3780343623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201537f0 msgr2=0x7fbd200c6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.225+0000 7fbd25831640 1 --2- 172.21.15.27:0/3780343623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201537f0 0x7fbd200c6730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009870 tx=0x7fbd0802ba10 comp rx=0 tx=0).stop 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.225+0000 7fbd25831640 1 -- 172.21.15.27:0/3780343623 shutdown_connections 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.225+0000 7fbd25831640 1 --2- 172.21.15.27:0/3780343623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd201537f0 0x7fbd200c6730 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.225+0000 7fbd25831640 1 --2- 172.21.15.27:0/3780343623 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2015d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.225+0000 7fbd25831640 1 -- 172.21.15.27:0/3780343623 >> 172.21.15.27:0/3780343623 conn(0x7fbd200ccbd0 msgr2=0x7fbd200ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.225+0000 7fbd25831640 1 -- 172.21.15.27:0/3780343623 shutdown_connections 2024-04-20T10:57:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.225+0000 7fbd25831640 1 -- 172.21.15.27:0/3780343623 wait complete. 2024-04-20T10:57:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.227+0000 7fbd25831640 1 Processor -- start 2024-04-20T10:57:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.227+0000 7fbd25831640 1 -- start start 2024-04-20T10:57:41.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.228+0000 7fbd25831640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2013c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd25831640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd20159260 0x7fbd20148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd25831640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd200cb320 0x7fbd20148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd25831640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2015cc50 con 0x7fbd20159260 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd25831640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2015c970 con 0x7fbd200cb320 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd25831640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbd200c75d0 con 0x7fbd20153410 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd1effd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2013c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd1effd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2013c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:43270/0 (socket says 172.21.15.27:43270) 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd17fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd20159260 0x7fbd20148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd1effd640 1 -- 172.21.15.27:0/1998822795 learned_addr learned my addr 172.21.15.27:0/1998822795 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd1f7fe640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd200cb320 0x7fbd20148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd17fff640 1 -- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 msgr2=0x7fbd2013c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd17fff640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2013c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd17fff640 1 -- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd200cb320 msgr2=0x7fbd20148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:41.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd17fff640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd200cb320 0x7fbd20148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd17fff640 1 -- 172.21.15.27:0/1998822795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd200cc8a0 con 0x7fbd20159260 2024-04-20T10:57:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd1effd640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2013c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.229+0000 7fbd1f7fe640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd200cb320 0x7fbd20148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.230+0000 7fbd17fff640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd20159260 0x7fbd20148730 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbd08033000 tx=0x7fbd080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.230+0000 7fbd1cff9640 1 -- 172.21.15.27:0/1998822795 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08037a90 con 0x7fbd20159260 2024-04-20T10:57:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.230+0000 7fbd1cff9640 1 -- 172.21.15.27:0/1998822795 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd08037c30 con 0x7fbd20159260 2024-04-20T10:57:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.230+0000 7fbd1cff9640 1 -- 172.21.15.27:0/1998822795 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08042480 con 0x7fbd20159260 2024-04-20T10:57:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.230+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2015ba90 con 0x7fbd20159260 2024-04-20T10:57:41.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.230+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2015c030 con 0x7fbd20159260 2024-04-20T10:57:41.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.231+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2004fcf0 con 0x7fbd20159260 2024-04-20T10:57:41.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.233+0000 7fbd1cff9640 1 -- 172.21.15.27:0/1998822795 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd08049020 con 0x7fbd20159260 2024-04-20T10:57:41.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.233+0000 7fbd1cff9640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbd0006c8f0 0x7fbd0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:41.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.234+0000 7fbd1cff9640 1 -- 172.21.15.27:0/1998822795 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fbd080ae9e0 con 0x7fbd20159260 2024-04-20T10:57:41.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.234+0000 7fbd1effd640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbd0006c8f0 0x7fbd0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:41.234 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.234+0000 7fbd1effd640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbd0006c8f0 0x7fbd0006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006fd0 tx=0x7fbd10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:41.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.241+0000 7fbd1cff9640 1 -- 172.21.15.27:0/1998822795 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd08050020 con 0x7fbd20159260 2024-04-20T10:57:41.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.702+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd200cb930 con 0x7fbd20159260 2024-04-20T10:57:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.702+0000 7fbd1cff9640 1 -- 172.21.15.27:0/1998822795 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+6104 (secure 0 0 0) 0x7fbd08033030 con 0x7fbd20159260 2024-04-20T10:57:41.702 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:41.702 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:38.042616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:57:41.702 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 20 2024-04-20T10:57:41.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbd0006c8f0 msgr2=0x7fbd0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:41.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbd0006c8f0 0x7fbd0006edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006fd0 tx=0x7fbd10008040 comp rx=0 tx=0).stop 2024-04-20T10:57:41.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd20159260 msgr2=0x7fbd20148730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd20159260 0x7fbd20148730 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbd08033000 tx=0x7fbd080046c0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 shutdown_connections 2024-04-20T10:57:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbd0006c8f0 0x7fbd0006edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd200cb320 0x7fbd20148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd20159260 0x7fbd20148730 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 --2- 172.21.15.27:0/1998822795 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd20153410 0x7fbd2013c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 >> 172.21.15.27:0/1998822795 conn(0x7fbd200ccbd0 msgr2=0x7fbd2006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 shutdown_connections 2024-04-20T10:57:41.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:41.705+0000 7fbd25831640 1 -- 172.21.15.27:0/1998822795 wait complete. 2024-04-20T10:57:41.721 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 20, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 20, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4259": { "addr": "172.21.15.137:6835/1164978945", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6834", "nonce": 1164978945, "type": "v2" }, { "addr": "172.21.15.137:6835", "nonce": 1164978945, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4259, "incarnation": 18, "join_fscid": -1, "name": "f", "rank": 4, "state": "up:reconnect", "state_seq": 33 }, "gid_4272": { "addr": "172.21.15.27:6835/859496751", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6832", "nonce": 859496751, "type": "v2" }, { "addr": "172.21.15.27:6835", "nonce": 859496751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4272, "incarnation": 14, "join_fscid": -1, "name": "c", "rank": 3, "state": "up:active", "state_seq": 9 }, "gid_4275": { "addr": "172.21.15.27:6837/1680990583", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6834", "nonce": 1680990583, "type": "v2" }, { "addr": "172.21.15.27:6837", "nonce": 1680990583, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4275, "incarnation": 12, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4278": { "addr": "172.21.15.27:6836/4195543922", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6833", "nonce": 4195543922, "type": "v2" }, { "addr": "172.21.15.27:6836", "nonce": 4195543922, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4278, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 23, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T10:57:38.042616+0000", "qdb_cluster": [ 4272, 4275, 4278, 4293 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 4278, "mds_2": 4275, "mds_3": 4272, "mds_4": 4259 } } } ], "standbys": [] } 2024-04-20T10:57:41.978 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:57:41.980 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:57:43.722 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:43.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.824+0000 7fbc1d0b3640 1 Processor -- start 2024-04-20T10:57:43.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.824+0000 7fbc1d0b3640 1 -- start start 2024-04-20T10:57:43.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.825+0000 7fbc1d0b3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc180d0c90 0x7fbc180d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:43.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.825+0000 7fbc1d0b3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1815b9c0 con 0x7fbc18155950 2024-04-20T10:57:43.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.825+0000 7fbc1d0b3640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1815bca0 con 0x7fbc180b54f0 2024-04-20T10:57:43.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.825+0000 7fbc1d0b3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1815bf80 con 0x7fbc180d0c90 2024-04-20T10:57:43.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.825+0000 7fbc1759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc180d0c90 0x7fbc180d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.825+0000 7fbc16d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbc18155950 0x7fbc180cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60598/0 (socket says 172.21.15.27:60598) 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.825+0000 7fbc16d9d640 1 -- 172.21.15.27:0/1531395811 learned_addr learned my addr 172.21.15.27:0/1531395811 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.825+0000 7fbc1759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbc180d0c90 0x7fbc180d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37678/0 (socket says 172.21.15.27:37678) 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.825+0000 7fbc1659c640 1 --1- 172.21.15.27:0/1531395811 >> v1:172.21.15.137:6789/0 conn(0x7fbc180b54f0 0x7fbc180b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:39576/0 (socket says 172.21.15.27:39576) 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.826+0000 7fbc15d9b640 1 -- 172.21.15.27:0/1531395811 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142408736 0 0) 0x7fbc1815bca0 con 0x7fbc180b54f0 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.826+0000 7fbc15d9b640 1 -- 172.21.15.27:0/1531395811 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc00003520 con 0x7fbc180b54f0 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.826+0000 7fbc1759e640 1 -- 172.21.15.27:0/1531395811 >> v1:172.21.15.137:6789/0 conn(0x7fbc180b54f0 legacy=0x7fbc180b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.826+0000 7fbc1759e640 1 -- 172.21.15.27:0/1531395811 >> v1:172.21.15.27:6789/0 conn(0x7fbc18155950 legacy=0x7fbc180cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.826+0000 7fbc15d9b640 1 -- 172.21.15.27:0/1531395811 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 124787488 0 0) 0x7fbc1815b9c0 con 0x7fbc18155950 2024-04-20T10:57:43.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.826+0000 7fbc1759e640 1 -- 172.21.15.27:0/1531395811 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1815c260 con 0x7fbc180d0c90 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.827+0000 7fbc1759e640 1 --2- 172.21.15.27:0/1531395811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 0x7fbc180d10a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009d90 tx=0x7fbc1002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe5fe8d89bfb86d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.827+0000 7fbc15d9b640 1 -- 172.21.15.27:0/1531395811 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10039070 con 0x7fbc180d0c90 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.827+0000 7fbc15d9b640 1 -- 172.21.15.27:0/1531395811 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1002cd80 con 0x7fbc180d0c90 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.827+0000 7fbc15d9b640 1 -- 172.21.15.27:0/1531395811 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10034640 con 0x7fbc180d0c90 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.828+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/1531395811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 msgr2=0x7fbc180d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.828+0000 7fbc1d0b3640 1 --2- 172.21.15.27:0/1531395811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 0x7fbc180d10a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009d90 tx=0x7fbc1002bfd0 comp rx=0 tx=0).stop 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.828+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/1531395811 shutdown_connections 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.828+0000 7fbc1d0b3640 1 --2- 172.21.15.27:0/1531395811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 0x7fbc180d10a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.828+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/1531395811 >> 172.21.15.27:0/1531395811 conn(0x7fbc180676c0 msgr2=0x7fbc18067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.828+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/1531395811 shutdown_connections 2024-04-20T10:57:43.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.828+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/1531395811 wait complete. 2024-04-20T10:57:43.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.830+0000 7fbc1d0b3640 1 Processor -- start 2024-04-20T10:57:43.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.830+0000 7fbc1d0b3640 1 -- start start 2024-04-20T10:57:43.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc1d0b3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 0x7fbc181f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:43.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc1d0b3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 0x7fbc18151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc1d0b3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc18155950 0x7fbc181478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc1d0b3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1815fb90 con 0x7fbc18155950 2024-04-20T10:57:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc1d0b3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180d15e0 con 0x7fbc180b54f0 2024-04-20T10:57:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc1d0b3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1815fd10 con 0x7fbc180d0c90 2024-04-20T10:57:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc16d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 0x7fbc181f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc16d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 0x7fbc181f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:44352/0 (socket says 172.21.15.27:44352) 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc16d9d640 1 -- 172.21.15.27:0/3716264630 learned_addr learned my addr 172.21.15.27:0/3716264630 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc1759e640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc18155950 0x7fbc181478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.831+0000 7fbc1659c640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 0x7fbc18151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbc16d9d640 1 -- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 msgr2=0x7fbc18151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbc16d9d640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 0x7fbc18151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbc16d9d640 1 -- 172.21.15.27:0/3716264630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc18155950 msgr2=0x7fbc181478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbc16d9d640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc18155950 0x7fbc181478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbc16d9d640 1 -- 172.21.15.27:0/3716264630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc181f4810 con 0x7fbc180b54f0 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbc16d9d640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 0x7fbc181f42d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800c4d0 tx=0x7fbc0800c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbbff7fe640 1 -- 172.21.15.27:0/3716264630 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08018070 con 0x7fbc180b54f0 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbbff7fe640 1 -- 172.21.15.27:0/3716264630 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc08007500 con 0x7fbc180b54f0 2024-04-20T10:57:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc180cf6e0 con 0x7fbc180b54f0 2024-04-20T10:57:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbbff7fe640 1 -- 172.21.15.27:0/3716264630 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08004a20 con 0x7fbc180b54f0 2024-04-20T10:57:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.832+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc18154880 con 0x7fbc180b54f0 2024-04-20T10:57:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.834+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1804fcf0 con 0x7fbc180b54f0 2024-04-20T10:57:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.834+0000 7fbbff7fe640 1 -- 172.21.15.27:0/3716264630 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc0801b070 con 0x7fbc180b54f0 2024-04-20T10:57:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.835+0000 7fbbff7fe640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbc04075130 0x7fbc040775f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.835+0000 7fbbff7fe640 1 -- 172.21.15.27:0/3716264630 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fbc0808afa0 con 0x7fbc180b54f0 2024-04-20T10:57:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.835+0000 7fbc1659c640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbc04075130 0x7fbc040775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.836+0000 7fbc1659c640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbc04075130 0x7fbc040775f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c007fd0 tx=0x7fbc0c007da0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:43.839+0000 7fbbff7fe640 1 -- 172.21.15.27:0/3716264630 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbc180cf6e0 con 0x7fbc180b54f0 2024-04-20T10:57:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.118+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc180cc930 con 0x7fbc180b54f0 2024-04-20T10:57:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.119+0000 7fbbff7fe640 1 -- 172.21.15.27:0/3716264630 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6101 (secure 0 0 0) 0x7fbc08058bc0 con 0x7fbc180b54f0 2024-04-20T10:57:44.118 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:44.119 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:43.063011+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:57:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 21 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbc04075130 msgr2=0x7fbc040775f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbc04075130 0x7fbc040775f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c007fd0 tx=0x7fbc0c007da0 comp rx=0 tx=0).stop 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 msgr2=0x7fbc181f42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 0x7fbc181f42d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800c4d0 tx=0x7fbc0800c990 comp rx=0 tx=0).stop 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 shutdown_connections 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbc04075130 0x7fbc040775f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc18155950 0x7fbc181478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 0x7fbc18151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 --2- 172.21.15.27:0/3716264630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 0x7fbc181f42d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 >> 172.21.15.27:0/3716264630 conn(0x7fbc180676c0 msgr2=0x7fbc180b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 shutdown_connections 2024-04-20T10:57:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:44.121+0000 7fbc1d0b3640 1 -- 172.21.15.27:0/3716264630 wait complete. 2024-04-20T10:57:44.132 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:57:46.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.288+0000 7f17237a4640 1 Processor -- start 2024-04-20T10:57:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.288+0000 7f17237a4640 1 -- start start 2024-04-20T10:57:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f17237a4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0b54f0 0x7f171c0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f17237a4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f171c0d0c90 0x7f171c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f17237a4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f171c0b5e70 con 0x7f171c155950 2024-04-20T10:57:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f17237a4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f171c15b9c0 con 0x7f171c0d0c90 2024-04-20T10:57:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f17237a4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f171c15bca0 con 0x7f171c0b54f0 2024-04-20T10:57:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f1721d41640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f171c0d0c90 0x7f171c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f1721540640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f171c155950 0x7f171c0cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37378/0 (socket says 172.21.15.27:37378) 2024-04-20T10:57:46.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f1721540640 1 -- 172.21.15.27:0/2243784230 learned_addr learned my addr 172.21.15.27:0/2243784230 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:46.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f1720d3f640 1 --2- 172.21.15.27:0/2243784230 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0b54f0 0x7f171c0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:46.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f1721d41640 1 -- 172.21.15.27:0/2243784230 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0b54f0 msgr2=0x7f171c0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:46.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f1721d41640 1 --2- 172.21.15.27:0/2243784230 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0b54f0 0x7f171c0b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:46.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f1721d41640 1 -- 172.21.15.27:0/2243784230 >> v1:172.21.15.27:6789/0 conn(0x7f171c155950 legacy=0x7f171c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:46.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.290+0000 7f1721d41640 1 -- 172.21.15.27:0/2243784230 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c15bf80 con 0x7f171c0d0c90 2024-04-20T10:57:46.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.292+0000 7f1721d41640 1 --2- 172.21.15.27:0/2243784230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0d0c90 0x7f171c0d1100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1714009d90 tx=0x7f171402bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9fab5998d75a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:46.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.292+0000 7f1713fff640 1 -- 172.21.15.27:0/2243784230 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714039070 con 0x7f171c0d0c90 2024-04-20T10:57:46.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.292+0000 7f1713fff640 1 -- 172.21.15.27:0/2243784230 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f171402caa0 con 0x7f171c0d0c90 2024-04-20T10:57:46.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.292+0000 7f1713fff640 1 -- 172.21.15.27:0/2243784230 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714034440 con 0x7f171c0d0c90 2024-04-20T10:57:46.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.293+0000 7f17237a4640 1 -- 172.21.15.27:0/2243784230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0d0c90 msgr2=0x7f171c0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:46.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.293+0000 7f17237a4640 1 --2- 172.21.15.27:0/2243784230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0d0c90 0x7f171c0d1100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1714009d90 tx=0x7f171402bd70 comp rx=0 tx=0).stop 2024-04-20T10:57:46.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.293+0000 7f17237a4640 1 -- 172.21.15.27:0/2243784230 shutdown_connections 2024-04-20T10:57:46.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.293+0000 7f17237a4640 1 --2- 172.21.15.27:0/2243784230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0d0c90 0x7f171c0d1100 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:46.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.293+0000 7f17237a4640 1 --2- 172.21.15.27:0/2243784230 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0b54f0 0x7f171c0b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:46.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.293+0000 7f17237a4640 1 -- 172.21.15.27:0/2243784230 >> 172.21.15.27:0/2243784230 conn(0x7f171c0676c0 msgr2=0x7f171c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:46.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.293+0000 7f17237a4640 1 -- 172.21.15.27:0/2243784230 shutdown_connections 2024-04-20T10:57:46.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.293+0000 7f17237a4640 1 -- 172.21.15.27:0/2243784230 wait complete. 2024-04-20T10:57:46.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.295+0000 7f17237a4640 1 Processor -- start 2024-04-20T10:57:46.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.295+0000 7f17237a4640 1 -- start start 2024-04-20T10:57:46.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.296+0000 7f17237a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0b54f0 0x7f171c15c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721540640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0b54f0 0x7f171c15c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721540640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0b54f0 0x7f171c15c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37504/0 (socket says 172.21.15.27:37504) 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721540640 1 -- 172.21.15.27:0/3674784140 learned_addr learned my addr 172.21.15.27:0/3674784140 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f17237a4640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0d0c90 0x7f171c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f17237a4640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f171c155950 0x7f171c14f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f171c0d1640 con 0x7f171c0b54f0 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f171c15c540 con 0x7f171c155950 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f171c15fb90 con 0x7f171c0d0c90 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1720d3f640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0d0c90 0x7f171c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721540640 1 -- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0d0c90 msgr2=0x7f171c1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721540640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0d0c90 0x7f171c1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721540640 1 -- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f171c155950 msgr2=0x7f171c14f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721d41640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f171c155950 0x7f171c14f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:46.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721540640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f171c155950 0x7f171c14f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721540640 1 -- 172.21.15.27:0/3674784140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c1f1c80 con 0x7f171c0b54f0 2024-04-20T10:57:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.297+0000 7f1721d41640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f171c155950 0x7f171c14f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:57:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.298+0000 7f1721540640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0b54f0 0x7f171c15c970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f171800c7b0 tx=0x7f171800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.298+0000 7f1711ffb640 1 -- 172.21.15.27:0/3674784140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718018070 con 0x7f171c0b54f0 2024-04-20T10:57:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.298+0000 7f1711ffb640 1 -- 172.21.15.27:0/3674784140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17180072d0 con 0x7f171c0b54f0 2024-04-20T10:57:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.298+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c1f1f60 con 0x7f171c0b54f0 2024-04-20T10:57:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.298+0000 7f1711ffb640 1 -- 172.21.15.27:0/3674784140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17180047e0 con 0x7f171c0b54f0 2024-04-20T10:57:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.298+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c1f2520 con 0x7f171c0b54f0 2024-04-20T10:57:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.299+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171c04fcf0 con 0x7f171c0b54f0 2024-04-20T10:57:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.300+0000 7f1711ffb640 1 -- 172.21.15.27:0/3674784140 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f171801b070 con 0x7f171c0b54f0 2024-04-20T10:57:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.300+0000 7f1711ffb640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f170006c8f0 0x7f170006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.301+0000 7f1711ffb640 1 -- 172.21.15.27:0/3674784140 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f171808b070 con 0x7f171c0b54f0 2024-04-20T10:57:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.301+0000 7f1720d3f640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f170006c8f0 0x7f170006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.301+0000 7f1720d3f640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f170006c8f0 0x7f170006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f171c155260 tx=0x7f170c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:46.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.305+0000 7f1711ffb640 1 -- 172.21.15.27:0/3674784140 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1718058d70 con 0x7f171c0b54f0 2024-04-20T10:57:46.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.607+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f171c0b3db0 con 0x7f171c0b54f0 2024-04-20T10:57:46.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.607+0000 7f1711ffb640 1 -- 172.21.15.27:0/3674784140 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+6101 (secure 0 0 0) 0x7f171805d070 con 0x7f171c0b54f0 2024-04-20T10:57:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:46.609 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:43.063011+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:57:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 21 2024-04-20T10:57:46.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f170006c8f0 msgr2=0x7f170006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:46.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f170006c8f0 0x7f170006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f171c155260 tx=0x7f170c008040 comp rx=0 tx=0).stop 2024-04-20T10:57:46.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0b54f0 msgr2=0x7f171c15c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0b54f0 0x7f171c15c970 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f171800c7b0 tx=0x7f171800cc70 comp rx=0 tx=0).stop 2024-04-20T10:57:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 shutdown_connections 2024-04-20T10:57:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f170006c8f0 0x7f170006edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f171c155950 0x7f171c14f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f171c0d0c90 0x7f171c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 --2- 172.21.15.27:0/3674784140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f171c0b54f0 0x7f171c15c970 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.609+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 >> 172.21.15.27:0/3674784140 conn(0x7f171c0676c0 msgr2=0x7f171c067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.610+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 shutdown_connections 2024-04-20T10:57:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:46.610+0000 7f17237a4640 1 -- 172.21.15.27:0/3674784140 wait complete. 2024-04-20T10:57:46.622 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:57:48.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.725+0000 7fb96cf96640 1 Processor -- start 2024-04-20T10:57:48.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.725+0000 7fb96cf96640 1 -- start start 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.725+0000 7fb96cf96640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fb96815f230 0x7fb96815c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb96cf96640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9680ccbd0 0x7fb968155410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb96cf96640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb96815f9a0 con 0x7fb96815ee50 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb96cf96640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fb9680cb870 con 0x7fb96815f230 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb96cf96640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fb9680cbb30 con 0x7fb9680ccbd0 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb96659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb96815ee50 0x7fb96806cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37392/0 (socket says 172.21.15.27:37392) 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb96659c640 1 -- 172.21.15.27:0/707261141 learned_addr learned my addr 172.21.15.27:0/707261141 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb965d9b640 1 --2- 172.21.15.27:0/707261141 >> v2:172.21.15.137:3300/0 conn(0x7fb96815f230 0x7fb96815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb966d9d640 1 --2- 172.21.15.27:0/707261141 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9680ccbd0 0x7fb968155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb96559a640 1 -- 172.21.15.27:0/707261141 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1145305527 0 0) 0x7fb958001110 con 0x7fb96815ee50 2024-04-20T10:57:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb96559a640 1 -- 172.21.15.27:0/707261141 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb950003520 con 0x7fb96815ee50 2024-04-20T10:57:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb965d9b640 1 -- 172.21.15.27:0/707261141 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9680ccbd0 msgr2=0x7fb968155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb965d9b640 1 --2- 172.21.15.27:0/707261141 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9680ccbd0 0x7fb968155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb965d9b640 1 -- 172.21.15.27:0/707261141 >> v1:172.21.15.27:6789/0 conn(0x7fb96815ee50 legacy=0x7fb96806cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb965d9b640 1 -- 172.21.15.27:0/707261141 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9680cbe10 con 0x7fb96815f230 2024-04-20T10:57:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.726+0000 7fb966d9d640 1 --2- 172.21.15.27:0/707261141 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9680ccbd0 0x7fb968155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:57:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.727+0000 7fb965d9b640 1 --2- 172.21.15.27:0/707261141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96815f230 0x7fb96815c180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0097a0 tx=0x7fb95c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2521f7871db3af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.727+0000 7fb96559a640 1 -- 172.21.15.27:0/707261141 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c034070 con 0x7fb96815f230 2024-04-20T10:57:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.727+0000 7fb96559a640 1 -- 172.21.15.27:0/707261141 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95c030610 con 0x7fb96815f230 2024-04-20T10:57:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.727+0000 7fb96559a640 1 -- 172.21.15.27:0/707261141 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0309e0 con 0x7fb96815f230 2024-04-20T10:57:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.728+0000 7fb96cf96640 1 -- 172.21.15.27:0/707261141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96815f230 msgr2=0x7fb96815c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.728+0000 7fb96cf96640 1 --2- 172.21.15.27:0/707261141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96815f230 0x7fb96815c180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0097a0 tx=0x7fb95c02b960 comp rx=0 tx=0).stop 2024-04-20T10:57:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.728+0000 7fb96cf96640 1 -- 172.21.15.27:0/707261141 shutdown_connections 2024-04-20T10:57:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.728+0000 7fb96cf96640 1 --2- 172.21.15.27:0/707261141 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9680ccbd0 0x7fb968155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.728+0000 7fb96cf96640 1 --2- 172.21.15.27:0/707261141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96815f230 0x7fb96815c180 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.728+0000 7fb96cf96640 1 -- 172.21.15.27:0/707261141 >> 172.21.15.27:0/707261141 conn(0x7fb96806e010 msgr2=0x7fb9680d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.728+0000 7fb96cf96640 1 -- 172.21.15.27:0/707261141 shutdown_connections 2024-04-20T10:57:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.728+0000 7fb96cf96640 1 -- 172.21.15.27:0/707261141 wait complete. 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.729+0000 7fb96cf96640 1 Processor -- start 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.729+0000 7fb96cf96640 1 -- start start 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96cf96640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9680ccbd0 0x7fb9681f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96cf96640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96815ee50 0x7fb9680ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96cf96640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb96815f230 0x7fb96814db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96cf96640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9680cc6b0 con 0x7fb9680ccbd0 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96cf96640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9680cc3d0 con 0x7fb96815ee50 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96cf96640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fb96815cae0 con 0x7fb96815f230 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9680ccbd0 0x7fb9681f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9680ccbd0 0x7fb9681f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37520/0 (socket says 172.21.15.27:37520) 2024-04-20T10:57:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96659c640 1 -- 172.21.15.27:0/1214888415 learned_addr learned my addr 172.21.15.27:0/1214888415 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb965d9b640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96815ee50 0x7fb9680ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb966d9d640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb96815f230 0x7fb96814db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96659c640 1 -- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb96815f230 msgr2=0x7fb96814db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96659c640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb96815f230 0x7fb96814db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96659c640 1 -- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96815ee50 msgr2=0x7fb9680ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96659c640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96815ee50 0x7fb9680ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96659c640 1 -- 172.21.15.27:0/1214888415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9680c7700 con 0x7fb9680ccbd0 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.730+0000 7fb96659c640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9680ccbd0 0x7fb9681f4530 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c3a0 tx=0x7fb95800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.731+0000 7fb956ffd640 1 -- 172.21.15.27:0/1214888415 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958017070 con 0x7fb9680ccbd0 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.731+0000 7fb956ffd640 1 -- 172.21.15.27:0/1214888415 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb958005ce0 con 0x7fb9680ccbd0 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.731+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9680c79e0 con 0x7fb9680ccbd0 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.731+0000 7fb956ffd640 1 -- 172.21.15.27:0/1214888415 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958007310 con 0x7fb9680ccbd0 2024-04-20T10:57:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.731+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb968156460 con 0x7fb9680ccbd0 2024-04-20T10:57:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.732+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb968156790 con 0x7fb9680ccbd0 2024-04-20T10:57:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.732+0000 7fb956ffd640 1 -- 172.21.15.27:0/1214888415 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb95801a070 con 0x7fb9680ccbd0 2024-04-20T10:57:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.733+0000 7fb956ffd640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb94806c8f0 0x7fb94806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.733+0000 7fb956ffd640 1 -- 172.21.15.27:0/1214888415 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fb95808afa0 con 0x7fb9680ccbd0 2024-04-20T10:57:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.733+0000 7fb965d9b640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb94806c8f0 0x7fb94806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.733+0000 7fb965d9b640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb94806c8f0 0x7fb94806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb95c036000 tx=0x7fb95c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.737+0000 7fb956ffd640 1 -- 172.21.15.27:0/1214888415 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb96815f9a0 con 0x7fb9680ccbd0 2024-04-20T10:57:48.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.962+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9680688e0 con 0x7fb9680ccbd0 2024-04-20T10:57:48.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.962+0000 7fb956ffd640 1 -- 172.21.15.27:0/1214888415 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+6106 (secure 0 0 0) 0x7fb958058c30 con 0x7fb9680ccbd0 2024-04-20T10:57:48.963 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:48.963 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4259,4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:57:48.964 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 22 2024-04-20T10:57:48.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.964+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb94806c8f0 msgr2=0x7fb94806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:48.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.964+0000 7fb96cf96640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb94806c8f0 0x7fb94806edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb95c036000 tx=0x7fb95c03b040 comp rx=0 tx=0).stop 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.964+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9680ccbd0 msgr2=0x7fb9681f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.964+0000 7fb96cf96640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9680ccbd0 0x7fb9681f4530 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb95800c3a0 tx=0x7fb95800c7a0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.964+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 shutdown_connections 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.964+0000 7fb96cf96640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb94806c8f0 0x7fb94806edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.964+0000 7fb96cf96640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb96815f230 0x7fb96814db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.964+0000 7fb96cf96640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96815ee50 0x7fb9680ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.964+0000 7fb96cf96640 1 --2- 172.21.15.27:0/1214888415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9680ccbd0 0x7fb9681f4530 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.965+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 >> 172.21.15.27:0/1214888415 conn(0x7fb96806e010 msgr2=0x7fb9680cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.965+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 shutdown_connections 2024-04-20T10:57:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:48.965+0000 7fb96cf96640 1 -- 172.21.15.27:0/1214888415 wait complete. 2024-04-20T10:57:48.976 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=4, replacing gid=4262 2024-04-20T10:57:48.977 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.b 2024-04-20T10:57:48.977 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2024-04-20T10:57:48.977 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T10:57:48.977 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T10:57:48.979 INFO:tasks.ceph.mds.b:Started 2024-04-20T10:57:48.979 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:57:49.050 INFO:tasks.ceph.mds.b.smithi137.stdout:starting mds.b at 2024-04-20T10:57:49.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.083+0000 7ff70a952640 1 Processor -- start 2024-04-20T10:57:49.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.083+0000 7ff70a952640 1 -- start start 2024-04-20T10:57:49.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff70a952640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff704155950 0x7ff7040cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff70a952640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff7040cc330 0x7ff7040b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff70a952640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7040b5a30 0x7ff7040d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff70a952640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff7040b5e80 con 0x7ff704155950 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff70a952640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7ff70415b9c0 con 0x7ff7040cc330 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff70a952640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff70415bca0 con 0x7ff7040b5a30 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff703fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff704155950 0x7ff7040cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff703fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff704155950 0x7ff7040cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37524/0 (socket says 172.21.15.27:37524) 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff703fff640 1 -- 172.21.15.27:0/607310609 learned_addr learned my addr 172.21.15.27:0/607310609 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff708eef640 1 --2- 172.21.15.27:0/607310609 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7040b5a30 0x7ff7040d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff7037fe640 1 --2- 172.21.15.27:0/607310609 >> v2:172.21.15.137:3300/0 conn(0x7ff7040cc330 0x7ff7040b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff703fff640 1 -- 172.21.15.27:0/607310609 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7040b5a30 msgr2=0x7ff7040d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff703fff640 1 --2- 172.21.15.27:0/607310609 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7040b5a30 0x7ff7040d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff703fff640 1 -- 172.21.15.27:0/607310609 >> v2:172.21.15.137:3300/0 conn(0x7ff7040cc330 msgr2=0x7ff7040b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:57:49.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff703fff640 1 --2- 172.21.15.27:0/607310609 >> v2:172.21.15.137:3300/0 conn(0x7ff7040cc330 0x7ff7040b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.084+0000 7ff703fff640 1 -- 172.21.15.27:0/607310609 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70415bf80 con 0x7ff704155950 2024-04-20T10:57:49.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.085+0000 7ff703fff640 1 --2- 172.21.15.27:0/607310609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff704155950 0x7ff7040cbdf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff7040b44e0 tx=0x7ff6f402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b5a9d3706d0adb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.085+0000 7ff702ffd640 1 -- 172.21.15.27:0/607310609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4034070 con 0x7ff704155950 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.085+0000 7ff702ffd640 1 -- 172.21.15.27:0/607310609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f4030860 con 0x7ff704155950 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.085+0000 7ff702ffd640 1 -- 172.21.15.27:0/607310609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4030c30 con 0x7ff704155950 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.086+0000 7ff70a952640 1 -- 172.21.15.27:0/607310609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff704155950 msgr2=0x7ff7040cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.086+0000 7ff70a952640 1 --2- 172.21.15.27:0/607310609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff704155950 0x7ff7040cbdf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff7040b44e0 tx=0x7ff6f402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.086+0000 7ff70a952640 1 -- 172.21.15.27:0/607310609 shutdown_connections 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.086+0000 7ff70a952640 1 --2- 172.21.15.27:0/607310609 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff7040b5a30 0x7ff7040d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.086+0000 7ff70a952640 1 --2- 172.21.15.27:0/607310609 >> v2:172.21.15.137:3300/0 conn(0x7ff7040cc330 0x7ff7040b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.086+0000 7ff70a952640 1 --2- 172.21.15.27:0/607310609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff704155950 0x7ff7040cbdf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.086+0000 7ff70a952640 1 -- 172.21.15.27:0/607310609 >> 172.21.15.27:0/607310609 conn(0x7ff7040676c0 msgr2=0x7ff704067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.086+0000 7ff70a952640 1 -- 172.21.15.27:0/607310609 shutdown_connections 2024-04-20T10:57:49.088 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.086+0000 7ff70a952640 1 -- 172.21.15.27:0/607310609 wait complete. 2024-04-20T10:57:49.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.087+0000 7ff70a952640 1 Processor -- start 2024-04-20T10:57:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff70a952640 1 -- start start 2024-04-20T10:57:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff70a952640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7040b5a30 0x7ff7041f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff70a952640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7040cc330 0x7ff70406d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff70a952640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff704155950 0x7ff70414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff70a952640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff70415fb90 con 0x7ff7040b5a30 2024-04-20T10:57:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff70a952640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff70415c540 con 0x7ff7040cc330 2024-04-20T10:57:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff70a952640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff70415fd10 con 0x7ff704155950 2024-04-20T10:57:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff703fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7040b5a30 0x7ff7041f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:49.091 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff703fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7040b5a30 0x7ff7041f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37534/0 (socket says 172.21.15.27:37534) 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff703fff640 1 -- 172.21.15.27:0/3977730089 learned_addr learned my addr 172.21.15.27:0/3977730089 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff7037fe640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7040cc330 0x7ff70406d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.088+0000 7ff708eef640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff704155950 0x7ff70414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff703fff640 1 -- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff704155950 msgr2=0x7ff70414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff703fff640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff704155950 0x7ff70414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff703fff640 1 -- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7040cc330 msgr2=0x7ff70406d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff703fff640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7040cc330 0x7ff70406d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff703fff640 1 -- 172.21.15.27:0/3977730089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7041e7960 con 0x7ff7040b5a30 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff703fff640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7040b5a30 0x7ff7041f2360 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff6f402b9e0 tx=0x7ff6f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff700ff9640 1 -- 172.21.15.27:0/3977730089 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4034040 con 0x7ff7040b5a30 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff700ff9640 1 -- 172.21.15.27:0/3977730089 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f4039d90 con 0x7ff7040b5a30 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff704150900 con 0x7ff7040b5a30 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff700ff9640 1 -- 172.21.15.27:0/3977730089 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40433f0 con 0x7ff7040b5a30 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.089+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff704150ec0 con 0x7ff7040b5a30 2024-04-20T10:57:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.090+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff70404fcf0 con 0x7ff7040b5a30 2024-04-20T10:57:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.091+0000 7ff700ff9640 1 -- 172.21.15.27:0/3977730089 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6f4037070 con 0x7ff7040b5a30 2024-04-20T10:57:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.091+0000 7ff700ff9640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff6e406c8f0 0x7ff6e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:57:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.091+0000 7ff700ff9640 1 -- 172.21.15.27:0/3977730089 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7ff6f40adb30 con 0x7ff7040b5a30 2024-04-20T10:57:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.091+0000 7ff7037fe640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff6e406c8f0 0x7ff6e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:57:49.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.092+0000 7ff7037fe640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff6e406c8f0 0x7ff6e406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff7040b3620 tx=0x7ff6ec009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:57:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.098+0000 7ff700ff9640 1 -- 172.21.15.27:0/3977730089 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff6f403e070 con 0x7ff7040b5a30 2024-04-20T10:57:49.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.322+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7040d0220 con 0x7ff7040b5a30 2024-04-20T10:57:49.323 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.323+0000 7ff700ff9640 1 -- 172.21.15.27:0/3977730089 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7ff6f407b8d0 con 0x7ff7040b5a30 2024-04-20T10:57:49.324 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:57:49.324 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:57:49.324 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:57:49.325 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.324+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff6e406c8f0 msgr2=0x7ff6e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff6e406c8f0 0x7ff6e406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff7040b3620 tx=0x7ff6ec009190 comp rx=0 tx=0).stop 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7040b5a30 msgr2=0x7ff7041f2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7040b5a30 0x7ff7041f2360 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff6f402b9e0 tx=0x7ff6f40046c0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 shutdown_connections 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff6e406c8f0 0x7ff6e406edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff704155950 0x7ff70414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7040cc330 0x7ff70406d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 --2- 172.21.15.27:0/3977730089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7040b5a30 0x7ff7041f2360 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 >> 172.21.15.27:0/3977730089 conn(0x7ff7040676c0 msgr2=0x7ff7040d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:57:49.326 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 shutdown_connections 2024-04-20T10:57:49.327 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:57:49.325+0000 7ff70a952640 1 -- 172.21.15.27:0/3977730089 wait complete. 2024-04-20T10:57:49.336 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2024-04-20T10:57:49.336 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 15 secs before thrashing 2024-04-20T10:57:50.573 INFO:tasks.workunit.client.0.smithi027.stderr:Updating files: 92% (11402/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by switching back to a branch. 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr: git switch -c 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr:Or undo this operation with: 2024-04-20T10:57:51.305 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-20T10:57:51.306 INFO:tasks.workunit.client.0.smithi027.stderr: git switch - 2024-04-20T10:57:51.306 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-20T10:57:51.306 INFO:tasks.workunit.client.0.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-20T10:57:51.306 INFO:tasks.workunit.client.0.smithi027.stderr: 2024-04-20T10:57:51.306 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-20T10:57:51.311 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-20T10:57:51.331 INFO:tasks.ceph.mds.d.smithi137.stderr:2024-04-20T10:57:51.331+0000 7f8f0d139640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-20T10:57:51.341 INFO:tasks.workunit.client.0.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-20T10:57:51.343 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T10:57:51.344 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-20T10:57:51.846 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-20T10:57:51.901 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-20T10:57:51.942 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T10:57:51.944 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T10:57:51.944 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-20T10:57:51.983 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T10:57:51.987 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T10:57:51.987 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-20T10:57:52.044 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-20T10:57:52.045 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-20T10:57:52.045 DEBUG:teuthology.orchestra.run.smithi027:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-20T10:57:52.105 INFO:tasks.workunit.client.0.smithi027.stderr:+ mkdir -p fsstress 2024-04-20T10:57:52.106 INFO:tasks.workunit.client.0.smithi027.stderr:+ pushd fsstress 2024-04-20T10:57:52.107 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-20T10:57:52.107 INFO:tasks.workunit.client.0.smithi027.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-20T10:57:52.616 INFO:tasks.workunit.client.0.smithi027.stderr:+ tar xzf ltp-full.tgz 2024-04-20T10:58:04.338 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.453+0000 7f146274a640 1 Processor -- start 2024-04-20T10:58:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.453+0000 7f146274a640 1 -- start start 2024-04-20T10:58:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f146274a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f145c155690 0x7f145c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f146274a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f145c06d770 0x7f145c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f146274a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f145c0cf580 con 0x7f145c0cf080 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f146274a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f145c06ca40 con 0x7f145c155690 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f146274a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f145c0ca2b0 con 0x7f145c06d770 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f145bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f145c155690 0x7f145c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f1460ce7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f145c0cf080 0x7f145c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50726/0 (socket says 172.21.15.27:50726) 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f1460ce7640 1 -- 172.21.15.27:0/895515105 learned_addr learned my addr 172.21.15.27:0/895515105 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f145b7fe640 1 --2- 172.21.15.27:0/895515105 >> v2:172.21.15.137:3300/0 conn(0x7f145c06d770 0x7f145c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f145affd640 1 -- 172.21.15.27:0/895515105 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003523639 0 0) 0x7f145c0cf580 con 0x7f145c0cf080 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f145affd640 1 -- 172.21.15.27:0/895515105 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1444003520 con 0x7f145c0cf080 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f145bfff640 1 -- 172.21.15.27:0/895515105 >> v2:172.21.15.137:3300/0 conn(0x7f145c06d770 msgr2=0x7f145c0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f145bfff640 1 --2- 172.21.15.27:0/895515105 >> v2:172.21.15.137:3300/0 conn(0x7f145c06d770 0x7f145c0ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f145bfff640 1 -- 172.21.15.27:0/895515105 >> v1:172.21.15.27:6789/0 conn(0x7f145c0cf080 legacy=0x7f145c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.454+0000 7f145bfff640 1 -- 172.21.15.27:0/895515105 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c0cf860 con 0x7f145c155690 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.455+0000 7f145bfff640 1 --2- 172.21.15.27:0/895515105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c155690 0x7f145c15b3f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f145c0cb610 tx=0x7f144c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a11cf38066c1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.455+0000 7f145affd640 1 -- 172.21.15.27:0/895515105 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c035020 con 0x7f145c155690 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.455+0000 7f145affd640 1 -- 172.21.15.27:0/895515105 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144c031830 con 0x7f145c155690 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.455+0000 7f145affd640 1 -- 172.21.15.27:0/895515105 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c031be0 con 0x7f145c155690 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.456+0000 7f146274a640 1 -- 172.21.15.27:0/895515105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c155690 msgr2=0x7f145c15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.456+0000 7f146274a640 1 --2- 172.21.15.27:0/895515105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c155690 0x7f145c15b3f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f145c0cb610 tx=0x7f144c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.456+0000 7f146274a640 1 -- 172.21.15.27:0/895515105 shutdown_connections 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.456+0000 7f146274a640 1 --2- 172.21.15.27:0/895515105 >> v2:172.21.15.137:3300/0 conn(0x7f145c06d770 0x7f145c0ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.456+0000 7f146274a640 1 --2- 172.21.15.27:0/895515105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c155690 0x7f145c15b3f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.456+0000 7f146274a640 1 -- 172.21.15.27:0/895515105 >> 172.21.15.27:0/895515105 conn(0x7f145c06c070 msgr2=0x7f145c0676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.456+0000 7f146274a640 1 -- 172.21.15.27:0/895515105 shutdown_connections 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.456+0000 7f146274a640 1 -- 172.21.15.27:0/895515105 wait complete. 2024-04-20T10:58:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.457+0000 7f146274a640 1 Processor -- start 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.457+0000 7f146274a640 1 -- start start 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f146274a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f145c06d770 0x7f145c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f146274a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c0cf080 0x7f145c1f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f146274a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f145c155690 0x7f145c1e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f146274a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f145c06d250 con 0x7f145c0cf080 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f146274a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f145c06cf70 con 0x7f145c06d770 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f146274a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f145c155a70 con 0x7f145c155690 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f145b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c0cf080 0x7f145c1f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f145b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c0cf080 0x7f145c1f1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56676/0 (socket says 172.21.15.27:56676) 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f145b7fe640 1 -- 172.21.15.27:0/1858796354 learned_addr learned my addr 172.21.15.27:0/1858796354 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f145bfff640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f145c06d770 0x7f145c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f1460ce7640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f145c155690 0x7f145c1e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f145bfff640 1 -- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f145c155690 msgr2=0x7f145c1e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f145bfff640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f145c155690 0x7f145c1e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f145bfff640 1 -- 172.21.15.27:0/1858796354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c0cf080 msgr2=0x7f145c1f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.458+0000 7f145bfff640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c0cf080 0x7f145c1f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.459+0000 7f145bfff640 1 -- 172.21.15.27:0/1858796354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c1f5490 con 0x7f145c06d770 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.459+0000 7f145b7fe640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c0cf080 0x7f145c1f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.459+0000 7f145bfff640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f145c06d770 0x7f145c158370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f144c033000 tx=0x7f144c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.459+0000 7f1458ff9640 1 -- 172.21.15.27:0/1858796354 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c040070 con 0x7f145c06d770 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.459+0000 7f1458ff9640 1 -- 172.21.15.27:0/1858796354 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144c03a8f0 con 0x7f145c06d770 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.459+0000 7f1458ff9640 1 -- 172.21.15.27:0/1858796354 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c035040 con 0x7f145c06d770 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.459+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c1f11c0 con 0x7f145c06d770 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.459+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c1f51b0 con 0x7f145c06d770 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.460+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145c04fcf0 con 0x7f145c06d770 2024-04-20T10:58:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.460+0000 7f1458ff9640 1 -- 172.21.15.27:0/1858796354 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f144c038020 con 0x7f145c06d770 2024-04-20T10:58:04.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.461+0000 7f1458ff9640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f144806c8f0 0x7f144806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.461+0000 7f1458ff9640 1 -- 172.21.15.27:0/1858796354 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f144c0acda0 con 0x7f145c06d770 2024-04-20T10:58:04.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.461+0000 7f145b7fe640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f144806c8f0 0x7f144806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.462+0000 7f145b7fe640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f144806c8f0 0x7f144806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f145c1e85f0 tx=0x7f1450009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:04.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.466+0000 7f1458ff9640 1 -- 172.21.15.27:0/1858796354 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f144c07ab40 con 0x7f145c06d770 2024-04-20T10:58:04.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.689+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f145c0c5b00 con 0x7f145c06d770 2024-04-20T10:58:04.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.690+0000 7f1458ff9640 1 -- 172.21.15.27:0/1858796354 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7f144c07ee40 con 0x7f145c06d770 2024-04-20T10:58:04.690 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:04.690 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:58:04.690 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.691+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f144806c8f0 msgr2=0x7f144806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.691+0000 7f146274a640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f144806c8f0 0x7f144806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f145c1e85f0 tx=0x7f1450009190 comp rx=0 tx=0).stop 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.691+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f145c06d770 msgr2=0x7f145c158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.691+0000 7f146274a640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f145c06d770 0x7f145c158370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f144c033000 tx=0x7f144c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.692+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 shutdown_connections 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.692+0000 7f146274a640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f144806c8f0 0x7f144806edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.692+0000 7f146274a640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f145c155690 0x7f145c1e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.692+0000 7f146274a640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f145c0cf080 0x7f145c1f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.692+0000 7f146274a640 1 --2- 172.21.15.27:0/1858796354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f145c06d770 0x7f145c158370 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.692+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 >> 172.21.15.27:0/1858796354 conn(0x7f145c06c070 msgr2=0x7f145c0ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.692+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 shutdown_connections 2024-04-20T10:58:04.692 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.692+0000 7f146274a640 1 -- 172.21.15.27:0/1858796354 wait complete. 2024-04-20T10:58:04.702 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=4) 2024-04-20T10:58:04.703 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-20T10:58:04.703 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:58:04.712 INFO:tasks.ceph.mds.f:Stopped 2024-04-20T10:58:04.713 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:04.817 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.818+0000 7f0e78a5c640 1 Processor -- start 2024-04-20T10:58:04.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.818+0000 7f0e78a5c640 1 -- start start 2024-04-20T10:58:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e78a5c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f0e74155950 0x7f0e740cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e78a5c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e7415b9c0 con 0x7f0e740b5c00 2024-04-20T10:58:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e78a5c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e7415bca0 con 0x7f0e740cc330 2024-04-20T10:58:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e78a5c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f0e7415bf80 con 0x7f0e74155950 2024-04-20T10:58:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e72d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0e740b5c00 0x7f0e740d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45236/0 (socket says 172.21.15.27:45236) 2024-04-20T10:58:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e72d9d640 1 -- 172.21.15.27:0/2142362163 learned_addr learned my addr 172.21.15.27:0/2142362163 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e7259c640 1 --2- 172.21.15.27:0/2142362163 >> v2:172.21.15.137:3300/0 conn(0x7f0e74155950 0x7f0e740cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e7159a640 1 -- 172.21.15.27:0/2142362163 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328152435 0 0) 0x7f0e7415b9c0 con 0x7f0e740b5c00 2024-04-20T10:58:04.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e7159a640 1 -- 172.21.15.27:0/2142362163 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e5c003520 con 0x7f0e740b5c00 2024-04-20T10:58:04.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e7159a640 1 -- 172.21.15.27:0/2142362163 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2957366761 0 0) 0x7f0e7415bca0 con 0x7f0e740cc330 2024-04-20T10:58:04.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e7159a640 1 -- 172.21.15.27:0/2142362163 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e7415b9c0 con 0x7f0e740cc330 2024-04-20T10:58:04.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e7259c640 1 -- 172.21.15.27:0/2142362163 >> v1:172.21.15.137:6789/0 conn(0x7f0e740cc330 legacy=0x7f0e740b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e7259c640 1 -- 172.21.15.27:0/2142362163 >> v1:172.21.15.27:6789/0 conn(0x7f0e740b5c00 legacy=0x7f0e740d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.819+0000 7f0e7259c640 1 -- 172.21.15.27:0/2142362163 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7415c260 con 0x7f0e74155950 2024-04-20T10:58:04.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.820+0000 7f0e7259c640 1 --2- 172.21.15.27:0/2142362163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e74155950 0x7f0e740cbdf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e640097a0 tx=0x7f0e6402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8256e6bfa5350318 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:04.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.820+0000 7f0e7159a640 1 -- 172.21.15.27:0/2142362163 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6402f810 con 0x7f0e74155950 2024-04-20T10:58:04.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.820+0000 7f0e7159a640 1 -- 172.21.15.27:0/2142362163 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6402f9b0 con 0x7f0e74155950 2024-04-20T10:58:04.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.820+0000 7f0e7159a640 1 -- 172.21.15.27:0/2142362163 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6402fd80 con 0x7f0e74155950 2024-04-20T10:58:04.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.821+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2142362163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e74155950 msgr2=0x7f0e740cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.821+0000 7f0e78a5c640 1 --2- 172.21.15.27:0/2142362163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e74155950 0x7f0e740cbdf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e640097a0 tx=0x7f0e6402b9f0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.821+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2142362163 shutdown_connections 2024-04-20T10:58:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.821+0000 7f0e78a5c640 1 --2- 172.21.15.27:0/2142362163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e74155950 0x7f0e740cbdf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.821+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2142362163 >> 172.21.15.27:0/2142362163 conn(0x7f0e740676c0 msgr2=0x7f0e74067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.821+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2142362163 shutdown_connections 2024-04-20T10:58:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.821+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2142362163 wait complete. 2024-04-20T10:58:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.822+0000 7f0e78a5c640 1 Processor -- start 2024-04-20T10:58:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e78a5c640 1 -- start start 2024-04-20T10:58:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e78a5c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0e740b5c00 0x7f0e741f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e78a5c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e740cc330 0x7f0e740c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e78a5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e74155950 0x7f0e741e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e78a5c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7415fd10 con 0x7f0e740cc330 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e78a5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7415fb90 con 0x7f0e74155950 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e78a5c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f0e74155d30 con 0x7f0e740b5c00 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e71d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e740cc330 0x7f0e740c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e71d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e740cc330 0x7f0e740c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35392/0 (socket says 172.21.15.27:35392) 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e71d9b640 1 -- 172.21.15.27:0/2406050889 learned_addr learned my addr 172.21.15.27:0/2406050889 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e7259c640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0e740b5c00 0x7f0e741f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.823+0000 7f0e72d9d640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e74155950 0x7f0e741e64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e71d9b640 1 -- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0e740b5c00 msgr2=0x7f0e741f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e71d9b640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0e740b5c00 0x7f0e741f5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e71d9b640 1 -- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e74155950 msgr2=0x7f0e741e64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e71d9b640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e74155950 0x7f0e741e64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e71d9b640 1 -- 172.21.15.27:0/2406050889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e741f1690 con 0x7f0e740cc330 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e71d9b640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e740cc330 0x7f0e740c60b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c430 tx=0x7f0e6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e62ffd640 1 -- 172.21.15.27:0/2406050889 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68017070 con 0x7f0e740cc330 2024-04-20T10:58:04.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e62ffd640 1 -- 172.21.15.27:0/2406050889 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e680052f0 con 0x7f0e740cc330 2024-04-20T10:58:04.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e741f1930 con 0x7f0e740cc330 2024-04-20T10:58:04.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e62ffd640 1 -- 172.21.15.27:0/2406050889 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68003830 con 0x7f0e740cc330 2024-04-20T10:58:04.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.824+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e741f1ef0 con 0x7f0e740cc330 2024-04-20T10:58:04.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.825+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7404fcf0 con 0x7f0e740cc330 2024-04-20T10:58:04.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.826+0000 7f0e62ffd640 1 -- 172.21.15.27:0/2406050889 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e6801a070 con 0x7f0e740cc330 2024-04-20T10:58:04.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.826+0000 7f0e62ffd640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0e5406c920 0x7f0e5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:04.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.826+0000 7f0e62ffd640 1 -- 172.21.15.27:0/2406050889 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f0e6808af80 con 0x7f0e740cc330 2024-04-20T10:58:04.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.826+0000 7f0e7259c640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0e5406c920 0x7f0e5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:04.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.827+0000 7f0e7259c640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0e5406c920 0x7f0e5406ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e64036000 tx=0x7f0e64008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:04.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:04.830+0000 7f0e62ffd640 1 -- 172.21.15.27:0/2406050889 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0e68058c10 con 0x7f0e740cc330 2024-04-20T10:58:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.060+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e740cd0a0 con 0x7f0e740cc330 2024-04-20T10:58:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.061+0000 7f0e62ffd640 1 -- 172.21.15.27:0/2406050889 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7f0e6805cf10 con 0x7f0e740cc330 2024-04-20T10:58:05.060 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:05.060 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:58:05.061 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:58:05.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0e5406c920 msgr2=0x7f0e5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0e5406c920 0x7f0e5406ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e64036000 tx=0x7f0e64008030 comp rx=0 tx=0).stop 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e740cc330 msgr2=0x7f0e740c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e740cc330 0x7f0e740c60b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c430 tx=0x7f0e6800c8f0 comp rx=0 tx=0).stop 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 shutdown_connections 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0e5406c920 0x7f0e5406ede0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e74155950 0x7f0e741e64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e740cc330 0x7f0e740c60b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 --2- 172.21.15.27:0/2406050889 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0e740b5c00 0x7f0e741f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 >> 172.21.15.27:0/2406050889 conn(0x7f0e740676c0 msgr2=0x7f0e74151610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 shutdown_connections 2024-04-20T10:58:05.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:05.063+0000 7f0e78a5c640 1 -- 172.21.15.27:0/2406050889 wait complete. 2024-04-20T10:58:05.073 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T10:58:07.074 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:07.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.190+0000 7fcb14e74640 1 Processor -- start 2024-04-20T10:58:07.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.190+0000 7fcb14e74640 1 -- start start 2024-04-20T10:58:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb14e74640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcb1015f230 0x7fcb1015c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb14e74640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcb100ccbd0 0x7fcb10155410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb14e74640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb1015f9a0 con 0x7fcb1015ee50 2024-04-20T10:58:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb14e74640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fcb100cb870 con 0x7fcb100ccbd0 2024-04-20T10:58:07.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb14e74640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fcb100cbb30 con 0x7fcb1015f230 2024-04-20T10:58:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb0ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcb100ccbd0 0x7fcb10155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb0ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcb100ccbd0 0x7fcb10155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35402/0 (socket says 172.21.15.27:35402) 2024-04-20T10:58:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb0ed9d640 1 -- 172.21.15.27:0/119061755 learned_addr learned my addr 172.21.15.27:0/119061755 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb0dd9b640 1 --2- 172.21.15.27:0/119061755 >> v2:172.21.15.137:3300/0 conn(0x7fcb1015f230 0x7fcb1015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb0ed9d640 1 -- 172.21.15.27:0/119061755 >> v2:172.21.15.137:3300/0 conn(0x7fcb1015f230 msgr2=0x7fcb1015c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb0ed9d640 1 --2- 172.21.15.27:0/119061755 >> v2:172.21.15.137:3300/0 conn(0x7fcb1015f230 0x7fcb1015c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb0ed9d640 1 -- 172.21.15.27:0/119061755 >> v1:172.21.15.137:6789/0 conn(0x7fcb1015ee50 legacy=0x7fcb1006cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.191+0000 7fcb0ed9d640 1 -- 172.21.15.27:0/119061755 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb100cbe10 con 0x7fcb100ccbd0 2024-04-20T10:58:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.192+0000 7fcb0ed9d640 1 --2- 172.21.15.27:0/119061755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 0x7fcb10155410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb101592a0 tx=0x7fcb0802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff8fed5c862e6f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.192+0000 7fcb0d59a640 1 -- 172.21.15.27:0/119061755 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0802cbb0 con 0x7fcb100ccbd0 2024-04-20T10:58:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.192+0000 7fcb0d59a640 1 -- 172.21.15.27:0/119061755 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0802cd50 con 0x7fcb100ccbd0 2024-04-20T10:58:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.192+0000 7fcb0d59a640 1 -- 172.21.15.27:0/119061755 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08034610 con 0x7fcb100ccbd0 2024-04-20T10:58:07.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.193+0000 7fcb14e74640 1 -- 172.21.15.27:0/119061755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 msgr2=0x7fcb10155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.193+0000 7fcb14e74640 1 --2- 172.21.15.27:0/119061755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 0x7fcb10155410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcb101592a0 tx=0x7fcb0802c010 comp rx=0 tx=0).stop 2024-04-20T10:58:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.193+0000 7fcb14e74640 1 -- 172.21.15.27:0/119061755 shutdown_connections 2024-04-20T10:58:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.193+0000 7fcb14e74640 1 --2- 172.21.15.27:0/119061755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 0x7fcb10155410 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.193+0000 7fcb14e74640 1 --2- 172.21.15.27:0/119061755 >> v2:172.21.15.137:3300/0 conn(0x7fcb1015f230 0x7fcb1015c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.193+0000 7fcb14e74640 1 -- 172.21.15.27:0/119061755 >> 172.21.15.27:0/119061755 conn(0x7fcb1006e010 msgr2=0x7fcb100d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.193+0000 7fcb14e74640 1 -- 172.21.15.27:0/119061755 shutdown_connections 2024-04-20T10:58:07.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.193+0000 7fcb14e74640 1 -- 172.21.15.27:0/119061755 wait complete. 2024-04-20T10:58:07.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb14e74640 1 Processor -- start 2024-04-20T10:58:07.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb14e74640 1 -- start start 2024-04-20T10:58:07.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb14e74640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 0x7fcb100c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:07.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb14e74640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb1015ee50 0x7fcb100ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:07.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb14e74640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb1015f230 0x7fcb100c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:07.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb14e74640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcb100cc6b0 con 0x7fcb100ccbd0 2024-04-20T10:58:07.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb14e74640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcb100cc3d0 con 0x7fcb1015ee50 2024-04-20T10:58:07.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb14e74640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1015cae0 con 0x7fcb1015f230 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb0e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 0x7fcb100c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb0e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 0x7fcb100c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35412/0 (socket says 172.21.15.27:35412) 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb0e59c640 1 -- 172.21.15.27:0/2034597333 learned_addr learned my addr 172.21.15.27:0/2034597333 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.195+0000 7fcb0dd9b640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb1015ee50 0x7fcb100ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb0ed9d640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb1015f230 0x7fcb100c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb0e59c640 1 -- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb1015f230 msgr2=0x7fcb100c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb0e59c640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb1015f230 0x7fcb100c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb0e59c640 1 -- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb1015ee50 msgr2=0x7fcb100ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb0e59c640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb1015ee50 0x7fcb100ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb0e59c640 1 -- 172.21.15.27:0/2034597333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb100c5c80 con 0x7fcb100ccbd0 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb0ed9d640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb1015f230 0x7fcb100c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb0e59c640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 0x7fcb100c75c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000c3f0 tx=0x7fcb0000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcafeffd640 1 -- 172.21.15.27:0/2034597333 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb000052d0 con 0x7fcb100ccbd0 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcafeffd640 1 -- 172.21.15.27:0/2034597333 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb00005490 con 0x7fcb100ccbd0 2024-04-20T10:58:07.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb100c5360 con 0x7fcb100ccbd0 2024-04-20T10:58:07.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcafeffd640 1 -- 172.21.15.27:0/2034597333 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00007260 con 0x7fcb100ccbd0 2024-04-20T10:58:07.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.196+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb100c6230 con 0x7fcb100ccbd0 2024-04-20T10:58:07.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.197+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb100688e0 con 0x7fcb100ccbd0 2024-04-20T10:58:07.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.198+0000 7fcafeffd640 1 -- 172.21.15.27:0/2034597333 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb00007400 con 0x7fcb100ccbd0 2024-04-20T10:58:07.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.198+0000 7fcafeffd640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcaf006c8f0 0x7fcaf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:07.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.198+0000 7fcafeffd640 1 -- 172.21.15.27:0/2034597333 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fcb0008b030 con 0x7fcb100ccbd0 2024-04-20T10:58:07.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.198+0000 7fcb0dd9b640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcaf006c8f0 0x7fcaf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:07.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.199+0000 7fcb0dd9b640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcaf006c8f0 0x7fcaf006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb100cb2b0 tx=0x7fcb04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.203+0000 7fcafeffd640 1 -- 172.21.15.27:0/2034597333 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcb1015f9a0 con 0x7fcb100ccbd0 2024-04-20T10:58:07.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.432+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb1004fcf0 con 0x7fcb100ccbd0 2024-04-20T10:58:07.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.432+0000 7fcafeffd640 1 -- 172.21.15.27:0/2034597333 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7fcb00058d40 con 0x7fcb100ccbd0 2024-04-20T10:58:07.432 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:07.432 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:58:07.433 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:58:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcaf006c8f0 msgr2=0x7fcaf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcaf006c8f0 0x7fcaf006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcb100cb2b0 tx=0x7fcb04008040 comp rx=0 tx=0).stop 2024-04-20T10:58:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 msgr2=0x7fcb100c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 0x7fcb100c75c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000c3f0 tx=0x7fcb0000c8b0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 shutdown_connections 2024-04-20T10:58:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcaf006c8f0 0x7fcaf006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb1015f230 0x7fcb100c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb1015ee50 0x7fcb100ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 --2- 172.21.15.27:0/2034597333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb100ccbd0 0x7fcb100c75c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 >> 172.21.15.27:0/2034597333 conn(0x7fcb1006e010 msgr2=0x7fcb10154730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 shutdown_connections 2024-04-20T10:58:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:07.434+0000 7fcb14e74640 1 -- 172.21.15.27:0/2034597333 wait complete. 2024-04-20T10:58:07.445 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T10:58:09.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:09.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.561+0000 7f566e3f4640 1 Processor -- start 2024-04-20T10:58:09.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.561+0000 7f566e3f4640 1 -- start start 2024-04-20T10:58:09.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f566e3f4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5668151f10 0x7f56680cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:09.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f566e3f4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f56680cd9f0 0x7f56680c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:09.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f566e3f4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56681522f0 con 0x7f56680c8cf0 2024-04-20T10:58:09.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f566e3f4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5668157f00 con 0x7f5668151f10 2024-04-20T10:58:09.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f566e3f4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f56681581e0 con 0x7f56680cd9f0 2024-04-20T10:58:09.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f5667fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5668151f10 0x7f56680cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:09.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f566c991640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f56680c8cf0 0x7f566815b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45248/0 (socket says 172.21.15.27:45248) 2024-04-20T10:58:09.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f566c991640 1 -- 172.21.15.27:0/3944228257 learned_addr learned my addr 172.21.15.27:0/3944228257 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:09.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f56677fe640 1 --2- 172.21.15.27:0/3944228257 >> v2:172.21.15.137:3300/0 conn(0x7f56680cd9f0 0x7f56680c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:09.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f5667fff640 1 -- 172.21.15.27:0/3944228257 >> v2:172.21.15.137:3300/0 conn(0x7f56680cd9f0 msgr2=0x7f56680c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:09.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f5667fff640 1 --2- 172.21.15.27:0/3944228257 >> v2:172.21.15.137:3300/0 conn(0x7f56680cd9f0 0x7f56680c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f5667fff640 1 -- 172.21.15.27:0/3944228257 >> v1:172.21.15.27:6789/0 conn(0x7f56680c8cf0 legacy=0x7f566815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:09.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.562+0000 7f5667fff640 1 -- 172.21.15.27:0/3944228257 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56681584c0 con 0x7f5668151f10 2024-04-20T10:58:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.563+0000 7f5667fff640 1 --2- 172.21.15.27:0/3944228257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5668151f10 0x7f56680cd4b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56680cb0c0 tx=0x7f565002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83749374a75fed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.563+0000 7f5666ffd640 1 -- 172.21.15.27:0/3944228257 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650034070 con 0x7f5668151f10 2024-04-20T10:58:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.563+0000 7f5666ffd640 1 -- 172.21.15.27:0/3944228257 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5650030610 con 0x7f5668151f10 2024-04-20T10:58:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.563+0000 7f5666ffd640 1 -- 172.21.15.27:0/3944228257 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500309e0 con 0x7f5668151f10 2024-04-20T10:58:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.564+0000 7f566e3f4640 1 -- 172.21.15.27:0/3944228257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5668151f10 msgr2=0x7f56680cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.564+0000 7f566e3f4640 1 --2- 172.21.15.27:0/3944228257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5668151f10 0x7f56680cd4b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56680cb0c0 tx=0x7f565002b960 comp rx=0 tx=0).stop 2024-04-20T10:58:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.564+0000 7f566e3f4640 1 -- 172.21.15.27:0/3944228257 shutdown_connections 2024-04-20T10:58:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.564+0000 7f566e3f4640 1 --2- 172.21.15.27:0/3944228257 >> v2:172.21.15.137:3300/0 conn(0x7f56680cd9f0 0x7f56680c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.564+0000 7f566e3f4640 1 --2- 172.21.15.27:0/3944228257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5668151f10 0x7f56680cd4b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.564+0000 7f566e3f4640 1 -- 172.21.15.27:0/3944228257 >> 172.21.15.27:0/3944228257 conn(0x7f566806d5b0 msgr2=0x7f566806d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.564+0000 7f566e3f4640 1 -- 172.21.15.27:0/3944228257 shutdown_connections 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.564+0000 7f566e3f4640 1 -- 172.21.15.27:0/3944228257 wait complete. 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.565+0000 7f566e3f4640 1 Processor -- start 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.565+0000 7f566e3f4640 1 -- start start 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f566e3f4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56680c8cf0 0x7f5668164a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f566e3f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56680cd9f0 0x7f5668154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f566e3f4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5668151f10 0x7f56681473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f566e3f4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f566815b1c0 con 0x7f56680c8cf0 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f566e3f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5668158a80 con 0x7f56680cd9f0 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f566e3f4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5668157bd0 con 0x7f5668151f10 2024-04-20T10:58:09.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f5667fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56680c8cf0 0x7f5668164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f5667fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56680c8cf0 0x7f5668164a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35438/0 (socket says 172.21.15.27:35438) 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f5667fff640 1 -- 172.21.15.27:0/1924997359 learned_addr learned my addr 172.21.15.27:0/1924997359 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f56677fe640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56680cd9f0 0x7f5668154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f566c991640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5668151f10 0x7f56681473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f5667fff640 1 -- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5668151f10 msgr2=0x7f56681473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f5667fff640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5668151f10 0x7f56681473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f5667fff640 1 -- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56680cd9f0 msgr2=0x7f5668154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f5667fff640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56680cd9f0 0x7f5668154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f5667fff640 1 -- 172.21.15.27:0/1924997359 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56681f1e50 con 0x7f56680c8cf0 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.566+0000 7f56677fe640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56680cd9f0 0x7f5668154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.567+0000 7f566c991640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5668151f10 0x7f56681473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.567+0000 7f5667fff640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56680c8cf0 0x7f5668164a30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f565002bf70 tx=0x7f56500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.567+0000 7f5664ff9640 1 -- 172.21.15.27:0/1924997359 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650034040 con 0x7f56680c8cf0 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.567+0000 7f5664ff9640 1 -- 172.21.15.27:0/1924997359 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f565003acc0 con 0x7f56680c8cf0 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.567+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56681f5e60 con 0x7f56680c8cf0 2024-04-20T10:58:09.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.567+0000 7f5664ff9640 1 -- 172.21.15.27:0/1924997359 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650044640 con 0x7f56680c8cf0 2024-04-20T10:58:09.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.567+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56681f23d0 con 0x7f56680c8cf0 2024-04-20T10:58:09.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.568+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56680cbae0 con 0x7f56680c8cf0 2024-04-20T10:58:09.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.568+0000 7f5664ff9640 1 -- 172.21.15.27:0/1924997359 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5650037070 con 0x7f56680c8cf0 2024-04-20T10:58:09.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.569+0000 7f5664ff9640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f564806c8f0 0x7f564806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:09.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.569+0000 7f5664ff9640 1 -- 172.21.15.27:0/1924997359 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f56500adb00 con 0x7f56680c8cf0 2024-04-20T10:58:09.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.569+0000 7f56677fe640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f564806c8f0 0x7f564806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:09.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.570+0000 7f56677fe640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f564806c8f0 0x7f564806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f56681649c0 tx=0x7f5658008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:09.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.573+0000 7f5664ff9640 1 -- 172.21.15.27:0/1924997359 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f56680cbae0 con 0x7f56680c8cf0 2024-04-20T10:58:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.800+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5668155810 con 0x7f56680c8cf0 2024-04-20T10:58:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.800+0000 7f5664ff9640 1 -- 172.21.15.27:0/1924997359 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7f5668155810 con 0x7f56680c8cf0 2024-04-20T10:58:09.800 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:09.800 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:58:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f564806c8f0 msgr2=0x7f564806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f564806c8f0 0x7f564806edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f56681649c0 tx=0x7f5658008040 comp rx=0 tx=0).stop 2024-04-20T10:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56680c8cf0 msgr2=0x7f5668164a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56680c8cf0 0x7f5668164a30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f565002bf70 tx=0x7f56500046c0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 shutdown_connections 2024-04-20T10:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f564806c8f0 0x7f564806edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5668151f10 0x7f56681473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56680cd9f0 0x7f5668154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 --2- 172.21.15.27:0/1924997359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56680c8cf0 0x7f5668164a30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:09.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 >> 172.21.15.27:0/1924997359 conn(0x7f566806d5b0 msgr2=0x7f566806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:09.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 shutdown_connections 2024-04-20T10:58:09.803 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:09.802+0000 7f566e3f4640 1 -- 172.21.15.27:0/1924997359 wait complete. 2024-04-20T10:58:09.813 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T10:58:11.814 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:11.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.928+0000 7f9f93a14640 1 Processor -- start 2024-04-20T10:58:11.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.928+0000 7f9f93a14640 1 -- start start 2024-04-20T10:58:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f93a14640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9f8c0b54f0 0x7f9f8c0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f93a14640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c0b5e70 con 0x7f9f8c0d0c90 2024-04-20T10:58:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f93a14640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c15b9c0 con 0x7f9f8c155950 2024-04-20T10:58:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f93a14640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c15bca0 con 0x7f9f8c0b54f0 2024-04-20T10:58:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f91fb1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9f8c0d0c90 0x7f9f8c0d10a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45256/0 (socket says 172.21.15.27:45256) 2024-04-20T10:58:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f91fb1640 1 -- 172.21.15.27:0/3492919417 learned_addr learned my addr 172.21.15.27:0/3492919417 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f90faf640 1 --2- 172.21.15.27:0/3492919417 >> v2:172.21.15.137:3300/0 conn(0x7f9f8c0b54f0 0x7f9f8c0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f83fff640 1 -- 172.21.15.27:0/3492919417 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1160858301 0 0) 0x7f9f8c0b5e70 con 0x7f9f8c0d0c90 2024-04-20T10:58:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f83fff640 1 -- 172.21.15.27:0/3492919417 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f78003520 con 0x7f9f8c0d0c90 2024-04-20T10:58:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f83fff640 1 -- 172.21.15.27:0/3492919417 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4224542018 0 0) 0x7f9f8c15b9c0 con 0x7f9f8c155950 2024-04-20T10:58:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f83fff640 1 -- 172.21.15.27:0/3492919417 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f8c0b5e70 con 0x7f9f8c155950 2024-04-20T10:58:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f90faf640 1 -- 172.21.15.27:0/3492919417 >> v1:172.21.15.137:6789/0 conn(0x7f9f8c155950 legacy=0x7f9f8c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f90faf640 1 -- 172.21.15.27:0/3492919417 >> v1:172.21.15.27:6789/0 conn(0x7f9f8c0d0c90 legacy=0x7f9f8c0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:11.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.929+0000 7f9f90faf640 1 -- 172.21.15.27:0/3492919417 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c15bf80 con 0x7f9f8c0b54f0 2024-04-20T10:58:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.930+0000 7f9f90faf640 1 --2- 172.21.15.27:0/3492919417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f8c0b54f0 0x7f9f8c0b5930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0097a0 tx=0x7f9f7c02b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfbb14dce7c494dd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.930+0000 7f9f83fff640 1 -- 172.21.15.27:0/3492919417 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c02f810 con 0x7f9f8c0b54f0 2024-04-20T10:58:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.930+0000 7f9f83fff640 1 -- 172.21.15.27:0/3492919417 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c02f9b0 con 0x7f9f8c0b54f0 2024-04-20T10:58:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.931+0000 7f9f83fff640 1 -- 172.21.15.27:0/3492919417 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c02fd80 con 0x7f9f8c0b54f0 2024-04-20T10:58:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.931+0000 7f9f93a14640 1 -- 172.21.15.27:0/3492919417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f8c0b54f0 msgr2=0x7f9f8c0b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.931+0000 7f9f93a14640 1 --2- 172.21.15.27:0/3492919417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f8c0b54f0 0x7f9f8c0b5930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0097a0 tx=0x7f9f7c02b9f0 comp rx=0 tx=0).stop 2024-04-20T10:58:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.931+0000 7f9f93a14640 1 -- 172.21.15.27:0/3492919417 shutdown_connections 2024-04-20T10:58:11.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.931+0000 7f9f93a14640 1 --2- 172.21.15.27:0/3492919417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f8c0b54f0 0x7f9f8c0b5930 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.931+0000 7f9f93a14640 1 -- 172.21.15.27:0/3492919417 >> 172.21.15.27:0/3492919417 conn(0x7f9f8c0676c0 msgr2=0x7f9f8c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.931+0000 7f9f93a14640 1 -- 172.21.15.27:0/3492919417 shutdown_connections 2024-04-20T10:58:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.931+0000 7f9f93a14640 1 -- 172.21.15.27:0/3492919417 wait complete. 2024-04-20T10:58:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.932+0000 7f9f93a14640 1 Processor -- start 2024-04-20T10:58:11.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.932+0000 7f9f93a14640 1 -- start start 2024-04-20T10:58:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f93a14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f8c0b54f0 0x7f9f8c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f93a14640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9f8c0d0c90 0x7f9f8c1f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f93a14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f8c155950 0x7f9f8c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f93a14640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c15fb90 con 0x7f9f8c155950 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f93a14640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c15c540 con 0x7f9f8c0b54f0 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f93a14640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c15fd10 con 0x7f9f8c0d0c90 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f91fb1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f8c155950 0x7f9f8c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f91fb1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f8c155950 0x7f9f8c15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35444/0 (socket says 172.21.15.27:35444) 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f91fb1640 1 -- 172.21.15.27:0/369182668 learned_addr learned my addr 172.21.15.27:0/369182668 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f917b0640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f8c0b54f0 0x7f9f8c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f90faf640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9f8c0d0c90 0x7f9f8c1f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f91fb1640 1 -- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9f8c0d0c90 msgr2=0x7f9f8c1f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f91fb1640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9f8c0d0c90 0x7f9f8c1f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f91fb1640 1 -- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f8c0b54f0 msgr2=0x7f9f8c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.933+0000 7f9f91fb1640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f8c0b54f0 0x7f9f8c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.934+0000 7f9f91fb1640 1 -- 172.21.15.27:0/369182668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c15b4e0 con 0x7f9f8c155950 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.934+0000 7f9f90faf640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9f8c0d0c90 0x7f9f8c1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.934+0000 7f9f91fb1640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f8c155950 0x7f9f8c15d590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400cb20 tx=0x7f9f8400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:11.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.934+0000 7f9f81ffb640 1 -- 172.21.15.27:0/369182668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84017070 con 0x7f9f8c155950 2024-04-20T10:58:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.934+0000 7f9f81ffb640 1 -- 172.21.15.27:0/369182668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8400ac40 con 0x7f9f8c155950 2024-04-20T10:58:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.934+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c15af20 con 0x7f9f8c155950 2024-04-20T10:58:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.934+0000 7f9f81ffb640 1 -- 172.21.15.27:0/369182668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84012400 con 0x7f9f8c155950 2024-04-20T10:58:11.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.934+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c1f1800 con 0x7f9f8c155950 2024-04-20T10:58:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.935+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c14c270 con 0x7f9f8c155950 2024-04-20T10:58:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.935+0000 7f9f81ffb640 1 -- 172.21.15.27:0/369182668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f8401a070 con 0x7f9f8c155950 2024-04-20T10:58:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.936+0000 7f9f81ffb640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9f7006c8f0 0x7f9f7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.936+0000 7f9f81ffb640 1 -- 172.21.15.27:0/369182668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f9f8408b850 con 0x7f9f8c155950 2024-04-20T10:58:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.936+0000 7f9f917b0640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9f7006c8f0 0x7f9f7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:11.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.937+0000 7f9f917b0640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9f7006c8f0 0x7f9f7006edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9f88001650 tx=0x7f9f88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:11.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:11.940+0000 7f9f81ffb640 1 -- 172.21.15.27:0/369182668 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9f8c14c270 con 0x7f9f8c155950 2024-04-20T10:58:12.113 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:58:12.116 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:58:12.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.164+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f8c04fcf0 con 0x7f9f8c155950 2024-04-20T10:58:12.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.165+0000 7f9f81ffb640 1 -- 172.21.15.27:0/369182668 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7f9f84059570 con 0x7f9f8c155950 2024-04-20T10:58:12.165 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:12.165 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:58:12.165 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9f7006c8f0 msgr2=0x7f9f7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9f7006c8f0 0x7f9f7006edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9f88001650 tx=0x7f9f88009040 comp rx=0 tx=0).stop 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f8c155950 msgr2=0x7f9f8c15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f8c155950 0x7f9f8c15d590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400cb20 tx=0x7f9f8400a060 comp rx=0 tx=0).stop 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 shutdown_connections 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9f7006c8f0 0x7f9f7006edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f8c155950 0x7f9f8c15d590 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9f8c0d0c90 0x7f9f8c1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 --2- 172.21.15.27:0/369182668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f8c0b54f0 0x7f9f8c15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 >> 172.21.15.27:0/369182668 conn(0x7f9f8c0676c0 msgr2=0x7f9f8c067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 shutdown_connections 2024-04-20T10:58:12.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:12.167+0000 7f9f93a14640 1 -- 172.21.15.27:0/369182668 wait complete. 2024-04-20T10:58:12.177 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T10:58:14.178 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:14.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.290+0000 7fbfae432640 1 Processor -- start 2024-04-20T10:58:14.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.290+0000 7fbfae432640 1 -- start start 2024-04-20T10:58:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfae432640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbfa8152320 0x7fbfa8070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfae432640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa806fcd0 0x7fbfa8071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfae432640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa80c9970 con 0x7fbfa8151f40 2024-04-20T10:58:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfae432640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa8070210 con 0x7fbfa8152320 2024-04-20T10:58:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfae432640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa815b750 con 0x7fbfa806fcd0 2024-04-20T10:58:14.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfa77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbfa8152320 0x7fbfa8070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfa77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbfa8152320 0x7fbfa8070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35460/0 (socket says 172.21.15.27:35460) 2024-04-20T10:58:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfa77fe640 1 -- 172.21.15.27:0/165205201 learned_addr learned my addr 172.21.15.27:0/165205201 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfa7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbfa8151f40 0x7fbfa806f5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45270/0 (socket says 172.21.15.27:45270) 2024-04-20T10:58:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfac9cf640 1 --2- 172.21.15.27:0/165205201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa806fcd0 0x7fbfa8071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfa77fe640 1 -- 172.21.15.27:0/165205201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa806fcd0 msgr2=0x7fbfa8071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfa77fe640 1 --2- 172.21.15.27:0/165205201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa806fcd0 0x7fbfa8071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfa77fe640 1 -- 172.21.15.27:0/165205201 >> v1:172.21.15.27:6789/0 conn(0x7fbfa8151f40 legacy=0x7fbfa806f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfa77fe640 1 -- 172.21.15.27:0/165205201 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa815ba30 con 0x7fbfa8152320 2024-04-20T10:58:14.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.291+0000 7fbfac9cf640 1 --2- 172.21.15.27:0/165205201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa806fcd0 0x7fbfa8071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:58:14.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.292+0000 7fbfa77fe640 1 --2- 172.21.15.27:0/165205201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8152320 0x7fbfa8070ac0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009870 tx=0x7fbf9c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa30f72140a00e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.292+0000 7fbfa6ffd640 1 -- 172.21.15.27:0/165205201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c02f830 con 0x7fbfa8152320 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.292+0000 7fbfa6ffd640 1 -- 172.21.15.27:0/165205201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf9c02f9d0 con 0x7fbfa8152320 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.292+0000 7fbfa6ffd640 1 -- 172.21.15.27:0/165205201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c02fda0 con 0x7fbfa8152320 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.293+0000 7fbfae432640 1 -- 172.21.15.27:0/165205201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8152320 msgr2=0x7fbfa8070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.293+0000 7fbfae432640 1 --2- 172.21.15.27:0/165205201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8152320 0x7fbfa8070ac0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009870 tx=0x7fbf9c02ba10 comp rx=0 tx=0).stop 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.293+0000 7fbfae432640 1 -- 172.21.15.27:0/165205201 shutdown_connections 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.293+0000 7fbfae432640 1 --2- 172.21.15.27:0/165205201 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa806fcd0 0x7fbfa8071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.293+0000 7fbfae432640 1 --2- 172.21.15.27:0/165205201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8152320 0x7fbfa8070ac0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.293+0000 7fbfae432640 1 -- 172.21.15.27:0/165205201 >> 172.21.15.27:0/165205201 conn(0x7fbfa80cee90 msgr2=0x7fbfa80cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.293+0000 7fbfae432640 1 -- 172.21.15.27:0/165205201 shutdown_connections 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.293+0000 7fbfae432640 1 -- 172.21.15.27:0/165205201 wait complete. 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.294+0000 7fbfae432640 1 Processor -- start 2024-04-20T10:58:14.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.294+0000 7fbfae432640 1 -- start start 2024-04-20T10:58:14.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfae432640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfa806fcd0 0x7fbfa80ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:14.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfae432640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8151f40 0x7fbfa815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:14.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfae432640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa8152320 0x7fbfa81e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:14.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfae432640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa815c2d0 con 0x7fbfa8151f40 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfae432640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa815bff0 con 0x7fbfa806fcd0 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfae432640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa806f0a0 con 0x7fbfa8152320 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8151f40 0x7fbfa815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8151f40 0x7fbfa815a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35470/0 (socket says 172.21.15.27:35470) 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa77fe640 1 -- 172.21.15.27:0/1232327061 learned_addr learned my addr 172.21.15.27:0/1232327061 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfac9cf640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa8152320 0x7fbfa81e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa7fff640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfa806fcd0 0x7fbfa80ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa77fe640 1 -- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa8152320 msgr2=0x7fbfa81e5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa77fe640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa8152320 0x7fbfa81e5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa77fe640 1 -- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfa806fcd0 msgr2=0x7fbfa80ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa77fe640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfa806fcd0 0x7fbfa80ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa77fe640 1 -- 172.21.15.27:0/1232327061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa80cebd0 con 0x7fbfa8151f40 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa7fff640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfa806fcd0 0x7fbfa80ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.295+0000 7fbfa77fe640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8151f40 0x7fbfa815a470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c02ba50 tx=0x7fbf9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:14.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.296+0000 7fbfa4ff9640 1 -- 172.21.15.27:0/1232327061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c047070 con 0x7fbfa8151f40 2024-04-20T10:58:14.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.296+0000 7fbfa4ff9640 1 -- 172.21.15.27:0/1232327061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf9c02f820 con 0x7fbfa8151f40 2024-04-20T10:58:14.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.296+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa80c5e50 con 0x7fbfa8151f40 2024-04-20T10:58:14.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.296+0000 7fbfa4ff9640 1 -- 172.21.15.27:0/1232327061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c042630 con 0x7fbfa8151f40 2024-04-20T10:58:14.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.296+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa80c5300 con 0x7fbfa8151f40 2024-04-20T10:58:14.298 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.297+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa804fcf0 con 0x7fbfa8151f40 2024-04-20T10:58:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.297+0000 7fbfa4ff9640 1 -- 172.21.15.27:0/1232327061 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf9c04a070 con 0x7fbfa8151f40 2024-04-20T10:58:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.297+0000 7fbfa4ff9640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbf9406c8f0 0x7fbf9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.298+0000 7fbfa4ff9640 1 -- 172.21.15.27:0/1232327061 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fbf9c0add40 con 0x7fbfa8151f40 2024-04-20T10:58:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.298+0000 7fbfa7fff640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbf9406c8f0 0x7fbf9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:14.299 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.298+0000 7fbfa7fff640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbf9406c8f0 0x7fbf9406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf98007fd0 tx=0x7fbf98007d60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:14.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.302+0000 7fbfa4ff9640 1 -- 172.21.15.27:0/1232327061 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbf9c033030 con 0x7fbfa8151f40 2024-04-20T10:58:14.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.527+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfa806e5d0 con 0x7fbfa8151f40 2024-04-20T10:58:14.527 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.528+0000 7fbfa4ff9640 1 -- 172.21.15.27:0/1232327061 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7fbf9c07ba50 con 0x7fbfa8151f40 2024-04-20T10:58:14.528 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:14.528 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:58:14.528 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbf9406c8f0 msgr2=0x7fbf9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbf9406c8f0 0x7fbf9406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf98007fd0 tx=0x7fbf98007d60 comp rx=0 tx=0).stop 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8151f40 msgr2=0x7fbfa815a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8151f40 0x7fbfa815a470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c02ba50 tx=0x7fbf9c004730 comp rx=0 tx=0).stop 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 shutdown_connections 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbf9406c8f0 0x7fbf9406edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbfa8152320 0x7fbfa81e5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfa8151f40 0x7fbfa815a470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 --2- 172.21.15.27:0/1232327061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfa806fcd0 0x7fbfa80ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 >> 172.21.15.27:0/1232327061 conn(0x7fbfa80cee90 msgr2=0x7fbfa8070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 shutdown_connections 2024-04-20T10:58:14.530 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:14.530+0000 7fbfae432640 1 -- 172.21.15.27:0/1232327061 wait complete. 2024-04-20T10:58:14.540 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T10:58:16.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.656+0000 7f9c8ec43640 1 Processor -- start 2024-04-20T10:58:16.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.656+0000 7f9c8ec43640 1 -- start start 2024-04-20T10:58:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c8ec43640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9c88155950 0x7f9c880cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c8ec43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 0x7f9c880b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c8ec43640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c880b5ea0 con 0x7f9c880b5a30 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c8ec43640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8815b9c0 con 0x7f9c88155950 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c8ec43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8815bca0 con 0x7f9c880cc330 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c8d1e0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9c880b5a30 0x7f9c880d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43008/0 (socket says 172.21.15.27:43008) 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c8d1e0640 1 -- 172.21.15.27:0/2499183157 learned_addr learned my addr 172.21.15.27:0/2499183157 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c8c9df640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9c88155950 0x7f9c880cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c87fff640 1 --2- 172.21.15.27:0/2499183157 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 0x7f9c880b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c877fe640 1 -- 172.21.15.27:0/2499183157 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4092935153 0 0) 0x7f9c880b5ea0 con 0x7f9c880b5a30 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.657+0000 7f9c877fe640 1 -- 172.21.15.27:0/2499183157 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c78003520 con 0x7f9c880b5a30 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.658+0000 7f9c8c9df640 1 -- 172.21.15.27:0/2499183157 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 msgr2=0x7f9c880b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.658+0000 7f9c8c9df640 1 --2- 172.21.15.27:0/2499183157 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 0x7f9c880b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.658+0000 7f9c8c9df640 1 -- 172.21.15.27:0/2499183157 >> v1:172.21.15.27:6789/0 conn(0x7f9c880b5a30 legacy=0x7f9c880d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.658+0000 7f9c8c9df640 1 -- 172.21.15.27:0/2499183157 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8815bf80 con 0x7f9c88155950 2024-04-20T10:58:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.658+0000 7f9c87fff640 1 --2- 172.21.15.27:0/2499183157 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 0x7f9c880b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:58:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.658+0000 7f9c8c9df640 1 --2- 172.21.15.27:0/2499183157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c88155950 0x7f9c880cbdf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0097a0 tx=0x7f9c7c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1322804b2f9fcbf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c877fe640 1 -- 172.21.15.27:0/2499183157 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c034070 con 0x7f9c88155950 2024-04-20T10:58:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c877fe640 1 -- 172.21.15.27:0/2499183157 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c030610 con 0x7f9c88155950 2024-04-20T10:58:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c877fe640 1 -- 172.21.15.27:0/2499183157 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0309e0 con 0x7f9c88155950 2024-04-20T10:58:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2499183157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c88155950 msgr2=0x7f9c880cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c8ec43640 1 --2- 172.21.15.27:0/2499183157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c88155950 0x7f9c880cbdf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c0097a0 tx=0x7f9c7c02b960 comp rx=0 tx=0).stop 2024-04-20T10:58:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2499183157 shutdown_connections 2024-04-20T10:58:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c8ec43640 1 --2- 172.21.15.27:0/2499183157 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 0x7f9c880b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c8ec43640 1 --2- 172.21.15.27:0/2499183157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c88155950 0x7f9c880cbdf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2499183157 >> 172.21.15.27:0/2499183157 conn(0x7f9c880676c0 msgr2=0x7f9c88067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2499183157 shutdown_connections 2024-04-20T10:58:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.659+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2499183157 wait complete. 2024-04-20T10:58:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.660+0000 7f9c8ec43640 1 Processor -- start 2024-04-20T10:58:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.660+0000 7f9c8ec43640 1 -- start start 2024-04-20T10:58:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.661+0000 7f9c8ec43640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c880b5a30 0x7f9c8814d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:16.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.661+0000 7f9c8ec43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 0x7f9c88150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.661+0000 7f9c8ec43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c88155950 0x7f9c88150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.661+0000 7f9c8ec43640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8815fb90 con 0x7f9c880b5a30 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.661+0000 7f9c8ec43640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8815c540 con 0x7f9c88155950 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.661+0000 7f9c8ec43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8815fd10 con 0x7f9c880cc330 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.661+0000 7f9c8c9df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c880b5a30 0x7f9c8814d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.661+0000 7f9c8c9df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c880b5a30 0x7f9c8814d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47562/0 (socket says 172.21.15.27:47562) 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.661+0000 7f9c8c9df640 1 -- 172.21.15.27:0/2272412801 learned_addr learned my addr 172.21.15.27:0/2272412801 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c8d1e0640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c88155950 0x7f9c88150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c87fff640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 0x7f9c88150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c8c9df640 1 -- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 msgr2=0x7f9c88150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c8c9df640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 0x7f9c88150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c8c9df640 1 -- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c88155950 msgr2=0x7f9c88150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c8c9df640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c88155950 0x7f9c88150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c8c9df640 1 -- 172.21.15.27:0/2272412801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8814e830 con 0x7f9c880b5a30 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c8c9df640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c880b5a30 0x7f9c8814d6b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009890 tx=0x7f9c7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c857fa640 1 -- 172.21.15.27:0/2272412801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c034040 con 0x7f9c880b5a30 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c857fa640 1 -- 172.21.15.27:0/2272412801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c030410 con 0x7f9c880b5a30 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8815eb50 con 0x7f9c880b5a30 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c857fa640 1 -- 172.21.15.27:0/2272412801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c006e10 con 0x7f9c880b5a30 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.662+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c880c6ea0 con 0x7f9c880b5a30 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.663+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8804fcf0 con 0x7f9c880b5a30 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.664+0000 7f9c857fa640 1 -- 172.21.15.27:0/2272412801 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c7c037070 con 0x7f9c880b5a30 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.664+0000 7f9c857fa640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9c6c06c920 0x7f9c6c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.664+0000 7f9c857fa640 1 -- 172.21.15.27:0/2272412801 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f9c7c0ad8c0 con 0x7f9c880b5a30 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.664+0000 7f9c87fff640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9c6c06c920 0x7f9c6c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.665+0000 7f9c87fff640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9c6c06c920 0x7f9c6c06ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9c880d1620 tx=0x7f9c74009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.669+0000 7f9c857fa640 1 -- 172.21.15.27:0/2272412801 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9c7c07b660 con 0x7f9c880b5a30 2024-04-20T10:58:16.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.894+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c880cf930 con 0x7f9c880b5a30 2024-04-20T10:58:16.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.895+0000 7f9c857fa640 1 -- 172.21.15.27:0/2272412801 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7f9c7c07f960 con 0x7f9c880b5a30 2024-04-20T10:58:16.895 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:16.895 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:58:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:58:16.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9c6c06c920 msgr2=0x7f9c6c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:16.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9c6c06c920 0x7f9c6c06ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9c880d1620 tx=0x7f9c74009190 comp rx=0 tx=0).stop 2024-04-20T10:58:16.897 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c880b5a30 msgr2=0x7f9c8814d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:16.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c880b5a30 0x7f9c8814d6b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009890 tx=0x7f9c7c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 shutdown_connections 2024-04-20T10:58:16.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9c6c06c920 0x7f9c6c06ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c88155950 0x7f9c88150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9c880cc330 0x7f9c88150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 --2- 172.21.15.27:0/2272412801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c880b5a30 0x7f9c8814d6b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:16.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 >> 172.21.15.27:0/2272412801 conn(0x7f9c880676c0 msgr2=0x7f9c880b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:16.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 shutdown_connections 2024-04-20T10:58:16.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:16.897+0000 7f9c8ec43640 1 -- 172.21.15.27:0/2272412801 wait complete. 2024-04-20T10:58:16.908 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T10:58:18.909 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:19.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.010+0000 7fc4edd9f640 1 Processor -- start 2024-04-20T10:58:19.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.011+0000 7fc4edd9f640 1 -- start start 2024-04-20T10:58:19.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.011+0000 7fc4edd9f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc4e80c4500 0x7fc4e8154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:19.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.011+0000 7fc4edd9f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc4e815be00 0x7fc4e8155530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:19.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.011+0000 7fc4edd9f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e80717b0 con 0x7fc4e80c4120 2024-04-20T10:58:19.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.011+0000 7fc4edd9f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e815e720 con 0x7fc4e80c4500 2024-04-20T10:58:19.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.011+0000 7fc4edd9f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e815e970 con 0x7fc4e815be00 2024-04-20T10:58:19.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.011+0000 7fc4e7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc4e80c4500 0x7fc4e8154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:19.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.012+0000 7fc4e7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc4e80c4500 0x7fc4e8154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47570/0 (socket says 172.21.15.27:47570) 2024-04-20T10:58:19.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.012+0000 7fc4e7fff640 1 -- 172.21.15.27:0/3227676691 learned_addr learned my addr 172.21.15.27:0/3227676691 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:19.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.012+0000 7fc4ed59e640 1 --2- 172.21.15.27:0/3227676691 >> v2:172.21.15.137:3300/0 conn(0x7fc4e815be00 0x7fc4e8155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:19.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.012+0000 7fc4e7fff640 1 -- 172.21.15.27:0/3227676691 >> v2:172.21.15.137:3300/0 conn(0x7fc4e815be00 msgr2=0x7fc4e8155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:19.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.012+0000 7fc4e7fff640 1 --2- 172.21.15.27:0/3227676691 >> v2:172.21.15.137:3300/0 conn(0x7fc4e815be00 0x7fc4e8155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.012+0000 7fc4e7fff640 1 -- 172.21.15.27:0/3227676691 >> v1:172.21.15.137:6789/0 conn(0x7fc4e80c4120 legacy=0x7fc4e8070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:19.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.012+0000 7fc4e7fff640 1 -- 172.21.15.27:0/3227676691 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e815ec50 con 0x7fc4e80c4500 2024-04-20T10:58:19.013 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.012+0000 7fc4e7fff640 1 --2- 172.21.15.27:0/3227676691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e80c4500 0x7fc4e8154ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc4e815e8a0 tx=0x7fc4d402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37b11421b6b2c7e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4e77fe640 1 -- 172.21.15.27:0/3227676691 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4034070 con 0x7fc4e80c4500 2024-04-20T10:58:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4e77fe640 1 -- 172.21.15.27:0/3227676691 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d4030860 con 0x7fc4e80c4500 2024-04-20T10:58:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4e77fe640 1 -- 172.21.15.27:0/3227676691 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4030c30 con 0x7fc4e80c4500 2024-04-20T10:58:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3227676691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e80c4500 msgr2=0x7fc4e8154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4edd9f640 1 --2- 172.21.15.27:0/3227676691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e80c4500 0x7fc4e8154ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc4e815e8a0 tx=0x7fc4d402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3227676691 shutdown_connections 2024-04-20T10:58:19.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4edd9f640 1 --2- 172.21.15.27:0/3227676691 >> v2:172.21.15.137:3300/0 conn(0x7fc4e815be00 0x7fc4e8155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4edd9f640 1 --2- 172.21.15.27:0/3227676691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e80c4500 0x7fc4e8154ff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3227676691 >> 172.21.15.27:0/3227676691 conn(0x7fc4e80676c0 msgr2=0x7fc4e8067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:19.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3227676691 shutdown_connections 2024-04-20T10:58:19.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.013+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3227676691 wait complete. 2024-04-20T10:58:19.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.014+0000 7fc4edd9f640 1 Processor -- start 2024-04-20T10:58:19.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.014+0000 7fc4edd9f640 1 -- start start 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4edd9f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80c4120 0x7fc4e8155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4edd9f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc4e80c4500 0x7fc4e815d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4edd9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e815be00 0x7fc4e81f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4edd9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e815f4f0 con 0x7fc4e815be00 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4edd9f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e815f210 con 0x7fc4e80c4120 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4edd9f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e80714d0 con 0x7fc4e80c4500 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4ed59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e815be00 0x7fc4e81f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4ed59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e815be00 0x7fc4e81f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47582/0 (socket says 172.21.15.27:47582) 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4ed59e640 1 -- 172.21.15.27:0/3769298621 learned_addr learned my addr 172.21.15.27:0/3769298621 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4ecd9d640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80c4120 0x7fc4e8155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.015+0000 7fc4e7fff640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc4e80c4500 0x7fc4e815d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4ed59e640 1 -- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc4e80c4500 msgr2=0x7fc4e815d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4ed59e640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc4e80c4500 0x7fc4e815d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4ed59e640 1 -- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80c4120 msgr2=0x7fc4e8155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4ed59e640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80c4120 0x7fc4e8155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4ed59e640 1 -- 172.21.15.27:0/3769298621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e814edf0 con 0x7fc4e815be00 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4ed59e640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e815be00 0x7fc4e81f5160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0007e30 tx=0x7fc4e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4e57fa640 1 -- 172.21.15.27:0/3769298621 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0014620 con 0x7fc4e815be00 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4e57fa640 1 -- 172.21.15.27:0/3769298621 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e0019070 con 0x7fc4e815be00 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e81e4880 con 0x7fc4e815be00 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4e57fa640 1 -- 172.21.15.27:0/3769298621 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0014a00 con 0x7fc4e815be00 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.016+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e81f4c40 con 0x7fc4e815be00 2024-04-20T10:58:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.017+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e8152650 con 0x7fc4e815be00 2024-04-20T10:58:19.019 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.018+0000 7fc4e57fa640 1 -- 172.21.15.27:0/3769298621 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4e0014ba0 con 0x7fc4e815be00 2024-04-20T10:58:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.018+0000 7fc4e57fa640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc4cc06c8f0 0x7fc4cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.018+0000 7fc4e57fa640 1 -- 172.21.15.27:0/3769298621 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fc4e008c420 con 0x7fc4e815be00 2024-04-20T10:58:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.018+0000 7fc4ecd9d640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc4cc06c8f0 0x7fc4cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:19.020 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.019+0000 7fc4ecd9d640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc4cc06c8f0 0x7fc4cc06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0016e0 tx=0x7fc4dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:19.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.022+0000 7fc4e57fa640 1 -- 172.21.15.27:0/3769298621 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc4e8152650 con 0x7fc4e815be00 2024-04-20T10:58:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.249+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4e8076610 con 0x7fc4e815be00 2024-04-20T10:58:19.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.249+0000 7fc4e57fa640 1 -- 172.21.15.27:0/3769298621 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7fc4e005a140 con 0x7fc4e815be00 2024-04-20T10:58:19.249 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:19.250 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:58:19.250 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc4cc06c8f0 msgr2=0x7fc4cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc4cc06c8f0 0x7fc4cc06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0016e0 tx=0x7fc4dc009040 comp rx=0 tx=0).stop 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e815be00 msgr2=0x7fc4e81f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e815be00 0x7fc4e81f5160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0007e30 tx=0x7fc4e000e4b0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 shutdown_connections 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc4cc06c8f0 0x7fc4cc06edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4e815be00 0x7fc4e81f5160 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc4e80c4500 0x7fc4e815d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 --2- 172.21.15.27:0/3769298621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80c4120 0x7fc4e8155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 >> 172.21.15.27:0/3769298621 conn(0x7fc4e80676c0 msgr2=0x7fc4e8151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 shutdown_connections 2024-04-20T10:58:19.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:19.252+0000 7fc4edd9f640 1 -- 172.21.15.27:0/3769298621 wait complete. 2024-04-20T10:58:19.262 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T10:58:21.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.378+0000 7fc053f3d640 1 Processor -- start 2024-04-20T10:58:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.379+0000 7fc053f3d640 1 -- start start 2024-04-20T10:58:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.379+0000 7fc053f3d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc04c15c860 0x7fc04c15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.379+0000 7fc053f3d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04c13f2c0 con 0x7fc04c13bab0 2024-04-20T10:58:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.379+0000 7fc053f3d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04c15d1e0 con 0x7fc04c13e330 2024-04-20T10:58:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.379+0000 7fc053f3d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fc04c0cc160 con 0x7fc04c15c860 2024-04-20T10:58:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.379+0000 7fc051cd9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc04c13bab0 0x7fc04c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43022/0 (socket says 172.21.15.27:43022) 2024-04-20T10:58:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.379+0000 7fc051cd9640 1 -- 172.21.15.27:0/1572248436 learned_addr learned my addr 172.21.15.27:0/1572248436 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.380+0000 7fc0514d8640 1 --2- 172.21.15.27:0/1572248436 >> v2:172.21.15.137:3300/0 conn(0x7fc04c15c860 0x7fc04c15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.380+0000 7fc050cd7640 1 -- 172.21.15.27:0/1572248436 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800597689 0 0) 0x7fc04c13f2c0 con 0x7fc04c13bab0 2024-04-20T10:58:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.380+0000 7fc050cd7640 1 -- 172.21.15.27:0/1572248436 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc034003520 con 0x7fc04c13bab0 2024-04-20T10:58:21.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.380+0000 7fc050cd7640 1 -- 172.21.15.27:0/1572248436 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 639500141 0 0) 0x7fc04c15d1e0 con 0x7fc04c13e330 2024-04-20T10:58:21.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.380+0000 7fc050cd7640 1 -- 172.21.15.27:0/1572248436 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc04c13f2c0 con 0x7fc04c13e330 2024-04-20T10:58:21.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.380+0000 7fc0514d8640 1 -- 172.21.15.27:0/1572248436 >> v1:172.21.15.137:6789/0 conn(0x7fc04c13e330 legacy=0x7fc04c13e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.380+0000 7fc0514d8640 1 -- 172.21.15.27:0/1572248436 >> v1:172.21.15.27:6789/0 conn(0x7fc04c13bab0 legacy=0x7fc04c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.381 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.380+0000 7fc0514d8640 1 -- 172.21.15.27:0/1572248436 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c159360 con 0x7fc04c15c860 2024-04-20T10:58:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.381+0000 7fc0514d8640 1 --2- 172.21.15.27:0/1572248436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c15c860 0x7fc04c15cca0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc0400097a0 tx=0x7fc04002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185a0df89f2ecf9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.381+0000 7fc050cd7640 1 -- 172.21.15.27:0/1572248436 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040034070 con 0x7fc04c15c860 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.381+0000 7fc050cd7640 1 -- 172.21.15.27:0/1572248436 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc040030610 con 0x7fc04c15c860 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.381+0000 7fc050cd7640 1 -- 172.21.15.27:0/1572248436 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0400309c0 con 0x7fc04c15c860 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.381+0000 7fc053f3d640 1 -- 172.21.15.27:0/1572248436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c15c860 msgr2=0x7fc04c15cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.382+0000 7fc053f3d640 1 --2- 172.21.15.27:0/1572248436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c15c860 0x7fc04c15cca0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc0400097a0 tx=0x7fc04002b960 comp rx=0 tx=0).stop 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.382+0000 7fc053f3d640 1 -- 172.21.15.27:0/1572248436 shutdown_connections 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.382+0000 7fc053f3d640 1 --2- 172.21.15.27:0/1572248436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c15c860 0x7fc04c15cca0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.382+0000 7fc053f3d640 1 -- 172.21.15.27:0/1572248436 >> 172.21.15.27:0/1572248436 conn(0x7fc04c06c070 msgr2=0x7fc04c0676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.382+0000 7fc053f3d640 1 -- 172.21.15.27:0/1572248436 shutdown_connections 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.382+0000 7fc053f3d640 1 -- 172.21.15.27:0/1572248436 wait complete. 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.383+0000 7fc053f3d640 1 Processor -- start 2024-04-20T10:58:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.383+0000 7fc053f3d640 1 -- start start 2024-04-20T10:58:21.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.383+0000 7fc053f3d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc04c13bab0 0x7fc04c14e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:21.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.383+0000 7fc053f3d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc04c13e330 0x7fc04c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:21.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc053f3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c15c860 0x7fc04c1f0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:21.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc053f3d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c159c00 con 0x7fc04c13bab0 2024-04-20T10:58:21.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc053f3d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c159920 con 0x7fc04c15c860 2024-04-20T10:58:21.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc053f3d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c13eed0 con 0x7fc04c13e330 2024-04-20T10:58:21.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc051cd9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc04c13bab0 0x7fc04c14e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:21.385 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc051cd9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc04c13bab0 0x7fc04c14e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47594/0 (socket says 172.21.15.27:47594) 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc051cd9640 1 -- 172.21.15.27:0/1118377584 learned_addr learned my addr 172.21.15.27:0/1118377584 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc0524da640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c15c860 0x7fc04c1f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc0514d8640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc04c13e330 0x7fc04c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc051cd9640 1 -- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc04c13e330 msgr2=0x7fc04c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc051cd9640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc04c13e330 0x7fc04c070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc051cd9640 1 -- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c15c860 msgr2=0x7fc04c1f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc051cd9640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c15c860 0x7fc04c1f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc051cd9640 1 -- 172.21.15.27:0/1118377584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c14e560 con 0x7fc04c13bab0 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc051cd9640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc04c13bab0 0x7fc04c14e020 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc04400c410 tx=0x7fc04400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc04a7fc640 1 -- 172.21.15.27:0/1118377584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044017070 con 0x7fc04c13bab0 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc04a7fc640 1 -- 172.21.15.27:0/1118377584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc044005ce0 con 0x7fc04c13bab0 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c1f0ff0 con 0x7fc04c13bab0 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc04a7fc640 1 -- 172.21.15.27:0/1118377584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0440072d0 con 0x7fc04c13bab0 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.384+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c1f0d10 con 0x7fc04c13bab0 2024-04-20T10:58:21.386 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.385+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04c04fcf0 con 0x7fc04c13bab0 2024-04-20T10:58:21.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.386+0000 7fc04a7fc640 1 -- 172.21.15.27:0/1118377584 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc04401a070 con 0x7fc04c13bab0 2024-04-20T10:58:21.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.386+0000 7fc04a7fc640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc03806c8f0 0x7fc03806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:21.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.386+0000 7fc04a7fc640 1 -- 172.21.15.27:0/1118377584 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fc04408afd0 con 0x7fc04c13bab0 2024-04-20T10:58:21.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.386+0000 7fc0514d8640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc03806c8f0 0x7fc03806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:21.387 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.387+0000 7fc0514d8640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc03806c8f0 0x7fc03806edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc040036000 tx=0x7fc04003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:21.390 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.391+0000 7fc04a7fc640 1 -- 172.21.15.27:0/1118377584 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc044058c50 con 0x7fc04c13bab0 2024-04-20T10:58:21.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.615+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc04c0cc500 con 0x7fc04c13bab0 2024-04-20T10:58:21.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.615+0000 7fc04a7fc640 1 -- 172.21.15.27:0/1118377584 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+6933 (secure 0 0 0) 0x7fc04405cf50 con 0x7fc04c13bab0 2024-04-20T10:58:21.615 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:21.615 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:57:48.090789+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4259},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4259":{"gid":4259,"name":"f","rank":4,"incarnation":18,"state":"up:active","state_seq":37,"addr":"172.21.15.137:6835/1164978945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1164978945},{"type":"v1","addr":"172.21.15.137:6835","nonce":1164978945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272,4259]},"id":1}]} 2024-04-20T10:58:21.616 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.617+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc03806c8f0 msgr2=0x7fc03806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.617+0000 7fc053f3d640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc03806c8f0 0x7fc03806edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc040036000 tx=0x7fc04003b040 comp rx=0 tx=0).stop 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc04c13bab0 msgr2=0x7fc04c14e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc04c13bab0 0x7fc04c14e020 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc04400c410 tx=0x7fc04400c8d0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 shutdown_connections 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc03806c8f0 0x7fc03806edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc04c15c860 0x7fc04c1f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc04c13e330 0x7fc04c070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 --2- 172.21.15.27:0/1118377584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc04c13bab0 0x7fc04c14e020 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 >> 172.21.15.27:0/1118377584 conn(0x7fc04c06c070 msgr2=0x7fc04c1390f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 shutdown_connections 2024-04-20T10:58:21.618 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:21.618+0000 7fc053f3d640 1 -- 172.21.15.27:0/1118377584 wait complete. 2024-04-20T10:58:21.628 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-04-20T10:58:23.629 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:23.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.743+0000 7f6aae550640 1 Processor -- start 2024-04-20T10:58:23.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.743+0000 7f6aae550640 1 -- start start 2024-04-20T10:58:23.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aae550640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa8155950 0x7f6aa80cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:23.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aae550640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa815b9c0 con 0x7f6aa80b5c00 2024-04-20T10:58:23.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aae550640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa815bca0 con 0x7f6aa80cc330 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aae550640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa815bf80 con 0x7f6aa8155950 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aacaed640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6aa80b5c00 0x7f6aa80d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43038/0 (socket says 172.21.15.27:43038) 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aacaed640 1 -- 172.21.15.27:0/29118368 learned_addr learned my addr 172.21.15.27:0/29118368 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aa7fff640 1 --2- 172.21.15.27:0/29118368 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa8155950 0x7f6aa80cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aa6ffd640 1 -- 172.21.15.27:0/29118368 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3994071242 0 0) 0x7f6aa815b9c0 con 0x7f6aa80b5c00 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aa6ffd640 1 -- 172.21.15.27:0/29118368 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a94003520 con 0x7f6aa80b5c00 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aa6ffd640 1 -- 172.21.15.27:0/29118368 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2849528805 0 0) 0x7f6aa815bca0 con 0x7f6aa80cc330 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.744+0000 7f6aa6ffd640 1 -- 172.21.15.27:0/29118368 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6aa815b9c0 con 0x7f6aa80cc330 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.745+0000 7f6aa7fff640 1 -- 172.21.15.27:0/29118368 >> v1:172.21.15.137:6789/0 conn(0x7f6aa80cc330 legacy=0x7f6aa80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.745+0000 7f6aa7fff640 1 -- 172.21.15.27:0/29118368 >> v1:172.21.15.27:6789/0 conn(0x7f6aa80b5c00 legacy=0x7f6aa80d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:23.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.745+0000 7f6aa7fff640 1 -- 172.21.15.27:0/29118368 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa815c260 con 0x7f6aa8155950 2024-04-20T10:58:23.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.745+0000 7f6aa7fff640 1 --2- 172.21.15.27:0/29118368 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa8155950 0x7f6aa80cbdf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6a900097d0 tx=0x7f6a9002b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826511ade94535f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aa6ffd640 1 -- 172.21.15.27:0/29118368 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90034070 con 0x7f6aa8155950 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aa6ffd640 1 -- 172.21.15.27:0/29118368 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90030880 con 0x7f6aa8155950 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aa6ffd640 1 -- 172.21.15.27:0/29118368 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90030c50 con 0x7f6aa8155950 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aae550640 1 -- 172.21.15.27:0/29118368 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa8155950 msgr2=0x7f6aa80cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aae550640 1 --2- 172.21.15.27:0/29118368 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa8155950 0x7f6aa80cbdf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6a900097d0 tx=0x7f6a9002b9c0 comp rx=0 tx=0).stop 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aae550640 1 -- 172.21.15.27:0/29118368 shutdown_connections 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aae550640 1 --2- 172.21.15.27:0/29118368 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa8155950 0x7f6aa80cbdf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aae550640 1 -- 172.21.15.27:0/29118368 >> 172.21.15.27:0/29118368 conn(0x7f6aa80676c0 msgr2=0x7f6aa8067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aae550640 1 -- 172.21.15.27:0/29118368 shutdown_connections 2024-04-20T10:58:23.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.746+0000 7f6aae550640 1 -- 172.21.15.27:0/29118368 wait complete. 2024-04-20T10:58:23.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.747+0000 7f6aae550640 1 Processor -- start 2024-04-20T10:58:23.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aae550640 1 -- start start 2024-04-20T10:58:23.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aae550640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6aa80b5c00 0x7f6aa8151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aae550640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa80cc330 0x7f6aa80c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aae550640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa8155950 0x7f6aa80c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aae550640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa815fd10 con 0x7f6aa80b5c00 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aae550640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa815fb90 con 0x7f6aa8155950 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aae550640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa8155d30 con 0x7f6aa80cc330 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6aa80b5c00 0x7f6aa8151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aa7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6aa80b5c00 0x7f6aa8151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47608/0 (socket says 172.21.15.27:47608) 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.748+0000 7f6aa7fff640 1 -- 172.21.15.27:0/2220467271 learned_addr learned my addr 172.21.15.27:0/2220467271 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa77fe640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa80cc330 0x7f6aa80c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aacaed640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa8155950 0x7f6aa80c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa7fff640 1 -- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa80cc330 msgr2=0x7f6aa80c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa7fff640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa80cc330 0x7f6aa80c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:23.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa7fff640 1 -- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa8155950 msgr2=0x7f6aa80c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:23.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa7fff640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa8155950 0x7f6aa80c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:23.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa7fff640 1 -- 172.21.15.27:0/2220467271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa815a830 con 0x7f6aa80b5c00 2024-04-20T10:58:23.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aacaed640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa8155950 0x7f6aa80c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:58:23.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa7fff640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6aa80b5c00 0x7f6aa8151fb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6a9002bfd0 tx=0x7f6a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:23.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa4ff9640 1 -- 172.21.15.27:0/2220467271 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90034040 con 0x7f6aa80b5c00 2024-04-20T10:58:23.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa4ff9640 1 -- 172.21.15.27:0/2220467271 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90030680 con 0x7f6aa80b5c00 2024-04-20T10:58:23.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aa4ff9640 1 -- 172.21.15.27:0/2220467271 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90006e10 con 0x7f6aa80b5c00 2024-04-20T10:58:23.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa80c8000 con 0x7f6aa80b5c00 2024-04-20T10:58:23.751 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.749+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa80c82c0 con 0x7f6aa80b5c00 2024-04-20T10:58:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.750+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa804fcf0 con 0x7f6aa80b5c00 2024-04-20T10:58:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.751+0000 7f6aa4ff9640 1 -- 172.21.15.27:0/2220467271 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a90037070 con 0x7f6aa80b5c00 2024-04-20T10:58:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.751+0000 7f6aa4ff9640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6a8806c8f0 0x7f6a8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.751+0000 7f6aa4ff9640 1 -- 172.21.15.27:0/2220467271 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f6a900adec0 con 0x7f6aa80b5c00 2024-04-20T10:58:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.752+0000 7f6aa77fe640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6a8806c8f0 0x7f6a8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:23.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.752+0000 7f6aa77fe640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6a8806c8f0 0x7f6a8806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6aa81527f0 tx=0x7f6a98009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:23.755 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.756+0000 7f6aa4ff9640 1 -- 172.21.15.27:0/2220467271 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6a9007bb80 con 0x7f6aa80b5c00 2024-04-20T10:58:23.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.978+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aa80cd0a0 con 0x7f6aa80b5c00 2024-04-20T10:58:23.979 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.979+0000 7f6aa4ff9640 1 -- 172.21.15.27:0/2220467271 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6100 (secure 0 0 0) 0x7f6a9007fe80 con 0x7f6aa80b5c00 2024-04-20T10:58:23.979 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:23.979 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:22.329782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:58:23.979 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6a8806c8f0 msgr2=0x7f6a8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6a8806c8f0 0x7f6a8806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6aa81527f0 tx=0x7f6a98009040 comp rx=0 tx=0).stop 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6aa80b5c00 msgr2=0x7f6aa8151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6aa80b5c00 0x7f6aa8151fb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6a9002bfd0 tx=0x7f6a900046c0 comp rx=0 tx=0).stop 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 shutdown_connections 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6a8806c8f0 0x7f6a8806edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa8155950 0x7f6aa80c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6aa80cc330 0x7f6aa80c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 --2- 172.21.15.27:0/2220467271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6aa80b5c00 0x7f6aa8151fb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 >> 172.21.15.27:0/2220467271 conn(0x7f6aa80676c0 msgr2=0x7f6aa80c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:23.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 shutdown_connections 2024-04-20T10:58:23.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:23.981+0000 7f6aae550640 1 -- 172.21.15.27:0/2220467271 wait complete. 2024-04-20T10:58:23.991 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-04-20T10:58:23.992 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T10:58:23.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:24.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.092+0000 7f7cde477640 1 Processor -- start 2024-04-20T10:58:24.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.092+0000 7f7cde477640 1 -- start start 2024-04-20T10:58:24.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.093+0000 7f7cde477640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7cd8153480 0x7f7cd80c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.093+0000 7f7cde477640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd8070a60 con 0x7f7cd80cc2a0 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.093+0000 7f7cde477640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd8071770 con 0x7f7cd81592e0 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.093+0000 7f7cde477640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd8071520 con 0x7f7cd8153480 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.093+0000 7f7cdca14640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7cd80cc2a0 0x7f7cd80cf350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43044/0 (socket says 172.21.15.27:43044) 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.093+0000 7f7cdca14640 1 -- 172.21.15.27:0/1816270894 learned_addr learned my addr 172.21.15.27:0/1816270894 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.093+0000 7f7cd7fff640 1 --2- 172.21.15.27:0/1816270894 >> v2:172.21.15.137:3300/0 conn(0x7f7cd8153480 0x7f7cd80c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.093+0000 7f7cd6ffd640 1 -- 172.21.15.27:0/1816270894 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3989359417 0 0) 0x7f7cd8070a60 con 0x7f7cd80cc2a0 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.093+0000 7f7cd6ffd640 1 -- 172.21.15.27:0/1816270894 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cc4003520 con 0x7f7cd80cc2a0 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.094+0000 7f7cd7fff640 1 -- 172.21.15.27:0/1816270894 >> v1:172.21.15.137:6789/0 conn(0x7f7cd81592e0 legacy=0x7f7cd80c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.094+0000 7f7cd6ffd640 1 -- 172.21.15.27:0/1816270894 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4046182376 0 0) 0x7f7cd8071770 con 0x7f7cd81592e0 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.094+0000 7f7cd7fff640 1 -- 172.21.15.27:0/1816270894 >> v1:172.21.15.27:6789/0 conn(0x7f7cd80cc2a0 legacy=0x7f7cd80cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:24.094 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.094+0000 7f7cd7fff640 1 -- 172.21.15.27:0/1816270894 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd8070d00 con 0x7f7cd8153480 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.094+0000 7f7cd7fff640 1 --2- 172.21.15.27:0/1816270894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 0x7f7cd80c3f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00097a0 tx=0x7f7cc002b740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f251742a3a011e7b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cd6ffd640 1 -- 172.21.15.27:0/1816270894 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0037070 con 0x7f7cd8153480 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cd6ffd640 1 -- 172.21.15.27:0/1816270894 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc0030510 con 0x7f7cd8153480 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cd6ffd640 1 -- 172.21.15.27:0/1816270894 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00019f0 con 0x7f7cd8153480 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cde477640 1 -- 172.21.15.27:0/1816270894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 msgr2=0x7f7cd80c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cde477640 1 --2- 172.21.15.27:0/1816270894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 0x7f7cd80c3f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00097a0 tx=0x7f7cc002b740 comp rx=0 tx=0).stop 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cde477640 1 -- 172.21.15.27:0/1816270894 shutdown_connections 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cde477640 1 --2- 172.21.15.27:0/1816270894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 0x7f7cd80c3f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cde477640 1 -- 172.21.15.27:0/1816270894 >> 172.21.15.27:0/1816270894 conn(0x7f7cd8067ae0 msgr2=0x7f7cd8067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cde477640 1 -- 172.21.15.27:0/1816270894 shutdown_connections 2024-04-20T10:58:24.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.095+0000 7f7cde477640 1 -- 172.21.15.27:0/1816270894 wait complete. 2024-04-20T10:58:24.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.096+0000 7f7cde477640 1 Processor -- start 2024-04-20T10:58:24.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cde477640 1 -- start start 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cde477640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cd80cc2a0 0x7f7cd81ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cde477640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 0x7f7cd8151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cde477640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cd81592e0 0x7f7cd815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cde477640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd80cfa60 con 0x7f7cd81592e0 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cde477640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd8071a50 con 0x7f7cd8153480 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cde477640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd80cfbe0 con 0x7f7cd80cc2a0 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cdca14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cd81592e0 0x7f7cd815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cdca14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cd81592e0 0x7f7cd815cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47624/0 (socket says 172.21.15.27:47624) 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.097+0000 7f7cdca14640 1 -- 172.21.15.27:0/3790510050 learned_addr learned my addr 172.21.15.27:0/3790510050 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cd7fff640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cd80cc2a0 0x7f7cd81ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cd77fe640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 0x7f7cd8151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cdca14640 1 -- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cd80cc2a0 msgr2=0x7f7cd81ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:24.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cdca14640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cd80cc2a0 0x7f7cd81ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cdca14640 1 -- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 msgr2=0x7f7cd8151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cdca14640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 0x7f7cd8151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cdca14640 1 -- 172.21.15.27:0/3790510050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd8151530 con 0x7f7cd81592e0 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cd7fff640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cd80cc2a0 0x7f7cd81ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cd77fe640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 0x7f7cd8151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cdca14640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cd81592e0 0x7f7cd815cf50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000cb40 tx=0x7f7cd000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cd4ff9640 1 -- 172.21.15.27:0/3790510050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0018070 con 0x7f7cd81592e0 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cd4ff9640 1 -- 172.21.15.27:0/3790510050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd000a890 con 0x7f7cd81592e0 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd8152630 con 0x7f7cd81592e0 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cd4ff9640 1 -- 172.21.15.27:0/3790510050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd000ac60 con 0x7f7cd81592e0 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.098+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd8152bf0 con 0x7f7cd81592e0 2024-04-20T10:58:24.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.099+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd804fcf0 con 0x7f7cd81592e0 2024-04-20T10:58:24.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.100+0000 7f7cd4ff9640 1 -- 172.21.15.27:0/3790510050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cd001b070 con 0x7f7cd81592e0 2024-04-20T10:58:24.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.100+0000 7f7cd4ff9640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb806c920 0x7f7cb806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:24.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.100+0000 7f7cd4ff9640 1 -- 172.21.15.27:0/3790510050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f7cd008b5f0 con 0x7f7cd81592e0 2024-04-20T10:58:24.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.100+0000 7f7cd7fff640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb806c920 0x7f7cb806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:24.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.101+0000 7f7cd7fff640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb806c920 0x7f7cb806ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0034000 tx=0x7f7cc0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:24.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.104+0000 7f7cd4ff9640 1 -- 172.21.15.27:0/3790510050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7cd0059900 con 0x7f7cd81592e0 2024-04-20T10:58:24.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.329+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cd81481f0 con 0x7f7cd81592e0 2024-04-20T10:58:24.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.329+0000 7f7cd4ff9640 1 -- 172.21.15.27:0/3790510050 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6100 (secure 0 0 0) 0x7f7cd0089e20 con 0x7f7cd81592e0 2024-04-20T10:58:24.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:24.329 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:22.329782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:58:24.330 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-20T10:58:24.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb806c920 msgr2=0x7f7cb806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb806c920 0x7f7cb806ede0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0034000 tx=0x7f7cc0008030 comp rx=0 tx=0).stop 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cd81592e0 msgr2=0x7f7cd815cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cd81592e0 0x7f7cd815cf50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000cb40 tx=0x7f7cd000a060 comp rx=0 tx=0).stop 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 shutdown_connections 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb806c920 0x7f7cb806ede0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cd81592e0 0x7f7cd815cf50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cd8153480 0x7f7cd8151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 --2- 172.21.15.27:0/3790510050 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cd80cc2a0 0x7f7cd81ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.331+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 >> 172.21.15.27:0/3790510050 conn(0x7f7cd8067ae0 msgr2=0x7f7cd80c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.332+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 shutdown_connections 2024-04-20T10:58:24.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:24.332+0000 7f7cde477640 1 -- 172.21.15.27:0/3790510050 wait complete. 2024-04-20T10:58:24.343 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 24, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 24, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4272": { "addr": "172.21.15.27:6835/859496751", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6832", "nonce": 859496751, "type": "v2" }, { "addr": "172.21.15.27:6835", "nonce": 859496751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4272, "incarnation": 14, "join_fscid": -1, "name": "c", "rank": 3, "state": "up:active", "state_seq": 9 }, "gid_4275": { "addr": "172.21.15.27:6837/1680990583", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6834", "nonce": 1680990583, "type": "v2" }, { "addr": "172.21.15.27:6837", "nonce": 1680990583, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4275, "incarnation": 12, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4278": { "addr": "172.21.15.27:6836/4195543922", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6833", "nonce": 4195543922, "type": "v2" }, { "addr": "172.21.15.27:6836", "nonce": 4195543922, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4278, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4894": { "addr": "172.21.15.137:6838/2896327439", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6836", "nonce": 2896327439, "type": "v2" }, { "addr": "172.21.15.137:6838", "nonce": 2896327439, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4894, "incarnation": 24, "join_fscid": -1, "name": "b", "rank": 4, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 24, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T10:58:22.329782+0000", "qdb_cluster": [ 4272, 4275, 4278, 4293 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 4278, "mds_2": 4275, "mds_3": 4272, "mds_4": 4894 } } } ], "standbys": [] } 2024-04-20T10:58:26.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:26.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.462+0000 7f409889a640 1 Processor -- start 2024-04-20T10:58:26.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.462+0000 7f409889a640 1 -- start start 2024-04-20T10:58:26.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.462+0000 7f409889a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40940d0c90 0x7f40940d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:26.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.462+0000 7f409889a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409415b9c0 con 0x7f40940b54f0 2024-04-20T10:58:26.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.462+0000 7f409889a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409415bca0 con 0x7f4094155950 2024-04-20T10:58:26.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.462+0000 7f409889a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f409415bf80 con 0x7f40940d0c90 2024-04-20T10:58:26.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.463+0000 7f408bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f40940b54f0 0x7f40940b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47248/0 (socket says 172.21.15.27:47248) 2024-04-20T10:58:26.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.463+0000 7f408bfff640 1 -- 172.21.15.27:0/3311372475 learned_addr learned my addr 172.21.15.27:0/3311372475 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:26.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.463+0000 7f4092d9d640 1 --2- 172.21.15.27:0/3311372475 >> v2:172.21.15.27:3300/0 conn(0x7f40940d0c90 0x7f40940d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.463+0000 7f4091d9b640 1 -- 172.21.15.27:0/3311372475 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139710372 0 0) 0x7f409415b9c0 con 0x7f40940b54f0 2024-04-20T10:58:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.463+0000 7f4091d9b640 1 -- 172.21.15.27:0/3311372475 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4080003520 con 0x7f40940b54f0 2024-04-20T10:58:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.463+0000 7f4092d9d640 1 -- 172.21.15.27:0/3311372475 >> v1:172.21.15.137:6789/0 conn(0x7f4094155950 legacy=0x7f40940cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.463+0000 7f4091d9b640 1 -- 172.21.15.27:0/3311372475 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1866792139 0 0) 0x7f409415bca0 con 0x7f4094155950 2024-04-20T10:58:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.463+0000 7f4092d9d640 1 -- 172.21.15.27:0/3311372475 >> v1:172.21.15.27:6789/0 conn(0x7f40940b54f0 legacy=0x7f40940b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.465 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.463+0000 7f4092d9d640 1 -- 172.21.15.27:0/3311372475 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409415c260 con 0x7f40940d0c90 2024-04-20T10:58:26.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.464+0000 7f4092d9d640 1 --2- 172.21.15.27:0/3311372475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940d0c90 0x7f40940d10a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f408c009d90 tx=0x7f408c02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85de721586dbe7d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.464+0000 7f4091d9b640 1 -- 172.21.15.27:0/3311372475 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c039070 con 0x7f40940d0c90 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.464+0000 7f4091d9b640 1 -- 172.21.15.27:0/3311372475 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c02cd80 con 0x7f40940d0c90 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.464+0000 7f4091d9b640 1 -- 172.21.15.27:0/3311372475 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c034660 con 0x7f40940d0c90 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.464+0000 7f409889a640 1 -- 172.21.15.27:0/3311372475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940d0c90 msgr2=0x7f40940d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.464+0000 7f409889a640 1 --2- 172.21.15.27:0/3311372475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940d0c90 0x7f40940d10a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f408c009d90 tx=0x7f408c02bfd0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.465+0000 7f409889a640 1 -- 172.21.15.27:0/3311372475 shutdown_connections 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.465+0000 7f409889a640 1 --2- 172.21.15.27:0/3311372475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940d0c90 0x7f40940d10a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.465+0000 7f409889a640 1 -- 172.21.15.27:0/3311372475 >> 172.21.15.27:0/3311372475 conn(0x7f40940676c0 msgr2=0x7f4094067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.465+0000 7f409889a640 1 -- 172.21.15.27:0/3311372475 shutdown_connections 2024-04-20T10:58:26.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.465+0000 7f409889a640 1 -- 172.21.15.27:0/3311372475 wait complete. 2024-04-20T10:58:26.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.466+0000 7f409889a640 1 Processor -- start 2024-04-20T10:58:26.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.466+0000 7f409889a640 1 -- start start 2024-04-20T10:58:26.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409889a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940b54f0 0x7f40941568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409889a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40940d0c90 0x7f409415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409889a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4094155950 0x7f409415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409889a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f409415fb90 con 0x7f40940b54f0 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409889a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f40940d15e0 con 0x7f4094155950 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409889a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f409415fd10 con 0x7f40940d0c90 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940b54f0 0x7f40941568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940b54f0 0x7f40941568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43426/0 (socket says 172.21.15.27:43426) 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409259c640 1 -- 172.21.15.27:0/3989419732 learned_addr learned my addr 172.21.15.27:0/3989419732 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f4092d9d640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4094155950 0x7f409415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f408bfff640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40940d0c90 0x7f409415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:26.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409259c640 1 -- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40940d0c90 msgr2=0x7f409415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409259c640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40940d0c90 0x7f409415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409259c640 1 -- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4094155950 msgr2=0x7f409415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409259c640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4094155950 0x7f409415a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409259c640 1 -- 172.21.15.27:0/3989419732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40941596c0 con 0x7f40940b54f0 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409259c640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940b54f0 0x7f40941568b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f408400c4e0 tx=0x7f408400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f408affd640 1 -- 172.21.15.27:0/3989419732 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084017070 con 0x7f40940b54f0 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f408affd640 1 -- 172.21.15.27:0/3989419732 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40840035f0 con 0x7f40940b54f0 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f408affd640 1 -- 172.21.15.27:0/3989419732 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840039e0 con 0x7f40940b54f0 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.467+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40941f1bf0 con 0x7f40940b54f0 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.468+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4094159c40 con 0x7f40940b54f0 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.468+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40940686d0 con 0x7f40940b54f0 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.469+0000 7f408affd640 1 -- 172.21.15.27:0/3989419732 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f408401a070 con 0x7f40940b54f0 2024-04-20T10:58:26.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.469+0000 7f408affd640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f407406c8f0 0x7f407406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.469+0000 7f408affd640 1 -- 172.21.15.27:0/3989419732 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f408408af50 con 0x7f40940b54f0 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.470+0000 7f408bfff640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f407406c8f0 0x7f407406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:26.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.470+0000 7f408bfff640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f407406c8f0 0x7f407406edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4094155360 tx=0x7f407c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:26.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.474+0000 7f408affd640 1 -- 172.21.15.27:0/3989419732 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f40940686d0 con 0x7f40940b54f0 2024-04-20T10:58:26.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.699+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f409404fcf0 con 0x7f40940b54f0 2024-04-20T10:58:26.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.700+0000 7f408affd640 1 -- 172.21.15.27:0/3989419732 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+6100 (secure 0 0 0) 0x7f4084058b80 con 0x7f40940b54f0 2024-04-20T10:58:26.701 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:26.701 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:22.329782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:58:26.701 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 24 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f407406c8f0 msgr2=0x7f407406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f407406c8f0 0x7f407406edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4094155360 tx=0x7f407c009040 comp rx=0 tx=0).stop 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940b54f0 msgr2=0x7f40941568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940b54f0 0x7f40941568b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f408400c4e0 tx=0x7f408400c9a0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 shutdown_connections 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f407406c8f0 0x7f407406edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4094155950 0x7f409415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40940d0c90 0x7f409415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 --2- 172.21.15.27:0/3989419732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40940b54f0 0x7f40941568b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 >> 172.21.15.27:0/3989419732 conn(0x7f40940676c0 msgr2=0x7f409406e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 shutdown_connections 2024-04-20T10:58:26.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:26.702+0000 7f409889a640 1 -- 172.21.15.27:0/3989419732 wait complete. 2024-04-20T10:58:26.713 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:58:28.714 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:28.827 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.827+0000 7f6675a8c640 1 Processor -- start 2024-04-20T10:58:28.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.827+0000 7f6675a8c640 1 -- start start 2024-04-20T10:58:28.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f6675a8c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6670155950 0x7f66700cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:28.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f6675a8c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66700b5c00 0x7f66700d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:28.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f6675a8c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667015b9c0 con 0x7f66700cc330 2024-04-20T10:58:28.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f6675a8c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f667015bca0 con 0x7f66700b5c00 2024-04-20T10:58:28.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f6675a8c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f667015bf80 con 0x7f6670155950 2024-04-20T10:58:28.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f666ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66700b5c00 0x7f66700d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:28.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f666ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66700b5c00 0x7f66700d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43442/0 (socket says 172.21.15.27:43442) 2024-04-20T10:58:28.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f666ffff640 1 -- 172.21.15.27:0/2128246523 learned_addr learned my addr 172.21.15.27:0/2128246523 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:28.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f666f7fe640 1 --2- 172.21.15.27:0/2128246523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6670155950 0x7f66700cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:28.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f666ffff640 1 -- 172.21.15.27:0/2128246523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6670155950 msgr2=0x7f66700cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:28.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f666ffff640 1 --2- 172.21.15.27:0/2128246523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6670155950 0x7f66700cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:28.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f666ffff640 1 -- 172.21.15.27:0/2128246523 >> v1:172.21.15.137:6789/0 conn(0x7f66700cc330 legacy=0x7f66700b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:28.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.828+0000 7f666ffff640 1 -- 172.21.15.27:0/2128246523 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667015c260 con 0x7f66700b5c00 2024-04-20T10:58:28.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.829+0000 7f666ffff640 1 --2- 172.21.15.27:0/2128246523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700b5c00 0x7f66700d0c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6668009e50 tx=0x7f666802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971f6e1dc471953b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:28.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.829+0000 7f666e7fc640 1 -- 172.21.15.27:0/2128246523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666802cb90 con 0x7f66700b5c00 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.829+0000 7f666e7fc640 1 -- 172.21.15.27:0/2128246523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666802cd30 con 0x7f66700b5c00 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.829+0000 7f666e7fc640 1 -- 172.21.15.27:0/2128246523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668034610 con 0x7f66700b5c00 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.830+0000 7f6675a8c640 1 -- 172.21.15.27:0/2128246523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700b5c00 msgr2=0x7f66700d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.830+0000 7f6675a8c640 1 --2- 172.21.15.27:0/2128246523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700b5c00 0x7f66700d0c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6668009e50 tx=0x7f666802bff0 comp rx=0 tx=0).stop 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.830+0000 7f6675a8c640 1 -- 172.21.15.27:0/2128246523 shutdown_connections 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.830+0000 7f6675a8c640 1 --2- 172.21.15.27:0/2128246523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700b5c00 0x7f66700d0c90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.830+0000 7f6675a8c640 1 --2- 172.21.15.27:0/2128246523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6670155950 0x7f66700cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.830+0000 7f6675a8c640 1 -- 172.21.15.27:0/2128246523 >> 172.21.15.27:0/2128246523 conn(0x7f66700676c0 msgr2=0x7f6670067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.830+0000 7f6675a8c640 1 -- 172.21.15.27:0/2128246523 shutdown_connections 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.830+0000 7f6675a8c640 1 -- 172.21.15.27:0/2128246523 wait complete. 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.831+0000 7f6675a8c640 1 Processor -- start 2024-04-20T10:58:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.831+0000 7f6675a8c640 1 -- start start 2024-04-20T10:58:28.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f6675a8c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66700b5c00 0x7f6670154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:28.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f6675a8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66700cc330 0x7f66701e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:28.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f6675a8c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6670155950 0x7f66700c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:28.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f6675a8c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f667015fd10 con 0x7f6670155950 2024-04-20T10:58:28.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f6675a8c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f667015fb90 con 0x7f66700cc330 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f6675a8c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6670155d30 con 0x7f66700b5c00 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6670155950 0x7f66700c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6670155950 0x7f66700c6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43450/0 (socket says 172.21.15.27:43450) 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666ffff640 1 -- 172.21.15.27:0/2632224165 learned_addr learned my addr 172.21.15.27:0/2632224165 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666f7fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66700b5c00 0x7f6670154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666effd640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66700cc330 0x7f66701e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666ffff640 1 -- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66700b5c00 msgr2=0x7f6670154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666ffff640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66700b5c00 0x7f6670154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666ffff640 1 -- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66700cc330 msgr2=0x7f66701e4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666ffff640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66700cc330 0x7f66701e4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666ffff640 1 -- 172.21.15.27:0/2632224165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66701f4b00 con 0x7f6670155950 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.832+0000 7f666ffff640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6670155950 0x7f66700c6bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6668009e50 tx=0x7f6668004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.833+0000 7f6657fff640 1 -- 172.21.15.27:0/2632224165 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668043020 con 0x7f6670155950 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.833+0000 7f6657fff640 1 -- 172.21.15.27:0/2632224165 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666803e450 con 0x7f6670155950 2024-04-20T10:58:28.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.833+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667015e340 con 0x7f6670155950 2024-04-20T10:58:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.833+0000 7f6657fff640 1 -- 172.21.15.27:0/2632224165 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668009510 con 0x7f6670155950 2024-04-20T10:58:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.833+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66701f5080 con 0x7f6670155950 2024-04-20T10:58:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.834+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667004fcf0 con 0x7f6670155950 2024-04-20T10:58:28.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.834+0000 7f6657fff640 1 -- 172.21.15.27:0/2632224165 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6668046020 con 0x7f6670155950 2024-04-20T10:58:28.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.835+0000 7f6657fff640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f665c06c920 0x7f665c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:28.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.835+0000 7f6657fff640 1 -- 172.21.15.27:0/2632224165 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f66680ade10 con 0x7f6670155950 2024-04-20T10:58:28.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.835+0000 7f666f7fe640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f665c06c920 0x7f665c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:28.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.835+0000 7f666f7fe640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f665c06c920 0x7f665c06ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6660009aa0 tx=0x7f6660009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:28.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:28.839+0000 7f6657fff640 1 -- 172.21.15.27:0/2632224165 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f666804d070 con 0x7f6670155950 2024-04-20T10:58:29.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.065+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66700b3dd0 con 0x7f6670155950 2024-04-20T10:58:29.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.065+0000 7f6657fff640 1 -- 172.21.15.27:0/2632224165 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+6102 (secure 0 0 0) 0x7f666807bad0 con 0x7f6670155950 2024-04-20T10:58:29.066 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:29.066 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:27.340030+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:resolve","state_seq":11,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:58:29.066 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-04-20T10:58:29.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f665c06c920 msgr2=0x7f665c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:29.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f665c06c920 0x7f665c06ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6660009aa0 tx=0x7f6660009190 comp rx=0 tx=0).stop 2024-04-20T10:58:29.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6670155950 msgr2=0x7f66700c6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:29.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6670155950 0x7f66700c6bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6668009e50 tx=0x7f6668004a40 comp rx=0 tx=0).stop 2024-04-20T10:58:29.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 shutdown_connections 2024-04-20T10:58:29.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f665c06c920 0x7f665c06ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:29.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6670155950 0x7f66700c6bf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:29.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66700cc330 0x7f66701e4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:29.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 --2- 172.21.15.27:0/2632224165 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66700b5c00 0x7f6670154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:29.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 >> 172.21.15.27:0/2632224165 conn(0x7f66700676c0 msgr2=0x7f66700d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:29.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 shutdown_connections 2024-04-20T10:58:29.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:29.067+0000 7f6675a8c640 1 -- 172.21.15.27:0/2632224165 wait complete. 2024-04-20T10:58:29.079 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:58:31.080 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:31.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.193+0000 7f223423a640 1 Processor -- start 2024-04-20T10:58:31.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.193+0000 7f223423a640 1 -- start start 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.194+0000 7f223423a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f222c155690 0x7f222c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.194+0000 7f223423a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f222c06d770 0x7f222c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.194+0000 7f223423a640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222c0cf580 con 0x7f222c0cf080 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.194+0000 7f223423a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f222c06ca40 con 0x7f222c155690 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.194+0000 7f223423a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f222c0ca2b0 con 0x7f222c06d770 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.194+0000 7f2231fd6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f222c155690 0x7f222c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.195+0000 7f2231fd6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f222c155690 0x7f222c15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43454/0 (socket says 172.21.15.27:43454) 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.195+0000 7f2231fd6640 1 -- 172.21.15.27:0/2611266713 learned_addr learned my addr 172.21.15.27:0/2611266713 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.195+0000 7f22317d5640 1 --2- 172.21.15.27:0/2611266713 >> v2:172.21.15.137:3300/0 conn(0x7f222c06d770 0x7f222c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.195+0000 7f2231fd6640 1 -- 172.21.15.27:0/2611266713 >> v2:172.21.15.137:3300/0 conn(0x7f222c06d770 msgr2=0x7f222c0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.195+0000 7f2231fd6640 1 --2- 172.21.15.27:0/2611266713 >> v2:172.21.15.137:3300/0 conn(0x7f222c06d770 0x7f222c0ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.195+0000 7f2231fd6640 1 -- 172.21.15.27:0/2611266713 >> v1:172.21.15.137:6789/0 conn(0x7f222c0cf080 legacy=0x7f222c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:31.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.195+0000 7f2231fd6640 1 -- 172.21.15.27:0/2611266713 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c0cf860 con 0x7f222c155690 2024-04-20T10:58:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f2231fd6640 1 --2- 172.21.15.27:0/2611266713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c155690 0x7f222c15b3f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f222c0cb610 tx=0x7f222802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0ff28565b928c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f2230fd4640 1 -- 172.21.15.27:0/2611266713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228034070 con 0x7f222c155690 2024-04-20T10:58:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f2230fd4640 1 -- 172.21.15.27:0/2611266713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2228030860 con 0x7f222c155690 2024-04-20T10:58:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f2230fd4640 1 -- 172.21.15.27:0/2611266713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228030c30 con 0x7f222c155690 2024-04-20T10:58:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f223423a640 1 -- 172.21.15.27:0/2611266713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c155690 msgr2=0x7f222c15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f223423a640 1 --2- 172.21.15.27:0/2611266713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c155690 0x7f222c15b3f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f222c0cb610 tx=0x7f222802b9a0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f223423a640 1 -- 172.21.15.27:0/2611266713 shutdown_connections 2024-04-20T10:58:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f223423a640 1 --2- 172.21.15.27:0/2611266713 >> v2:172.21.15.137:3300/0 conn(0x7f222c06d770 0x7f222c0ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f223423a640 1 --2- 172.21.15.27:0/2611266713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c155690 0x7f222c15b3f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f223423a640 1 -- 172.21.15.27:0/2611266713 >> 172.21.15.27:0/2611266713 conn(0x7f222c06c070 msgr2=0x7f222c0676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:31.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.196+0000 7f223423a640 1 -- 172.21.15.27:0/2611266713 shutdown_connections 2024-04-20T10:58:31.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.197+0000 7f223423a640 1 -- 172.21.15.27:0/2611266713 wait complete. 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.198+0000 7f223423a640 1 Processor -- start 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.198+0000 7f223423a640 1 -- start start 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.198+0000 7f223423a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c06d770 0x7f222c1f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.198+0000 7f223423a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f222c0cf080 0x7f222c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.198+0000 7f223423a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c155690 0x7f222c1f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.198+0000 7f223423a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f222c06d250 con 0x7f222c06d770 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.198+0000 7f223423a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f222c06cf70 con 0x7f222c155690 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.198+0000 7f223423a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f222c155a70 con 0x7f222c0cf080 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f2231fd6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c06d770 0x7f222c1f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f2231fd6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c06d770 0x7f222c1f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43468/0 (socket says 172.21.15.27:43468) 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f2231fd6640 1 -- 172.21.15.27:0/3792093796 learned_addr learned my addr 172.21.15.27:0/3792093796 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:31.201 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f22317d5640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f222c0cf080 0x7f222c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f22327d7640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c155690 0x7f222c1f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f2231fd6640 1 -- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f222c0cf080 msgr2=0x7f222c14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f2231fd6640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f222c0cf080 0x7f222c14de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f2231fd6640 1 -- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c155690 msgr2=0x7f222c1f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f2231fd6640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c155690 0x7f222c1f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f2231fd6640 1 -- 172.21.15.27:0/3792093796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c1f1700 con 0x7f222c06d770 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f2231fd6640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c06d770 0x7f222c1f1ec0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f222802b9e0 tx=0x7f22280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f22227fc640 1 -- 172.21.15.27:0/3792093796 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228034040 con 0x7f222c06d770 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f22227fc640 1 -- 172.21.15.27:0/3792093796 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2228039d90 con 0x7f222c06d770 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222c14eaa0 con 0x7f222c06d770 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f22227fc640 1 -- 172.21.15.27:0/3792093796 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228043610 con 0x7f222c06d770 2024-04-20T10:58:31.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.199+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222c151a90 con 0x7f222c06d770 2024-04-20T10:58:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.200+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222c1f14f0 con 0x7f222c06d770 2024-04-20T10:58:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.201+0000 7f22227fc640 1 -- 172.21.15.27:0/3792093796 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2228037070 con 0x7f222c06d770 2024-04-20T10:58:31.203 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.201+0000 7f22227fc640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f221006c920 0x7f221006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.201+0000 7f22227fc640 1 -- 172.21.15.27:0/3792093796 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f22280adc30 con 0x7f222c06d770 2024-04-20T10:58:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.201+0000 7f22317d5640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f221006c920 0x7f221006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:31.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.202+0000 7f22317d5640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f221006c920 0x7f221006ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f222c157760 tx=0x7f221c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:31.206 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.206+0000 7f22227fc640 1 -- 172.21.15.27:0/3792093796 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f222803e070 con 0x7f222c06d770 2024-04-20T10:58:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.427+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f222c04fcf0 con 0x7f222c06d770 2024-04-20T10:58:31.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.428+0000 7f22227fc640 1 -- 172.21.15.27:0/3792093796 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+6102 (secure 0 0 0) 0x7f222807ba10 con 0x7f222c06d770 2024-04-20T10:58:31.429 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:31.429 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:27.340030+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:resolve","state_seq":11,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:58:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2024-04-20T10:58:31.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f221006c920 msgr2=0x7f221006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f221006c920 0x7f221006ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f222c157760 tx=0x7f221c009190 comp rx=0 tx=0).stop 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c06d770 msgr2=0x7f222c1f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c06d770 0x7f222c1f1ec0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f222802b9e0 tx=0x7f22280046c0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 shutdown_connections 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f221006c920 0x7f221006ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f222c155690 0x7f222c1f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f222c0cf080 0x7f222c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 --2- 172.21.15.27:0/3792093796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f222c06d770 0x7f222c1f1ec0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.429+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 >> 172.21.15.27:0/3792093796 conn(0x7f222c06c070 msgr2=0x7f222c0c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.430+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 shutdown_connections 2024-04-20T10:58:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:31.430+0000 7f223423a640 1 -- 172.21.15.27:0/3792093796 wait complete. 2024-04-20T10:58:31.441 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:58:33.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:33.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.554+0000 7fb7d3606640 1 Processor -- start 2024-04-20T10:58:33.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.554+0000 7fb7d3606640 1 -- start start 2024-04-20T10:58:33.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.554+0000 7fb7d3606640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7cc1556f0 0x7fb7cc06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:33.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7d3606640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7cc15ce40 con 0x7fb7cc15f710 2024-04-20T10:58:33.558 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7d3606640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7cc15ec70 con 0x7fb7cc15c9d0 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7d3606640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fb7cc06ead0 con 0x7fb7cc1556f0 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7d0ba1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb7cc15f710 0x7fb7cc0cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47254/0 (socket says 172.21.15.27:47254) 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7d0ba1640 1 -- 172.21.15.27:0/855932547 learned_addr learned my addr 172.21.15.27:0/855932547 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 633486172 0 0) 0x7fb7cc15ce40 con 0x7fb7cc15f710 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7d13a2640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7cc1556f0 0x7fb7cc06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7b8003520 con 0x7fb7cc15f710 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3829599945 0 0) 0x7fb7b8003520 con 0x7fb7cc15f710 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7cc1556f0 msgr2=0x7fb7cc06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7cc1556f0 0x7fb7cc06ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 >> v1:172.21.15.137:6789/0 conn(0x7fb7cc15c9d0 legacy=0x7fb7cc165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc06e510 con 0x7fb7cc15f710 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7fb7bc001670 con 0x7fb7cc15f710 2024-04-20T10:58:33.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 >> v1:172.21.15.27:6789/0 conn(0x7fb7cc15f710 legacy=0x7fb7cc0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7b80039e0 0x7fb7b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7b80043b0 0x7fb7b8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7cc1556f0 0x7fb7b80050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc06ead0 con 0x7fb7b80043b0 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b800b9f0 con 0x7fb7b80039e0 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.555+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b800bbf0 con 0x7fb7cc1556f0 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.556+0000 7fb7d13a2640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7b80039e0 0x7fb7b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.556+0000 7fb7d0ba1640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7b80043b0 0x7fb7b8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.556+0000 7fb7d1ba3640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7cc1556f0 0x7fb7b80050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.556+0000 7fb7d13a2640 1 -- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7cc1556f0 msgr2=0x7fb7b80050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.556+0000 7fb7d13a2640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7cc1556f0 0x7fb7b80050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:33.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.556+0000 7fb7d13a2640 1 -- 172.21.15.27:0/855932547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7b80043b0 msgr2=0x7fb7b8004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.556+0000 7fb7d13a2640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7b80043b0 0x7fb7b8004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.556+0000 7fb7d13a2640 1 -- 172.21.15.27:0/855932547 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b8003520 con 0x7fb7b80039e0 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.556+0000 7fb7d0ba1640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7b80043b0 0x7fb7b8004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7d13a2640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7b80039e0 0x7fb7b8003de0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009ac0 tx=0x7fb7b8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b76b21eada191b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8039070 con 0x7fb7b80039e0 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7c802c6b0 con 0x7fb7b80039e0 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7c3fff640 1 -- 172.21.15.27:0/855932547 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8009690 con 0x7fb7b80039e0 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7d3606640 1 -- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7b80039e0 msgr2=0x7fb7b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7d3606640 1 --2- 172.21.15.27:0/855932547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7b80039e0 0x7fb7b8003de0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009ac0 tx=0x7fb7b8005830 comp rx=0 tx=0).stop 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7d1ba3640 1 -- 172.21.15.27:0/855932547 reap_dead start 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7d3606640 1 -- 172.21.15.27:0/855932547 shutdown_connections 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7d3606640 1 -- 172.21.15.27:0/855932547 >> 172.21.15.27:0/855932547 conn(0x7fb7cc067ae0 msgr2=0x7fb7cc067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7d3606640 1 -- 172.21.15.27:0/855932547 shutdown_connections 2024-04-20T10:58:33.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.557+0000 7fb7d3606640 1 -- 172.21.15.27:0/855932547 wait complete. 2024-04-20T10:58:33.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d3606640 1 Processor -- start 2024-04-20T10:58:33.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d3606640 1 -- start start 2024-04-20T10:58:33.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d3606640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7b80039e0 0x7fb7cc0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:33.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d3606640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7cc0c4a20 0x7fb7cc14fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:33.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d3606640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc0cbbe0 0x7fb7cc1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d3606640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc15f1f0 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d3606640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc15ef10 con 0x7fb7cc0cbbe0 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d3606640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fb7cc0c9800 con 0x7fb7b80039e0 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d0ba1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7cc0c4a20 0x7fb7cc14fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d0ba1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7cc0c4a20 0x7fb7cc14fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43486/0 (socket says 172.21.15.27:43486) 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d0ba1640 1 -- 172.21.15.27:0/852759747 learned_addr learned my addr 172.21.15.27:0/852759747 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.559+0000 7fb7d13a2640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7b80039e0 0x7fb7cc0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7d1ba3640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc0cbbe0 0x7fb7cc1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7d0ba1640 1 -- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7b80039e0 msgr2=0x7fb7cc0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7d0ba1640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7b80039e0 0x7fb7cc0cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7d0ba1640 1 -- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc0cbbe0 msgr2=0x7fb7cc1566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7d0ba1640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc0cbbe0 0x7fb7cc1566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7d0ba1640 1 -- 172.21.15.27:0/852759747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc158950 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7d0ba1640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7cc0c4a20 0x7fb7cc14fee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007290 tx=0x7fb7bc007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7c1ffb640 1 -- 172.21.15.27:0/852759747 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00f960 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7c1ffb640 1 -- 172.21.15.27:0/852759747 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7bc018070 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc1f4ea0 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7c1ffb640 1 -- 172.21.15.27:0/852759747 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00fd00 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.560+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc158bc0 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.561+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7cc0688e0 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.562+0000 7fb7c1ffb640 1 -- 172.21.15.27:0/852759747 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7bc01d460 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.562+0000 7fb7c1ffb640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb7b006c920 0x7fb7b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.562+0000 7fb7c1ffb640 1 -- 172.21.15.27:0/852759747 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7fb7bc08bb60 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.562+0000 7fb7d13a2640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb7b006c920 0x7fb7b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:33.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.563+0000 7fb7d13a2640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb7b006c920 0x7fb7b006ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8036000 tx=0x7fb7c8030000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:33.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.567+0000 7fb7c1ffb640 1 -- 172.21.15.27:0/852759747 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb7bc059970 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.789+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7cc04fcf0 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.789+0000 7fb7c1ffb640 1 -- 172.21.15.27:0/852759747 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6104 (secure 0 0 0) 0x7fb7bc05dc70 con 0x7fb7cc0c4a20 2024-04-20T10:58:33.790 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:33.791 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:33.191317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:58:33.791 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-04-20T10:58:33.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.791+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb7b006c920 msgr2=0x7fb7b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.791+0000 7fb7d3606640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb7b006c920 0x7fb7b006ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8036000 tx=0x7fb7c8030000 comp rx=0 tx=0).stop 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.791+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7cc0c4a20 msgr2=0x7fb7cc14fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.791+0000 7fb7d3606640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7cc0c4a20 0x7fb7cc14fee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007290 tx=0x7fb7bc007750 comp rx=0 tx=0).stop 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.791+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 shutdown_connections 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.791+0000 7fb7d3606640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb7b006c920 0x7fb7b006ede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.791+0000 7fb7d3606640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7cc0cbbe0 0x7fb7cc1566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.792+0000 7fb7d3606640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7cc0c4a20 0x7fb7cc14fee0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.792+0000 7fb7d3606640 1 --2- 172.21.15.27:0/852759747 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb7b80039e0 0x7fb7cc0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.792+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 >> 172.21.15.27:0/852759747 conn(0x7fb7cc067ae0 msgr2=0x7fb7cc067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.792+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 shutdown_connections 2024-04-20T10:58:33.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:33.792+0000 7fb7d3606640 1 -- 172.21.15.27:0/852759747 wait complete. 2024-04-20T10:58:33.803 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:58:35.804 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:35.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.907+0000 7f86cee43640 1 Processor -- start 2024-04-20T10:58:35.908 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.907+0000 7f86cee43640 1 -- start start 2024-04-20T10:58:35.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86cee43640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f86c8152260 0x7f86c80719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:35.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86cee43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c8152640 0x7f86c80c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:35.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86cee43640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86c815ba80 con 0x7f86c80cd6f0 2024-04-20T10:58:35.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86cee43640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f86c815bd60 con 0x7f86c8152260 2024-04-20T10:58:35.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86cee43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f86c815c040 con 0x7f86c8152640 2024-04-20T10:58:35.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86cd3e0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f86c80cd6f0 0x7f86c8158760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48550/0 (socket says 172.21.15.27:48550) 2024-04-20T10:58:35.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86cd3e0640 1 -- 172.21.15.27:0/2194870120 learned_addr learned my addr 172.21.15.27:0/2194870120 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:35.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86ccbdf640 1 --2- 172.21.15.27:0/2194870120 >> v2:172.21.15.137:3300/0 conn(0x7f86c8152260 0x7f86c80719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86c7fff640 1 --2- 172.21.15.27:0/2194870120 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c8152640 0x7f86c80c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86c77fe640 1 -- 172.21.15.27:0/2194870120 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385916325 0 0) 0x7f86c815ba80 con 0x7f86c80cd6f0 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86c77fe640 1 -- 172.21.15.27:0/2194870120 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86b8003520 con 0x7f86c80cd6f0 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86ccbdf640 1 -- 172.21.15.27:0/2194870120 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c8152640 msgr2=0x7f86c80c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86ccbdf640 1 --2- 172.21.15.27:0/2194870120 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c8152640 0x7f86c80c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86ccbdf640 1 -- 172.21.15.27:0/2194870120 >> v1:172.21.15.27:6789/0 conn(0x7f86c80cd6f0 legacy=0x7f86c8158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86c77fe640 1 -- 172.21.15.27:0/2194870120 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1055859606 0 0) 0x7f86b8003520 con 0x7f86c80cd6f0 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86ccbdf640 1 -- 172.21.15.27:0/2194870120 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c815c320 con 0x7f86c8152260 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.908+0000 7f86c7fff640 1 --2- 172.21.15.27:0/2194870120 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c8152640 0x7f86c80c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.909+0000 7f86ccbdf640 1 --2- 172.21.15.27:0/2194870120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86c8152260 0x7f86c80719c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0097a0 tx=0x7f86bc02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542004ffb3d1cee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.909+0000 7f86c77fe640 1 -- 172.21.15.27:0/2194870120 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc038070 con 0x7f86c8152260 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.909+0000 7f86c77fe640 1 -- 172.21.15.27:0/2194870120 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86bc031500 con 0x7f86c8152260 2024-04-20T10:58:35.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.909+0000 7f86c77fe640 1 -- 172.21.15.27:0/2194870120 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c00019f0 con 0x7f86c8152260 2024-04-20T10:58:35.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.910+0000 7f86cee43640 1 -- 172.21.15.27:0/2194870120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86c8152260 msgr2=0x7f86c80719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:35.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.910+0000 7f86cee43640 1 --2- 172.21.15.27:0/2194870120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86c8152260 0x7f86c80719c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0097a0 tx=0x7f86bc02b820 comp rx=0 tx=0).stop 2024-04-20T10:58:35.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.910+0000 7f86cee43640 1 -- 172.21.15.27:0/2194870120 shutdown_connections 2024-04-20T10:58:35.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.910+0000 7f86cee43640 1 --2- 172.21.15.27:0/2194870120 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c8152640 0x7f86c80c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:35.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.910+0000 7f86cee43640 1 --2- 172.21.15.27:0/2194870120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86c8152260 0x7f86c80719c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:35.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.910+0000 7f86cee43640 1 -- 172.21.15.27:0/2194870120 >> 172.21.15.27:0/2194870120 conn(0x7f86c80ccbd0 msgr2=0x7f86c80ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:35.914 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.910+0000 7f86cee43640 1 -- 172.21.15.27:0/2194870120 shutdown_connections 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.910+0000 7f86cee43640 1 -- 172.21.15.27:0/2194870120 wait complete. 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.911+0000 7f86cee43640 1 Processor -- start 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.911+0000 7f86cee43640 1 -- start start 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86cee43640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c80cd6f0 0x7f86c80cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86cee43640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c8152260 0x7f86c81595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86cee43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86c8152640 0x7f86c8159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86cee43640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f86c815c600 con 0x7f86c8152260 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86cee43640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86c8074260 con 0x7f86c8152640 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86cee43640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f86c815b720 con 0x7f86c80cd6f0 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86c7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c8152260 0x7f86c81595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86c7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c8152260 0x7f86c81595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33520/0 (socket says 172.21.15.27:33520) 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86c7fff640 1 -- 172.21.15.27:0/245485145 learned_addr learned my addr 172.21.15.27:0/245485145 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86cd3e0640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86c8152640 0x7f86c8159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:35.915 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86ccbdf640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c80cd6f0 0x7f86c80cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86c7fff640 1 -- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c80cd6f0 msgr2=0x7f86c80cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86c7fff640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c80cd6f0 0x7f86c80cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86c7fff640 1 -- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86c8152640 msgr2=0x7f86c8159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86c7fff640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86c8152640 0x7f86c8159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.912+0000 7f86c7fff640 1 -- 172.21.15.27:0/245485145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c81480b0 con 0x7f86c8152260 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.913+0000 7f86c7fff640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c8152260 0x7f86c81595d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f86b400ec10 tx=0x7f86b400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.913+0000 7f86c57fa640 1 -- 172.21.15.27:0/245485145 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4017070 con 0x7f86c8152260 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.913+0000 7f86c57fa640 1 -- 172.21.15.27:0/245485145 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b400cc30 con 0x7f86c8152260 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.913+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c8147af0 con 0x7f86c8152260 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.913+0000 7f86c57fa640 1 -- 172.21.15.27:0/245485145 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4013470 con 0x7f86c8152260 2024-04-20T10:58:35.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.913+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c8146340 con 0x7f86c8152260 2024-04-20T10:58:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.914+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c806ca20 con 0x7f86c8152260 2024-04-20T10:58:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.914+0000 7f86c57fa640 1 -- 172.21.15.27:0/245485145 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86b401a070 con 0x7f86c8152260 2024-04-20T10:58:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.915+0000 7f86c57fa640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f86ac06c920 0x7f86ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.915+0000 7f86c57fa640 1 -- 172.21.15.27:0/245485145 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f86b408bf60 con 0x7f86c8152260 2024-04-20T10:58:35.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.915+0000 7f86ccbdf640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f86ac06c920 0x7f86ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:35.918 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.915+0000 7f86ccbdf640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f86ac06c920 0x7f86ac06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86bc035000 tx=0x7f86bc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:35.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:35.919+0000 7f86c57fa640 1 -- 172.21.15.27:0/245485145 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f86c806ca20 con 0x7f86c8152260 2024-04-20T10:58:36.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.142+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86c804fcf0 con 0x7f86c8152260 2024-04-20T10:58:36.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.142+0000 7f86c57fa640 1 -- 172.21.15.27:0/245485145 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+6104 (secure 0 0 0) 0x7f86b4059b10 con 0x7f86c8152260 2024-04-20T10:58:36.143 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:36.143 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:33.191317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293]},"id":1}]} 2024-04-20T10:58:36.144 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2024-04-20T10:58:36.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f86ac06c920 msgr2=0x7f86ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:36.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f86ac06c920 0x7f86ac06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f86bc035000 tx=0x7f86bc008030 comp rx=0 tx=0).stop 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c8152260 msgr2=0x7f86c81595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c8152260 0x7f86c81595d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f86b400ec10 tx=0x7f86b400c520 comp rx=0 tx=0).stop 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 shutdown_connections 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f86ac06c920 0x7f86ac06ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86c8152640 0x7f86c8159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86c8152260 0x7f86c81595d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 --2- 172.21.15.27:0/245485145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f86c80cd6f0 0x7f86c80cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.144+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 >> 172.21.15.27:0/245485145 conn(0x7f86c80ccbd0 msgr2=0x7f86c8158760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.145+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 shutdown_connections 2024-04-20T10:58:36.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:36.145+0000 7f86cee43640 1 -- 172.21.15.27:0/245485145 wait complete. 2024-04-20T10:58:36.156 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 26, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 26, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4272": { "addr": "172.21.15.27:6835/859496751", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6832", "nonce": 859496751, "type": "v2" }, { "addr": "172.21.15.27:6835", "nonce": 859496751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4272, "incarnation": 14, "join_fscid": -1, "name": "c", "rank": 3, "state": "up:active", "state_seq": 9 }, "gid_4275": { "addr": "172.21.15.27:6837/1680990583", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6834", "nonce": 1680990583, "type": "v2" }, { "addr": "172.21.15.27:6837", "nonce": 1680990583, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4275, "incarnation": 12, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4278": { "addr": "172.21.15.27:6836/4195543922", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6833", "nonce": 4195543922, "type": "v2" }, { "addr": "172.21.15.27:6836", "nonce": 4195543922, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4278, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4894": { "addr": "172.21.15.137:6838/2896327439", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6836", "nonce": 2896327439, "type": "v2" }, { "addr": "172.21.15.137:6838", "nonce": 2896327439, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4894, "incarnation": 24, "join_fscid": -1, "name": "b", "rank": 4, "state": "up:reconnect", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 24, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T10:58:33.191317+0000", "qdb_cluster": [ 4272, 4275, 4278, 4293 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 4278, "mds_2": 4275, "mds_3": 4272, "mds_4": 4894 } } } ], "standbys": [] } 2024-04-20T10:58:38.157 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.271+0000 7f9a28732640 1 Processor -- start 2024-04-20T10:58:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.271+0000 7f9a28732640 1 -- start start 2024-04-20T10:58:38.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a28732640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9a200cbf50 0x7f9a200c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:38.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a28732640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f9a200c6500 0x7f9a200d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:38.273 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a28732640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a200cd290 con 0x7f9a20155800 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a28732640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f9a200ce610 con 0x7f9a200cbf50 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a28732640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f9a200ce180 con 0x7f9a200c6500 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a264ce640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9a20155800 0x7f9a2015bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48560/0 (socket says 172.21.15.27:48560) 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a264ce640 1 -- 172.21.15.27:0/2624922029 learned_addr learned my addr 172.21.15.27:0/2624922029 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a25ccd640 1 --2- 172.21.15.27:0/2624922029 >> v2:172.21.15.27:3300/0 conn(0x7f9a200cbf50 0x7f9a200c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a26ccf640 1 --2- 172.21.15.27:0/2624922029 >> v2:172.21.15.137:3300/0 conn(0x7f9a200c6500 0x7f9a200d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a254cc640 1 -- 172.21.15.27:0/2624922029 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2929657621 0 0) 0x7f9a200cd290 con 0x7f9a20155800 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a254cc640 1 -- 172.21.15.27:0/2624922029 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a08003520 con 0x7f9a20155800 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a25ccd640 1 -- 172.21.15.27:0/2624922029 >> v2:172.21.15.137:3300/0 conn(0x7f9a200c6500 msgr2=0x7f9a200d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a25ccd640 1 --2- 172.21.15.27:0/2624922029 >> v2:172.21.15.137:3300/0 conn(0x7f9a200c6500 0x7f9a200d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a25ccd640 1 -- 172.21.15.27:0/2624922029 >> v1:172.21.15.27:6789/0 conn(0x7f9a20155800 legacy=0x7f9a2015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.272+0000 7f9a25ccd640 1 -- 172.21.15.27:0/2624922029 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a200cd570 con 0x7f9a200cbf50 2024-04-20T10:58:38.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.273+0000 7f9a25ccd640 1 --2- 172.21.15.27:0/2624922029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200cbf50 0x7f9a200c5fc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a200c6990 tx=0x7f9a1402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0e94af724e1dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:38.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.273+0000 7f9a254cc640 1 -- 172.21.15.27:0/2624922029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140315a0 con 0x7f9a200cbf50 2024-04-20T10:58:38.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.273+0000 7f9a254cc640 1 -- 172.21.15.27:0/2624922029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a14037070 con 0x7f9a200cbf50 2024-04-20T10:58:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.273+0000 7f9a254cc640 1 -- 172.21.15.27:0/2624922029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14031960 con 0x7f9a200cbf50 2024-04-20T10:58:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.274+0000 7f9a28732640 1 -- 172.21.15.27:0/2624922029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200cbf50 msgr2=0x7f9a200c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.274+0000 7f9a28732640 1 --2- 172.21.15.27:0/2624922029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200cbf50 0x7f9a200c5fc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a200c6990 tx=0x7f9a1402b840 comp rx=0 tx=0).stop 2024-04-20T10:58:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.274+0000 7f9a28732640 1 -- 172.21.15.27:0/2624922029 shutdown_connections 2024-04-20T10:58:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.274+0000 7f9a28732640 1 --2- 172.21.15.27:0/2624922029 >> v2:172.21.15.137:3300/0 conn(0x7f9a200c6500 0x7f9a200d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.274+0000 7f9a28732640 1 --2- 172.21.15.27:0/2624922029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200cbf50 0x7f9a200c5fc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.274+0000 7f9a28732640 1 -- 172.21.15.27:0/2624922029 >> 172.21.15.27:0/2624922029 conn(0x7f9a20067ae0 msgr2=0x7f9a20067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.274+0000 7f9a28732640 1 -- 172.21.15.27:0/2624922029 shutdown_connections 2024-04-20T10:58:38.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.274+0000 7f9a28732640 1 -- 172.21.15.27:0/2624922029 wait complete. 2024-04-20T10:58:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.275+0000 7f9a28732640 1 Processor -- start 2024-04-20T10:58:38.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.275+0000 7f9a28732640 1 -- start start 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a28732640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200c6500 0x7f9a20150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a28732640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a200cbf50 0x7f9a2014fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a28732640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a20155800 0x7f9a201500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a28732640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a200cde10 con 0x7f9a200c6500 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a28732640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a200cdb30 con 0x7f9a20155800 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a28732640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f9a20155be0 con 0x7f9a200cbf50 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a264ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200c6500 0x7f9a20150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a264ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200c6500 0x7f9a20150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33540/0 (socket says 172.21.15.27:33540) 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a264ce640 1 -- 172.21.15.27:0/2416575207 learned_addr learned my addr 172.21.15.27:0/2416575207 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a26ccf640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a20155800 0x7f9a201500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.276+0000 7f9a25ccd640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a200cbf50 0x7f9a2014fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a264ce640 1 -- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a200cbf50 msgr2=0x7f9a2014fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a264ce640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a200cbf50 0x7f9a2014fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:38.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a264ce640 1 -- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a20155800 msgr2=0x7f9a201500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a264ce640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a20155800 0x7f9a201500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a264ce640 1 -- 172.21.15.27:0/2416575207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a201586f0 con 0x7f9a200c6500 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a26ccf640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a20155800 0x7f9a201500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a264ce640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200c6500 0x7f9a20150f10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800c3c0 tx=0x7f9a1800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a1effd640 1 -- 172.21.15.27:0/2416575207 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18017070 con 0x7f9a200c6500 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a1effd640 1 -- 172.21.15.27:0/2416575207 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a18005ce0 con 0x7f9a200c6500 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a1effd640 1 -- 172.21.15.27:0/2416575207 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a180072f0 con 0x7f9a200c6500 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a20158990 con 0x7f9a200c6500 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.277+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201522a0 con 0x7f9a200c6500 2024-04-20T10:58:38.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.278+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a200688e0 con 0x7f9a200c6500 2024-04-20T10:58:38.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.279+0000 7f9a1effd640 1 -- 172.21.15.27:0/2416575207 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a1801a070 con 0x7f9a200c6500 2024-04-20T10:58:38.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.279+0000 7f9a1effd640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9a0c06c8f0 0x7f9a0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:38.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.279+0000 7f9a1effd640 1 -- 172.21.15.27:0/2416575207 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f9a1808af80 con 0x7f9a200c6500 2024-04-20T10:58:38.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.279+0000 7f9a25ccd640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9a0c06c8f0 0x7f9a0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:38.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.280+0000 7f9a25ccd640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9a0c06c8f0 0x7f9a0c06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a14036000 tx=0x7f9a1403d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:38.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.284+0000 7f9a1effd640 1 -- 172.21.15.27:0/2416575207 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9a18059320 con 0x7f9a200c6500 2024-04-20T10:58:38.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.506+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a2004fcf0 con 0x7f9a200c6500 2024-04-20T10:58:38.507 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.507+0000 7f9a1effd640 1 -- 172.21.15.27:0/2416575207 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6101 (secure 0 0 0) 0x7f9a18058c20 con 0x7f9a200c6500 2024-04-20T10:58:38.508 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:38.508 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:38.206540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:58:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9a0c06c8f0 msgr2=0x7f9a0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9a0c06c8f0 0x7f9a0c06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a14036000 tx=0x7f9a1403d040 comp rx=0 tx=0).stop 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200c6500 msgr2=0x7f9a20150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200c6500 0x7f9a20150f10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800c3c0 tx=0x7f9a1800c880 comp rx=0 tx=0).stop 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 shutdown_connections 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f9a0c06c8f0 0x7f9a0c06edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a20155800 0x7f9a201500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f9a200cbf50 0x7f9a2014fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 --2- 172.21.15.27:0/2416575207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a200c6500 0x7f9a20150f10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 >> 172.21.15.27:0/2416575207 conn(0x7f9a20067ae0 msgr2=0x7f9a201533e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 shutdown_connections 2024-04-20T10:58:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:38.509+0000 7f9a28732640 1 -- 172.21.15.27:0/2416575207 wait complete. 2024-04-20T10:58:38.520 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:58:40.521 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:40.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.633+0000 7f2b3a52b640 1 Processor -- start 2024-04-20T10:58:40.634 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.633+0000 7f2b3a52b640 1 -- start start 2024-04-20T10:58:40.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.634+0000 7f2b3a52b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2b3415eaf0 0x7f2b34166e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:40.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.634+0000 7f2b3a52b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b3415f9b0 con 0x7f2b340c1f00 2024-04-20T10:58:40.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.634+0000 7f2b3a52b640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b3415f110 con 0x7f2b34155910 2024-04-20T10:58:40.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.634+0000 7f2b3a52b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f2b34155cf0 con 0x7f2b3415eaf0 2024-04-20T10:58:40.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.634+0000 7f2b337fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2b340c1f00 0x7f2b340c2910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48564/0 (socket says 172.21.15.27:48564) 2024-04-20T10:58:40.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.634+0000 7f2b337fe640 1 -- 172.21.15.27:0/1811414832 learned_addr learned my addr 172.21.15.27:0/1811414832 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:40.638 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.634+0000 7f2b38ac8640 1 --2- 172.21.15.27:0/1811414832 >> v2:172.21.15.137:3300/0 conn(0x7f2b3415eaf0 0x7f2b34166e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 773331019 0 0) 0x7f2b3415f9b0 con 0x7f2b340c1f00 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b1c003520 con 0x7f2b340c1f00 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734463550 0 0) 0x7f2b3415f110 con 0x7f2b34155910 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b3415f9b0 con 0x7f2b34155910 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1056531743 0 0) 0x7f2b1c003520 con 0x7f2b340c1f00 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 >> v2:172.21.15.137:3300/0 conn(0x7f2b3415eaf0 msgr2=0x7f2b34166e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 --2- 172.21.15.27:0/1811414832 >> v2:172.21.15.137:3300/0 conn(0x7f2b3415eaf0 0x7f2b34166e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 >> v1:172.21.15.137:6789/0 conn(0x7f2b34155910 legacy=0x7f2b3406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3415f3f0 con 0x7f2b340c1f00 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f2b28001670 con 0x7f2b340c1f00 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b38ac8640 1 --2- 172.21.15.27:0/1811414832 >> v2:172.21.15.137:3300/0 conn(0x7f2b3415eaf0 0x7f2b34166e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 >> v1:172.21.15.27:6789/0 conn(0x7f2b340c1f00 legacy=0x7f2b340c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b1c003b40 0x7f2b1c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:40.639 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b1c0051c0 0x7f2b1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34155cf0 con 0x7f2b1c004800 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c00c040 con 0x7f2b1c003b40 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1c00c240 con 0x7f2b1c0051c0 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b337fe640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b33fff640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b1c003b40 0x7f2b1c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b38ac8640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b1c0051c0 0x7f2b1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b337fe640 1 -- 172.21.15.27:0/1811414832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b1c0051c0 msgr2=0x7f2b1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b337fe640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b1c0051c0 0x7f2b1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b337fe640 1 -- 172.21.15.27:0/1811414832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b1c003b40 msgr2=0x7f2b1c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b337fe640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b1c003b40 0x7f2b1c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b337fe640 1 -- 172.21.15.27:0/1811414832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c003520 con 0x7f2b1c004800 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.635+0000 7f2b38ac8640 1 -- 172.21.15.27:0/1811414832 reap_dead start 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.636+0000 7f2b337fe640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b1c004c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c160 tx=0x7f2b1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c649d6c81a43c9b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:40.640 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.636+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28007ca0 con 0x7f2b1c004800 2024-04-20T10:58:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.636+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b28030070 con 0x7f2b1c004800 2024-04-20T10:58:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.636+0000 7f2b32ffd640 1 -- 172.21.15.27:0/1811414832 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28005d50 con 0x7f2b1c004800 2024-04-20T10:58:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.637+0000 7f2b3a52b640 1 -- 172.21.15.27:0/1811414832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 msgr2=0x7f2b1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.637+0000 7f2b3a52b640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b1c004c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c160 tx=0x7f2b1c005830 comp rx=0 tx=0).stop 2024-04-20T10:58:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.637+0000 7f2b3a52b640 1 -- 172.21.15.27:0/1811414832 shutdown_connections 2024-04-20T10:58:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.637+0000 7f2b3a52b640 1 --2- 172.21.15.27:0/1811414832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b1c004c80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.637+0000 7f2b3a52b640 1 -- 172.21.15.27:0/1811414832 >> 172.21.15.27:0/1811414832 conn(0x7f2b340ccee0 msgr2=0x7f2b340cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.637+0000 7f2b3a52b640 1 -- 172.21.15.27:0/1811414832 shutdown_connections 2024-04-20T10:58:40.641 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.637+0000 7f2b3a52b640 1 -- 172.21.15.27:0/1811414832 wait complete. 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.638+0000 7f2b3a52b640 1 Processor -- start 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.638+0000 7f2b3a52b640 1 -- start start 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b3a52b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b34152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b3a52b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b340719f0 0x7f2b34071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b3a52b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b341f55a0 0x7f2b341e4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b3a52b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3406ca00 con 0x7f2b1c004800 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b3a52b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2b340c7a70 con 0x7f2b341f55a0 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b3a52b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3406cce0 con 0x7f2b340719f0 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b33fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b34152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b33fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b34152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33564/0 (socket says 172.21.15.27:33564) 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b33fff640 1 -- 172.21.15.27:0/2686526410 learned_addr learned my addr 172.21.15.27:0/2686526410 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b38ac8640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b341f55a0 0x7f2b341e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:40.644 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.639+0000 7f2b337fe640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b340719f0 0x7f2b34071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b33fff640 1 -- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b340719f0 msgr2=0x7f2b34071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b33fff640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b340719f0 0x7f2b34071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b33fff640 1 -- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b341f55a0 msgr2=0x7f2b341e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b33fff640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b341f55a0 0x7f2b341e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b33fff640 1 -- 172.21.15.27:0/2686526410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b340b1c20 con 0x7f2b1c004800 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b337fe640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b340719f0 0x7f2b34071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b33fff640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b34152e80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400f7a0 tx=0x7f2b2400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b30ff9640 1 -- 172.21.15.27:0/2686526410 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2401a070 con 0x7f2b1c004800 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b30ff9640 1 -- 172.21.15.27:0/2686526410 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b24005110 con 0x7f2b1c004800 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b341f4de0 con 0x7f2b1c004800 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b30ff9640 1 -- 172.21.15.27:0/2686526410 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24007050 con 0x7f2b1c004800 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.640+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b340b22c0 con 0x7f2b1c004800 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.641+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3404fcf0 con 0x7f2b1c004800 2024-04-20T10:58:40.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.642+0000 7f2b30ff9640 1 -- 172.21.15.27:0/2686526410 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b2401d070 con 0x7f2b1c004800 2024-04-20T10:58:40.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.642+0000 7f2b30ff9640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b2006c8f0 0x7f2b2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:40.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.642+0000 7f2b30ff9640 1 -- 172.21.15.27:0/2686526410 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f2b2408b350 con 0x7f2b1c004800 2024-04-20T10:58:40.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.642+0000 7f2b337fe640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b2006c8f0 0x7f2b2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:40.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.643+0000 7f2b337fe640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b2006c8f0 0x7f2b2006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2b28034000 tx=0x7f2b28007000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:40.647 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.646+0000 7f2b30ff9640 1 -- 172.21.15.27:0/2686526410 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2b340b1c20 con 0x7f2b1c004800 2024-04-20T10:58:40.871 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.871+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b340c9030 con 0x7f2b1c004800 2024-04-20T10:58:40.872 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.871+0000 7f2b30ff9640 1 -- 172.21.15.27:0/2686526410 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6101 (secure 0 0 0) 0x7f2b24059010 con 0x7f2b1c004800 2024-04-20T10:58:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:40.872 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:38.206540+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:58:40.873 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2024-04-20T10:58:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b2006c8f0 msgr2=0x7f2b2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b2006c8f0 0x7f2b2006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2b28034000 tx=0x7f2b28007000 comp rx=0 tx=0).stop 2024-04-20T10:58:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 msgr2=0x7f2b34152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b34152e80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400f7a0 tx=0x7f2b2400fc60 comp rx=0 tx=0).stop 2024-04-20T10:58:40.874 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 shutdown_connections 2024-04-20T10:58:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b2006c8f0 0x7f2b2006edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b341f55a0 0x7f2b341e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b340719f0 0x7f2b34071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 --2- 172.21.15.27:0/2686526410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b1c004800 0x7f2b34152e80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 >> 172.21.15.27:0/2686526410 conn(0x7f2b340ccee0 msgr2=0x7f2b3415d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 shutdown_connections 2024-04-20T10:58:40.875 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:40.873+0000 7f2b3a52b640 1 -- 172.21.15.27:0/2686526410 wait complete. 2024-04-20T10:58:40.885 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:58:42.276 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:58:42.279 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:58:42.886 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:42.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.998+0000 7f558ca48640 1 Processor -- start 2024-04-20T10:58:42.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.998+0000 7f558ca48640 1 -- start start 2024-04-20T10:58:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.999+0000 7f558ca48640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5588151f40 0x7f558806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:43.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.999+0000 7f558ca48640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f558815e990 0x7f558806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.999+0000 7f558ca48640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55880c9970 con 0x7f5588152320 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.999+0000 7f558ca48640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5588070210 con 0x7f558815e990 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.999+0000 7f558ca48640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f558815b750 con 0x7f5588151f40 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.999+0000 7f5586d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f558815e990 0x7f558806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.999+0000 7f5585d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5588152320 0x7f558806ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48568/0 (socket says 172.21.15.27:48568) 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.999+0000 7f5585d9b640 1 -- 172.21.15.27:0/2111543778 learned_addr learned my addr 172.21.15.27:0/2111543778 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:42.999+0000 7f558659c640 1 --2- 172.21.15.27:0/2111543778 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5588151f40 0x7f558806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.000+0000 7f5586d9d640 1 -- 172.21.15.27:0/2111543778 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5588151f40 msgr2=0x7f558806f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.000+0000 7f5586d9d640 1 --2- 172.21.15.27:0/2111543778 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5588151f40 0x7f558806f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.000+0000 7f5586d9d640 1 -- 172.21.15.27:0/2111543778 >> v1:172.21.15.27:6789/0 conn(0x7f5588152320 legacy=0x7f558806ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.000+0000 7f5586d9d640 1 -- 172.21.15.27:0/2111543778 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558815ba30 con 0x7f558815e990 2024-04-20T10:58:43.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.000+0000 7f558559a640 1 -- 172.21.15.27:0/2111543778 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1892677406 0 0) 0x7f55880c9970 con 0x7f5588152320 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f5586d9d640 1 --2- 172.21.15.27:0/2111543778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558815e990 0x7f558806fb00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5578009d90 tx=0x7f557802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e39833a8ac8353f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558559a640 1 -- 172.21.15.27:0/2111543778 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578039070 con 0x7f558815e990 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558559a640 1 -- 172.21.15.27:0/2111543778 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f557802cce0 con 0x7f558815e990 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558559a640 1 -- 172.21.15.27:0/2111543778 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578034660 con 0x7f558815e990 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558ca48640 1 -- 172.21.15.27:0/2111543778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558815e990 msgr2=0x7f558806fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558ca48640 1 --2- 172.21.15.27:0/2111543778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558815e990 0x7f558806fb00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5578009d90 tx=0x7f557802bf30 comp rx=0 tx=0).stop 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558ca48640 1 -- 172.21.15.27:0/2111543778 shutdown_connections 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558ca48640 1 --2- 172.21.15.27:0/2111543778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f558815e990 0x7f558806fb00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558ca48640 1 --2- 172.21.15.27:0/2111543778 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5588151f40 0x7f558806f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558ca48640 1 -- 172.21.15.27:0/2111543778 >> 172.21.15.27:0/2111543778 conn(0x7f55880cee90 msgr2=0x7f55880cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558ca48640 1 -- 172.21.15.27:0/2111543778 shutdown_connections 2024-04-20T10:58:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.001+0000 7f558ca48640 1 -- 172.21.15.27:0/2111543778 wait complete. 2024-04-20T10:58:43.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.003+0000 7f558ca48640 1 Processor -- start 2024-04-20T10:58:43.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.003+0000 7f558ca48640 1 -- start start 2024-04-20T10:58:43.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f558ca48640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5588151f40 0x7f5588158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f558ca48640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5588152320 0x7f55881e4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f558ca48640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f558815e990 0x7f55881f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f558ca48640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f558815c2d0 con 0x7f5588152320 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f558ca48640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f558815bff0 con 0x7f5588151f40 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f558ca48640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f558806f0a0 con 0x7f558815e990 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5585d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5588152320 0x7f55881e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5585d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5588152320 0x7f55881e4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33572/0 (socket says 172.21.15.27:33572) 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5585d9b640 1 -- 172.21.15.27:0/415317056 learned_addr learned my addr 172.21.15.27:0/415317056 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f558659c640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5588151f40 0x7f5588158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5586d9d640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f558815e990 0x7f55881f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5585d9b640 1 -- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f558815e990 msgr2=0x7f55881f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5585d9b640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f558815e990 0x7f55881f2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5585d9b640 1 -- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5588151f40 msgr2=0x7f5588158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5585d9b640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5588151f40 0x7f5588158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5585d9b640 1 -- 172.21.15.27:0/415317056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55881f16d0 con 0x7f5588152320 2024-04-20T10:58:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f558659c640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5588151f40 0x7f5588158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:58:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f5585d9b640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5588152320 0x7f55881e4520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f557000c4e0 tx=0x7f557000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f557effd640 1 -- 172.21.15.27:0/415317056 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570018070 con 0x7f5588152320 2024-04-20T10:58:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f557effd640 1 -- 172.21.15.27:0/415317056 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5570007500 con 0x7f5588152320 2024-04-20T10:58:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55881e4bd0 con 0x7f5588152320 2024-04-20T10:58:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.004+0000 7f557effd640 1 -- 172.21.15.27:0/415317056 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570004a20 con 0x7f5588152320 2024-04-20T10:58:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.005+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55881e5190 con 0x7f5588152320 2024-04-20T10:58:43.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.006+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558806e010 con 0x7f5588152320 2024-04-20T10:58:43.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.006+0000 7f557effd640 1 -- 172.21.15.27:0/415317056 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f557001b070 con 0x7f5588152320 2024-04-20T10:58:43.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.007+0000 7f557effd640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f556806c8f0 0x7f556806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:43.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.007+0000 7f557effd640 1 -- 172.21.15.27:0/415317056 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f557008aed0 con 0x7f5588152320 2024-04-20T10:58:43.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.007+0000 7f558659c640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f556806c8f0 0x7f556806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:43.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.007+0000 7f558659c640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f556806c8f0 0x7f556806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5580009f30 tx=0x7f5580009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:43.012 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.011+0000 7f557effd640 1 -- 172.21.15.27:0/415317056 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f558806e010 con 0x7f5588152320 2024-04-20T10:58:43.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.238+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f558804fcf0 con 0x7f5588152320 2024-04-20T10:58:43.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.238+0000 7f557effd640 1 -- 172.21.15.27:0/415317056 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+6106 (secure 0 0 0) 0x7f5570058c50 con 0x7f5588152320 2024-04-20T10:58:43.239 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:43.240 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293,4894]},"id":1}]} 2024-04-20T10:58:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-04-20T10:58:43.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f556806c8f0 msgr2=0x7f556806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:43.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f556806c8f0 0x7f556806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5580009f30 tx=0x7f5580009190 comp rx=0 tx=0).stop 2024-04-20T10:58:43.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5588152320 msgr2=0x7f55881e4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:43.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5588152320 0x7f55881e4520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f557000c4e0 tx=0x7f557000c9a0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 shutdown_connections 2024-04-20T10:58:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f556806c8f0 0x7f556806edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f558815e990 0x7f55881f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5588152320 0x7f55881e4520 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 --2- 172.21.15.27:0/415317056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5588151f40 0x7f5588158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 >> 172.21.15.27:0/415317056 conn(0x7f55880cee90 msgr2=0x7f558806d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 shutdown_connections 2024-04-20T10:58:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:43.240+0000 7f558ca48640 1 -- 172.21.15.27:0/415317056 wait complete. 2024-04-20T10:58:43.252 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=4, replacing gid=4259 2024-04-20T10:58:43.252 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.f 2024-04-20T10:58:46.253 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2024-04-20T10:58:46.253 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-20T10:58:46.253 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-20T10:58:46.256 INFO:tasks.ceph.mds.f:Started 2024-04-20T10:58:46.256 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:46.328 INFO:tasks.ceph.mds.f.smithi137.stdout:starting mds.f at 2024-04-20T10:58:46.334 INFO:tasks.ceph.mds.d.smithi137.stderr:2024-04-20T10:58:46.334+0000 7f8f0d139640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-20T10:58:46.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.370+0000 7fbcefac0640 1 Processor -- start 2024-04-20T10:58:46.370 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.371+0000 7fbcefac0640 1 -- start start 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.371+0000 7fbcefac0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce80c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.371+0000 7fbcefac0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce80c8760 con 0x7fbce8155320 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.371+0000 7fbcefac0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce815b4d0 con 0x7fbce80722b0 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.371+0000 7fbcefac0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbce815b7b0 con 0x7fbce8073470 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.371+0000 7fbced85c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbce8155320 0x7fbce806f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45814/0 (socket says 172.21.15.27:45814) 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.372+0000 7fbced85c640 1 -- 172.21.15.27:0/3713741248 learned_addr learned my addr 172.21.15.27:0/3713741248 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.372+0000 7fbced05b640 1 --2- 172.21.15.27:0/3713741248 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce80c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.372+0000 7fbcec85a640 1 -- 172.21.15.27:0/3713741248 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 769364627 0 0) 0x7fbce80c8760 con 0x7fbce8155320 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.372+0000 7fbcec85a640 1 -- 172.21.15.27:0/3713741248 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcd0003520 con 0x7fbce8155320 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.372+0000 7fbcec85a640 1 -- 172.21.15.27:0/3713741248 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379305833 0 0) 0x7fbce815b4d0 con 0x7fbce80722b0 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.372+0000 7fbcec85a640 1 -- 172.21.15.27:0/3713741248 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbce80c8760 con 0x7fbce80722b0 2024-04-20T10:58:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.372+0000 7fbced05b640 1 -- 172.21.15.27:0/3713741248 >> v1:172.21.15.137:6789/0 conn(0x7fbce80722b0 legacy=0x7fbce8072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:46.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.372+0000 7fbced05b640 1 -- 172.21.15.27:0/3713741248 >> v1:172.21.15.27:6789/0 conn(0x7fbce8155320 legacy=0x7fbce806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:46.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.372+0000 7fbced05b640 1 -- 172.21.15.27:0/3713741248 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce815ba90 con 0x7fbce8073470 2024-04-20T10:58:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.373+0000 7fbced05b640 1 --2- 172.21.15.27:0/3713741248 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce80c9e90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0097d0 tx=0x7fbcdc02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb88508e76a3023e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.373+0000 7fbcec85a640 1 -- 172.21.15.27:0/3713741248 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc035020 con 0x7fbce8073470 2024-04-20T10:58:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.373+0000 7fbcec85a640 1 -- 172.21.15.27:0/3713741248 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcdc031890 con 0x7fbce8073470 2024-04-20T10:58:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.373+0000 7fbcec85a640 1 -- 172.21.15.27:0/3713741248 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc031c40 con 0x7fbce8073470 2024-04-20T10:58:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.374+0000 7fbcefac0640 1 -- 172.21.15.27:0/3713741248 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 msgr2=0x7fbce80c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.374+0000 7fbcefac0640 1 --2- 172.21.15.27:0/3713741248 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce80c9e90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0097d0 tx=0x7fbcdc02b930 comp rx=0 tx=0).stop 2024-04-20T10:58:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.374+0000 7fbcefac0640 1 -- 172.21.15.27:0/3713741248 shutdown_connections 2024-04-20T10:58:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.374+0000 7fbcefac0640 1 --2- 172.21.15.27:0/3713741248 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce80c9e90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:46.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.374+0000 7fbcefac0640 1 -- 172.21.15.27:0/3713741248 >> 172.21.15.27:0/3713741248 conn(0x7fbce80cd9c0 msgr2=0x7fbce80cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:46.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.374+0000 7fbcefac0640 1 -- 172.21.15.27:0/3713741248 shutdown_connections 2024-04-20T10:58:46.375 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.374+0000 7fbcefac0640 1 -- 172.21.15.27:0/3713741248 wait complete. 2024-04-20T10:58:46.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.375+0000 7fbcefac0640 1 Processor -- start 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.375+0000 7fbcefac0640 1 -- start start 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcefac0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbce80722b0 0x7fbce81f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcefac0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcefac0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce8155320 0x7fbce81f1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcefac0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbce80c8340 con 0x7fbce8155320 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcefac0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbce815c050 con 0x7fbce80722b0 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcefac0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbce8155700 con 0x7fbce8073470 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcee05d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce8155320 0x7fbce81f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbced05b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcee05d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce8155320 0x7fbce81f1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43794/0 (socket says 172.21.15.27:43794) 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbced05b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce815c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:52186/0 (socket says 172.21.15.27:52186) 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcee05d640 1 -- 172.21.15.27:0/2623373570 learned_addr learned my addr 172.21.15.27:0/2623373570 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbced85c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbce80722b0 0x7fbce81f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:46.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcee05d640 1 -- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 msgr2=0x7fbce815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcee05d640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce815c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcee05d640 1 -- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbce80722b0 msgr2=0x7fbce81f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcee05d640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbce80722b0 0x7fbce81f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.376+0000 7fbcee05d640 1 -- 172.21.15.27:0/2623373570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce815d1c0 con 0x7fbce8155320 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.377+0000 7fbcee05d640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce8155320 0x7fbce81f1030 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c770 tx=0x7fbcd800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.377+0000 7fbce67fc640 1 -- 172.21.15.27:0/2623373570 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8003b70 con 0x7fbce8155320 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.377+0000 7fbce67fc640 1 -- 172.21.15.27:0/2623373570 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd8003d30 con 0x7fbce8155320 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.377+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce81f4330 con 0x7fbce8155320 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.377+0000 7fbce67fc640 1 -- 172.21.15.27:0/2623373570 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800aa30 con 0x7fbce8155320 2024-04-20T10:58:46.378 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.377+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce81f48f0 con 0x7fbce8155320 2024-04-20T10:58:46.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.378+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce81f4c30 con 0x7fbce8155320 2024-04-20T10:58:46.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.378+0000 7fbce67fc640 1 -- 172.21.15.27:0/2623373570 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcd800abd0 con 0x7fbce8155320 2024-04-20T10:58:46.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.379+0000 7fbce67fc640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcd406c920 0x7fbcd406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:46.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.379+0000 7fbce67fc640 1 -- 172.21.15.27:0/2623373570 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7fbcd808ba00 con 0x7fbce8155320 2024-04-20T10:58:46.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.379+0000 7fbced85c640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcd406c920 0x7fbcd406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:46.379 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.379+0000 7fbced85c640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcd406c920 0x7fbcd406ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000f60 tx=0x7fbce0007d30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:46.383 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.383+0000 7fbce67fc640 1 -- 172.21.15.27:0/2623373570 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbce81f4c30 con 0x7fbce8155320 2024-04-20T10:58:46.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.608+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbce804fcf0 con 0x7fbce8155320 2024-04-20T10:58:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.609+0000 7fbce67fc640 1 -- 172.21.15.27:0/2623373570 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+6106 (secure 0 0 0) 0x7fbcd80596c0 con 0x7fbce8155320 2024-04-20T10:58:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:46.609 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4272,4275,4278,4293,4894]},"id":1}]} 2024-04-20T10:58:46.609 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2024-04-20T10:58:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcd406c920 msgr2=0x7fbcd406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcd406c920 0x7fbcd406ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000f60 tx=0x7fbce0007d30 comp rx=0 tx=0).stop 2024-04-20T10:58:46.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce8155320 msgr2=0x7fbce81f1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce8155320 0x7fbce81f1030 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c770 tx=0x7fbcd800cc30 comp rx=0 tx=0).stop 2024-04-20T10:58:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 shutdown_connections 2024-04-20T10:58:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcd406c920 0x7fbcd406ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce8155320 0x7fbce81f1030 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbce8073470 0x7fbce815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 --2- 172.21.15.27:0/2623373570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbce80722b0 0x7fbce81f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 >> 172.21.15.27:0/2623373570 conn(0x7fbce80cd9c0 msgr2=0x7fbce8072700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 shutdown_connections 2024-04-20T10:58:46.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:46.611+0000 7fbcefac0640 1 -- 172.21.15.27:0/2623373570 wait complete. 2024-04-20T10:58:46.622 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2024-04-20T10:58:48.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:58:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.736+0000 7f6697206640 1 Processor -- start 2024-04-20T10:58:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.736+0000 7f6697206640 1 -- start start 2024-04-20T10:58:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.737+0000 7f6697206640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6690153480 0x7f66900c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.737+0000 7f6697206640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66901592e0 0x7f66900c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.737+0000 7f6697206640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66900cc2a0 0x7f66900c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.737+0000 7f6697206640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6690071520 con 0x7f66901592e0 2024-04-20T10:58:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.737+0000 7f6697206640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f6690070a60 con 0x7f6690153480 2024-04-20T10:58:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.737+0000 7f6697206640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6690071770 con 0x7f66900cc2a0 2024-04-20T10:58:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f668ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66901592e0 0x7f66900c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f668ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66901592e0 0x7f66900c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43808/0 (socket says 172.21.15.27:43808) 2024-04-20T10:58:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f668ffff640 1 -- 172.21.15.27:0/1548832303 learned_addr learned my addr 172.21.15.27:0/1548832303 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f66957a3640 1 --2- 172.21.15.27:0/1548832303 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66900cc2a0 0x7f66900c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f6694fa2640 1 --2- 172.21.15.27:0/1548832303 >> v2:172.21.15.137:3300/0 conn(0x7f6690153480 0x7f66900c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f668ffff640 1 -- 172.21.15.27:0/1548832303 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66900cc2a0 msgr2=0x7f66900c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f668ffff640 1 --2- 172.21.15.27:0/1548832303 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66900cc2a0 0x7f66900c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f668ffff640 1 -- 172.21.15.27:0/1548832303 >> v2:172.21.15.137:3300/0 conn(0x7f6690153480 msgr2=0x7f66900c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:58:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f668ffff640 1 --2- 172.21.15.27:0/1548832303 >> v2:172.21.15.137:3300/0 conn(0x7f6690153480 0x7f66900c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.738+0000 7f668ffff640 1 -- 172.21.15.27:0/1548832303 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6690070d00 con 0x7f66901592e0 2024-04-20T10:58:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f668ffff640 1 --2- 172.21.15.27:0/1548832303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66901592e0 0x7f66900c9800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f669006fca0 tx=0x7f668402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527e64bf86187912 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f668f7fe640 1 -- 172.21.15.27:0/1548832303 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684035020 con 0x7f66901592e0 2024-04-20T10:58:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f668f7fe640 1 -- 172.21.15.27:0/1548832303 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6684031870 con 0x7f66901592e0 2024-04-20T10:58:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f668f7fe640 1 -- 172.21.15.27:0/1548832303 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684031c40 con 0x7f66901592e0 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f6697206640 1 -- 172.21.15.27:0/1548832303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66901592e0 msgr2=0x7f66900c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f6697206640 1 --2- 172.21.15.27:0/1548832303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66901592e0 0x7f66900c9800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f669006fca0 tx=0x7f668402b910 comp rx=0 tx=0).stop 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f6697206640 1 -- 172.21.15.27:0/1548832303 shutdown_connections 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f6697206640 1 --2- 172.21.15.27:0/1548832303 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66900cc2a0 0x7f66900c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f6697206640 1 --2- 172.21.15.27:0/1548832303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66901592e0 0x7f66900c9800 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f6697206640 1 --2- 172.21.15.27:0/1548832303 >> v2:172.21.15.137:3300/0 conn(0x7f6690153480 0x7f66900c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.739+0000 7f6697206640 1 -- 172.21.15.27:0/1548832303 >> 172.21.15.27:0/1548832303 conn(0x7f6690067ae0 msgr2=0x7f6690067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.740+0000 7f6697206640 1 -- 172.21.15.27:0/1548832303 shutdown_connections 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.740+0000 7f6697206640 1 -- 172.21.15.27:0/1548832303 wait complete. 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.741+0000 7f6697206640 1 Processor -- start 2024-04-20T10:58:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.741+0000 7f6697206640 1 -- start start 2024-04-20T10:58:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f6697206640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66900cc2a0 0x7f6690146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f6697206640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6690153480 0x7f6690155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f6697206640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66901592e0 0x7f6690156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f6697206640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f66900c5f20 con 0x7f6690153480 2024-04-20T10:58:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f6697206640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6690071a50 con 0x7f66900cc2a0 2024-04-20T10:58:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f6697206640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f66900cf350 con 0x7f66901592e0 2024-04-20T10:58:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6690153480 0x7f6690155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6690153480 0x7f6690155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43824/0 (socket says 172.21.15.27:43824) 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668ffff640 1 -- 172.21.15.27:0/2148150438 learned_addr learned my addr 172.21.15.27:0/2148150438 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f6694fa2640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66900cc2a0 0x7f6690146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f66957a3640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66901592e0 0x7f6690156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668ffff640 1 -- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66901592e0 msgr2=0x7f6690156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668ffff640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66901592e0 0x7f6690156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668ffff640 1 -- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66900cc2a0 msgr2=0x7f6690146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668ffff640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66900cc2a0 0x7f6690146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668ffff640 1 -- 172.21.15.27:0/2148150438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6690148f40 con 0x7f6690153480 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f6694fa2640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66900cc2a0 0x7f6690146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668ffff640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6690153480 0x7f6690155bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f668402b950 tx=0x7f66840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.742+0000 7f668d7fa640 1 -- 172.21.15.27:0/2148150438 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684040070 con 0x7f6690153480 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.743+0000 7f668d7fa640 1 -- 172.21.15.27:0/2148150438 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6684031e10 con 0x7f6690153480 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.743+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66901ef630 con 0x7f6690153480 2024-04-20T10:58:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.743+0000 7f668d7fa640 1 -- 172.21.15.27:0/2148150438 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684035040 con 0x7f6690153480 2024-04-20T10:58:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.743+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66900caca0 con 0x7f6690153480 2024-04-20T10:58:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.743+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66900688e0 con 0x7f6690153480 2024-04-20T10:58:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.744+0000 7f668d7fa640 1 -- 172.21.15.27:0/2148150438 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6684038020 con 0x7f6690153480 2024-04-20T10:58:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.744+0000 7f668d7fa640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f667c06c8f0 0x7f667c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:58:48.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.745+0000 7f668d7fa640 1 -- 172.21.15.27:0/2148150438 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f66840ad840 con 0x7f6690153480 2024-04-20T10:58:48.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.745+0000 7f6694fa2640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f667c06c8f0 0x7f667c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:58:48.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.745+0000 7f6694fa2640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f667c06c8f0 0x7f667c06edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f668800e740 tx=0x7f668800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:58:48.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.749+0000 7f668d7fa640 1 -- 172.21.15.27:0/2148150438 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f668407b500 con 0x7f6690153480 2024-04-20T10:58:48.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.975+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f669004fcf0 con 0x7f6690153480 2024-04-20T10:58:48.975 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.975+0000 7f668d7fa640 1 -- 172.21.15.27:0/2148150438 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6933 (secure 0 0 0) 0x7f668407f800 con 0x7f6690153480 2024-04-20T10:58:48.975 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:58:48.976 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5029,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:58:48.976 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-04-20T10:58:48.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f667c06c8f0 msgr2=0x7f667c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:48.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f667c06c8f0 0x7f667c06edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f668800e740 tx=0x7f668800b040 comp rx=0 tx=0).stop 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6690153480 msgr2=0x7f6690155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6690153480 0x7f6690155bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f668402b950 tx=0x7f66840046c0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 shutdown_connections 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f667c06c8f0 0x7f667c06edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f66901592e0 0x7f6690156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6690153480 0x7f6690155bf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 --2- 172.21.15.27:0/2148150438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66900cc2a0 0x7f6690146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 >> 172.21.15.27:0/2148150438 conn(0x7f6690067ae0 msgr2=0x7f6690067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 shutdown_connections 2024-04-20T10:58:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:58:48.977+0000 7f6697206640 1 -- 172.21.15.27:0/2148150438 wait complete. 2024-04-20T10:58:48.988 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2024-04-20T10:58:48.988 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 33 secs before thrashing 2024-04-20T10:59:07.367 INFO:tasks.workunit.client.0.smithi027.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-20T10:59:07.368 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-20T10:59:07.368 INFO:tasks.workunit.client.0.smithi027.stderr:+ make 2024-04-20T10:59:07.662 INFO:tasks.workunit.client.0.smithi027.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-20T10:59:07.899 INFO:tasks.workunit.client.0.smithi027.stderr:++ readlink -f fsstress 2024-04-20T10:59:07.899 INFO:tasks.workunit.client.0.smithi027.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-20T10:59:07.899 INFO:tasks.workunit.client.0.smithi027.stderr:+ popd 2024-04-20T10:59:07.900 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-20T10:59:07.900 INFO:tasks.workunit.client.0.smithi027.stderr:+ popd 2024-04-20T10:59:07.900 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-20T10:59:07.901 INFO:tasks.workunit.client.0.smithi027.stderr:++ mktemp -d -p . 2024-04-20T10:59:07.901 INFO:tasks.workunit.client.0.smithi027.stderr:+ T=./tmp.uXDY8wZYfi 2024-04-20T10:59:07.901 INFO:tasks.workunit.client.0.smithi027.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.uXDY8wZYfi -l 1 -n 1000 -p 10 -v 2024-04-20T10:59:07.902 INFO:tasks.workunit.client.0.smithi027.stdout:seed = 1714299256 2024-04-20T10:59:07.902 INFO:tasks.workunit.client.0.smithi027.stdout:0/0: link - no file 2024-04-20T10:59:07.902 INFO:tasks.workunit.client.0.smithi027.stdout:1/0: dwrite - no filename 2024-04-20T10:59:07.902 INFO:tasks.workunit.client.0.smithi027.stdout:1/1: stat - no entries 2024-04-20T10:59:07.902 INFO:tasks.workunit.client.0.smithi027.stdout:0/1: mkdir d0 0 2024-04-20T10:59:07.903 INFO:tasks.workunit.client.0.smithi027.stdout:0/2: rename d0 to d0/d1 22 2024-04-20T10:59:07.903 INFO:tasks.workunit.client.0.smithi027.stdout:0/3: chown d0 5878 1 2024-04-20T10:59:07.903 INFO:tasks.workunit.client.0.smithi027.stdout:0/4: dread - no filename 2024-04-20T10:59:07.903 INFO:tasks.workunit.client.0.smithi027.stdout:0/5: dread - no filename 2024-04-20T10:59:07.903 INFO:tasks.workunit.client.0.smithi027.stdout:0/6: fdatasync - no filename 2024-04-20T10:59:07.904 INFO:tasks.workunit.client.0.smithi027.stdout:1/2: creat f0 x:0 0 0 2024-04-20T10:59:07.904 INFO:tasks.workunit.client.0.smithi027.stdout:0/7: creat d0/f2 x:0 0 0 2024-04-20T10:59:07.904 INFO:tasks.workunit.client.0.smithi027.stdout:0/8: write d0/f2 [77163,99675] 0 2024-04-20T10:59:07.904 INFO:tasks.workunit.client.0.smithi027.stdout:0/9: write d0/f2 [748341,96924] 0 2024-04-20T10:59:07.904 INFO:tasks.workunit.client.0.smithi027.stdout:0/10: write d0/f2 [8951,58204] 0 2024-04-20T10:59:07.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/11: creat d0/f3 x:0 0 0 2024-04-20T10:59:07.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/0: symlink l0 0 2024-04-20T10:59:07.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/12: write d0/f3 [730273,73598] 0 2024-04-20T10:59:07.908 INFO:tasks.workunit.client.0.smithi027.stdout:3/0: chown . 32015 1 2024-04-20T10:59:07.908 INFO:tasks.workunit.client.0.smithi027.stdout:3/1: dwrite - no filename 2024-04-20T10:59:07.909 INFO:tasks.workunit.client.0.smithi027.stdout:2/1: creat f1 x:0 0 0 2024-04-20T10:59:07.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/13: rename d0/f3 to d0/f4 0 2024-04-20T10:59:07.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/14: write d0/f4 [955595,123518] 0 2024-04-20T10:59:07.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/2: mknod c0 0 2024-04-20T10:59:07.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/3: chown c0 10 1 2024-04-20T10:59:07.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/4: chown c0 322931922 1 2024-04-20T10:59:07.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/5: chown c0 299697 1 2024-04-20T10:59:07.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/6: dwrite - no filename 2024-04-20T10:59:07.911 INFO:tasks.workunit.client.0.smithi027.stdout:2/2: unlink l0 0 2024-04-20T10:59:07.911 INFO:tasks.workunit.client.0.smithi027.stdout:2/3: rmdir - no directory 2024-04-20T10:59:07.912 INFO:tasks.workunit.client.0.smithi027.stdout:4/0: mkdir d0 0 2024-04-20T10:59:07.912 INFO:tasks.workunit.client.0.smithi027.stdout:4/1: write - no filename 2024-04-20T10:59:07.913 INFO:tasks.workunit.client.0.smithi027.stdout:3/7: rename c0 to c1 0 2024-04-20T10:59:07.913 INFO:tasks.workunit.client.0.smithi027.stdout:3/8: chown c1 50328530 1 2024-04-20T10:59:07.913 INFO:tasks.workunit.client.0.smithi027.stdout:3/9: write - no filename 2024-04-20T10:59:07.913 INFO:tasks.workunit.client.0.smithi027.stdout:2/4: link f1 f2 0 2024-04-20T10:59:07.914 INFO:tasks.workunit.client.0.smithi027.stdout:6/0: chown . 37375725 1 2024-04-20T10:59:07.914 INFO:tasks.workunit.client.0.smithi027.stdout:5/0: creat f0 x:0 0 0 2024-04-20T10:59:07.915 INFO:tasks.workunit.client.0.smithi027.stdout:4/2: mknod d0/c1 0 2024-04-20T10:59:07.916 INFO:tasks.workunit.client.0.smithi027.stdout:3/10: mkdir d2 0 2024-04-20T10:59:07.916 INFO:tasks.workunit.client.0.smithi027.stdout:2/5: creat f3 x:0 0 0 2024-04-20T10:59:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/0: dread - no filename 2024-04-20T10:59:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/1: chown . 73 1 2024-04-20T10:59:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/2: dread - no filename 2024-04-20T10:59:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/3: dwrite - no filename 2024-04-20T10:59:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/4: chown . 1095 1 2024-04-20T10:59:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/5: unlink - no file 2024-04-20T10:59:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/6: dwrite - no filename 2024-04-20T10:59:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/1: symlink l0 0 2024-04-20T10:59:07.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/3: symlink d0/l2 0 2024-04-20T10:59:07.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/4: write - no filename 2024-04-20T10:59:07.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/5: dread - no filename 2024-04-20T10:59:07.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/6: stat d0/c1 0 2024-04-20T10:59:07.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/7: truncate - no filename 2024-04-20T10:59:07.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/0: truncate - no filename 2024-04-20T10:59:07.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/1: dwrite - no filename 2024-04-20T10:59:07.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/2: read - no filename 2024-04-20T10:59:07.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/3: dread - no filename 2024-04-20T10:59:07.919 INFO:tasks.workunit.client.0.smithi027.stdout:7/7: mknod c0 0 2024-04-20T10:59:07.919 INFO:tasks.workunit.client.0.smithi027.stdout:7/8: dread - no filename 2024-04-20T10:59:07.920 INFO:tasks.workunit.client.0.smithi027.stdout:7/9: dread - no filename 2024-04-20T10:59:07.920 INFO:tasks.workunit.client.0.smithi027.stdout:6/2: creat f1 x:0 0 0 2024-04-20T10:59:07.920 INFO:tasks.workunit.client.0.smithi027.stdout:6/3: dread - f1 zero size 2024-04-20T10:59:07.921 INFO:tasks.workunit.client.0.smithi027.stdout:6/4: truncate f1 227930 0 2024-04-20T10:59:07.921 INFO:tasks.workunit.client.0.smithi027.stdout:6/5: truncate f1 842899 0 2024-04-20T10:59:07.921 INFO:tasks.workunit.client.0.smithi027.stdout:4/8: unlink d0/c1 0 2024-04-20T10:59:07.922 INFO:tasks.workunit.client.0.smithi027.stdout:9/0: write - no filename 2024-04-20T10:59:07.922 INFO:tasks.workunit.client.0.smithi027.stdout:9/1: dwrite - no filename 2024-04-20T10:59:07.922 INFO:tasks.workunit.client.0.smithi027.stdout:9/2: stat - no entries 2024-04-20T10:59:07.922 INFO:tasks.workunit.client.0.smithi027.stdout:9/3: truncate - no filename 2024-04-20T10:59:07.922 INFO:tasks.workunit.client.0.smithi027.stdout:9/4: link - no file 2024-04-20T10:59:07.923 INFO:tasks.workunit.client.0.smithi027.stdout:8/4: symlink l0 0 2024-04-20T10:59:07.923 INFO:tasks.workunit.client.0.smithi027.stdout:8/5: truncate - no filename 2024-04-20T10:59:07.923 INFO:tasks.workunit.client.0.smithi027.stdout:8/6: truncate - no filename 2024-04-20T10:59:07.923 INFO:tasks.workunit.client.0.smithi027.stdout:8/7: truncate - no filename 2024-04-20T10:59:07.923 INFO:tasks.workunit.client.0.smithi027.stdout:7/10: creat f1 x:0 0 0 2024-04-20T10:59:07.924 INFO:tasks.workunit.client.0.smithi027.stdout:1/3: dwrite f0 [0,4194304] 0 2024-04-20T10:59:07.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/9: mknod d0/c3 0 2024-04-20T10:59:07.925 INFO:tasks.workunit.client.0.smithi027.stdout:9/5: symlink l0 0 2024-04-20T10:59:07.925 INFO:tasks.workunit.client.0.smithi027.stdout:9/6: chown l0 342 1 2024-04-20T10:59:07.925 INFO:tasks.workunit.client.0.smithi027.stdout:9/7: dwrite - no filename 2024-04-20T10:59:07.925 INFO:tasks.workunit.client.0.smithi027.stdout:9/8: truncate - no filename 2024-04-20T10:59:07.925 INFO:tasks.workunit.client.0.smithi027.stdout:9/9: dread - no filename 2024-04-20T10:59:07.926 INFO:tasks.workunit.client.0.smithi027.stdout:6/6: dread f1 [0,4194304] 0 2024-04-20T10:59:07.926 INFO:tasks.workunit.client.0.smithi027.stdout:6/7: chown f1 75 1 2024-04-20T10:59:07.926 INFO:tasks.workunit.client.0.smithi027.stdout:6/8: chown l0 56 1 2024-04-20T10:59:07.926 INFO:tasks.workunit.client.0.smithi027.stdout:8/8: unlink l0 0 2024-04-20T10:59:07.927 INFO:tasks.workunit.client.0.smithi027.stdout:7/11: creat f2 x:0 0 0 2024-04-20T10:59:07.927 INFO:tasks.workunit.client.0.smithi027.stdout:7/12: dread - f2 zero size 2024-04-20T10:59:07.929 INFO:tasks.workunit.client.0.smithi027.stdout:1/4: creat f1 x:0 0 0 2024-04-20T10:59:07.929 INFO:tasks.workunit.client.0.smithi027.stdout:1/5: chown f1 3941 1 2024-04-20T10:59:07.930 INFO:tasks.workunit.client.0.smithi027.stdout:4/10: creat d0/f4 x:0 0 0 2024-04-20T10:59:07.931 INFO:tasks.workunit.client.0.smithi027.stdout:9/10: rename l0 to l1 0 2024-04-20T10:59:07.931 INFO:tasks.workunit.client.0.smithi027.stdout:9/11: fsync - no filename 2024-04-20T10:59:07.931 INFO:tasks.workunit.client.0.smithi027.stdout:9/12: read - no filename 2024-04-20T10:59:07.931 INFO:tasks.workunit.client.0.smithi027.stdout:9/13: rmdir - no directory 2024-04-20T10:59:07.931 INFO:tasks.workunit.client.0.smithi027.stdout:9/14: fsync - no filename 2024-04-20T10:59:07.931 INFO:tasks.workunit.client.0.smithi027.stdout:6/9: mknod c2 0 2024-04-20T10:59:07.936 INFO:tasks.workunit.client.0.smithi027.stdout:1/6: unlink f0 0 2024-04-20T10:59:07.936 INFO:tasks.workunit.client.0.smithi027.stdout:8/9: creat f1 x:0 0 0 2024-04-20T10:59:07.936 INFO:tasks.workunit.client.0.smithi027.stdout:0/15: fsync d0/f4 0 2024-04-20T10:59:07.937 INFO:tasks.workunit.client.0.smithi027.stdout:4/11: mknod d0/c5 0 2024-04-20T10:59:07.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/15: symlink l2 0 2024-04-20T10:59:07.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/16: dwrite - no filename 2024-04-20T10:59:07.937 INFO:tasks.workunit.client.0.smithi027.stdout:4/12: write d0/f4 [918569,60080] 0 2024-04-20T10:59:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:3/11: rename c1 to d2/c3 0 2024-04-20T10:59:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:3/12: write - no filename 2024-04-20T10:59:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:3/13: readlink - no filename 2024-04-20T10:59:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:3/14: chown d2/c3 1 1 2024-04-20T10:59:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:3/15: read - no filename 2024-04-20T10:59:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:4/13: write d0/f4 [457429,61377] 0 2024-04-20T10:59:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:4/14: write d0/f4 [1338273,91139] 0 2024-04-20T10:59:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:5/1: dwrite f0 [0,4194304] 0 2024-04-20T10:59:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:8/10: rename f1 to f2 0 2024-04-20T10:59:07.939 INFO:tasks.workunit.client.0.smithi027.stdout:8/11: chown f2 4 1 2024-04-20T10:59:07.939 INFO:tasks.workunit.client.0.smithi027.stdout:8/12: dread - f2 zero size 2024-04-20T10:59:07.943 INFO:tasks.workunit.client.0.smithi027.stdout:1/7: creat f2 x:0 0 0 2024-04-20T10:59:07.943 INFO:tasks.workunit.client.0.smithi027.stdout:2/6: getdents . 0 2024-04-20T10:59:07.943 INFO:tasks.workunit.client.0.smithi027.stdout:3/16: creat d2/f4 x:0 0 0 2024-04-20T10:59:07.943 INFO:tasks.workunit.client.0.smithi027.stdout:3/17: chown d2/c3 3659134 1 2024-04-20T10:59:07.944 INFO:tasks.workunit.client.0.smithi027.stdout:4/15: creat d0/f6 x:0 0 0 2024-04-20T10:59:07.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/8: mkdir d3 0 2024-04-20T10:59:07.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/9: write f1 [623033,72885] 0 2024-04-20T10:59:07.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/10: chown f2 30339327 1 2024-04-20T10:59:07.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/11: readlink - no filename 2024-04-20T10:59:07.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/12: chown f2 522 1 2024-04-20T10:59:07.944 INFO:tasks.workunit.client.0.smithi027.stdout:2/7: mknod c4 0 2024-04-20T10:59:07.957 INFO:tasks.workunit.client.0.smithi027.stdout:3/18: link d2/f4 d2/f5 0 2024-04-20T10:59:07.960 INFO:tasks.workunit.client.0.smithi027.stdout:1/13: unlink f1 0 2024-04-20T10:59:07.960 INFO:tasks.workunit.client.0.smithi027.stdout:1/14: write f2 [763466,129485] 0 2024-04-20T10:59:07.960 INFO:tasks.workunit.client.0.smithi027.stdout:7/13: dwrite f2 [0,4194304] 0 2024-04-20T10:59:07.961 INFO:tasks.workunit.client.0.smithi027.stdout:7/14: fsync f1 0 2024-04-20T10:59:07.961 INFO:tasks.workunit.client.0.smithi027.stdout:7/15: truncate f1 588866 0 2024-04-20T10:59:07.961 INFO:tasks.workunit.client.0.smithi027.stdout:7/16: fsync f1 0 2024-04-20T10:59:07.961 INFO:tasks.workunit.client.0.smithi027.stdout:2/8: unlink f2 0 2024-04-20T10:59:07.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/19: creat d2/f6 x:0 0 0 2024-04-20T10:59:07.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/20: write d2/f4 [363733,20246] 0 2024-04-20T10:59:07.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/21: read - d2/f6 zero size 2024-04-20T10:59:07.980 INFO:tasks.workunit.client.0.smithi027.stdout:7/17: symlink l3 0 2024-04-20T10:59:07.987 INFO:tasks.workunit.client.0.smithi027.stdout:3/22: rmdir d2 39 2024-04-20T10:59:08.071 INFO:tasks.workunit.client.0.smithi027.stdout:7/18: read f2 [1374341,125774] 0 2024-04-20T10:59:08.253 INFO:tasks.workunit.client.0.smithi027.stdout:6/10: dwrite f1 [0,4194304] 0 2024-04-20T10:59:08.266 INFO:tasks.workunit.client.0.smithi027.stdout:5/2: dread f0 [0,4194304] 0 2024-04-20T10:59:08.296 INFO:tasks.workunit.client.0.smithi027.stdout:6/11: write f1 [1564240,115278] 0 2024-04-20T10:59:08.313 INFO:tasks.workunit.client.0.smithi027.stdout:6/12: dwrite f1 [0,4194304] 0 2024-04-20T10:59:08.498 INFO:tasks.workunit.client.0.smithi027.stdout:3/23: creat d2/f7 x:0 0 0 2024-04-20T10:59:08.498 INFO:tasks.workunit.client.0.smithi027.stdout:7/19: write f2 [4149061,60250] 0 2024-04-20T10:59:08.498 INFO:tasks.workunit.client.0.smithi027.stdout:4/16: truncate d0/f4 106768 0 2024-04-20T10:59:08.499 INFO:tasks.workunit.client.0.smithi027.stdout:4/17: stat d0/l2 0 2024-04-20T10:59:08.831 INFO:tasks.workunit.client.0.smithi027.stdout:0/16: dwrite d0/f2 [0,4194304] 0 2024-04-20T10:59:08.865 INFO:tasks.workunit.client.0.smithi027.stdout:5/3: link f0 f1 0 2024-04-20T10:59:08.905 INFO:tasks.workunit.client.0.smithi027.stdout:1/15: dread f2 [0,4194304] 0 2024-04-20T10:59:08.905 INFO:tasks.workunit.client.0.smithi027.stdout:1/16: truncate f2 1874590 0 2024-04-20T10:59:09.162 INFO:tasks.workunit.client.0.smithi027.stdout:6/13: rename f1 to f3 0 2024-04-20T10:59:09.162 INFO:tasks.workunit.client.0.smithi027.stdout:3/24: rename d2/c3 to d2/c8 0 2024-04-20T10:59:09.162 INFO:tasks.workunit.client.0.smithi027.stdout:3/25: fdatasync d2/f6 0 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:6/14: truncate f3 4220409 0 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:7/20: mkdir d4 0 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:7/21: getdents d4 0 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:5/4: rename f1 to f2 0 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:0/17: mknod d0/c5 0 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:4/18: link d0/l2 d0/l7 0 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:4/19: chown d0/l2 30866008 1 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:4/20: dread - d0/f6 zero size 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:7/22: dread f1 [0,4194304] 0 2024-04-20T10:59:09.163 INFO:tasks.workunit.client.0.smithi027.stdout:7/23: write f1 [1515195,92488] 0 2024-04-20T10:59:09.182 INFO:tasks.workunit.client.0.smithi027.stdout:5/5: dwrite f0 [0,4194304] 0 2024-04-20T10:59:09.183 INFO:tasks.workunit.client.0.smithi027.stdout:0/18: dwrite d0/f4 [0,4194304] 0 2024-04-20T10:59:09.183 INFO:tasks.workunit.client.0.smithi027.stdout:0/19: fdatasync d0/f2 0 2024-04-20T10:59:09.190 INFO:tasks.workunit.client.0.smithi027.stdout:5/6: write f0 [77850,95706] 0 2024-04-20T10:59:09.190 INFO:tasks.workunit.client.0.smithi027.stdout:5/7: write f2 [4397907,62489] 0 2024-04-20T10:59:09.190 INFO:tasks.workunit.client.0.smithi027.stdout:5/8: chown f0 128695609 1 2024-04-20T10:59:09.867 INFO:tasks.workunit.client.0.smithi027.stdout:1/17: mknod d3/c4 0 2024-04-20T10:59:09.867 INFO:tasks.workunit.client.0.smithi027.stdout:6/15: creat f4 x:0 0 0 2024-04-20T10:59:09.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/26: creat d2/f9 x:0 0 0 2024-04-20T10:59:09.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/27: chown d2/f7 1 1 2024-04-20T10:59:09.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/21: rmdir d0 39 2024-04-20T10:59:09.869 INFO:tasks.workunit.client.0.smithi027.stdout:4/22: fsync d0/f6 0 2024-04-20T10:59:09.869 INFO:tasks.workunit.client.0.smithi027.stdout:7/24: mkdir d4/d5 0 2024-04-20T10:59:09.871 INFO:tasks.workunit.client.0.smithi027.stdout:3/28: dread d2/f5 [0,4194304] 0 2024-04-20T10:59:09.871 INFO:tasks.workunit.client.0.smithi027.stdout:0/20: link d0/f4 d0/f6 0 2024-04-20T10:59:09.873 INFO:tasks.workunit.client.0.smithi027.stdout:7/25: dread f1 [0,4194304] 0 2024-04-20T10:59:09.874 INFO:tasks.workunit.client.0.smithi027.stdout:6/16: mknod c5 0 2024-04-20T10:59:09.874 INFO:tasks.workunit.client.0.smithi027.stdout:6/17: rmdir - no directory 2024-04-20T10:59:09.876 INFO:tasks.workunit.client.0.smithi027.stdout:4/23: symlink d0/l8 0 2024-04-20T10:59:09.877 INFO:tasks.workunit.client.0.smithi027.stdout:3/29: symlink d2/la 0 2024-04-20T10:59:09.882 INFO:tasks.workunit.client.0.smithi027.stdout:0/21: dread d0/f4 [0,4194304] 0 2024-04-20T10:59:09.883 INFO:tasks.workunit.client.0.smithi027.stdout:7/26: creat d4/d5/f6 x:0 0 0 2024-04-20T10:59:09.884 INFO:tasks.workunit.client.0.smithi027.stdout:6/18: symlink l6 0 2024-04-20T10:59:09.885 INFO:tasks.workunit.client.0.smithi027.stdout:4/24: creat d0/f9 x:0 0 0 2024-04-20T10:59:09.886 INFO:tasks.workunit.client.0.smithi027.stdout:0/22: unlink d0/f6 0 2024-04-20T10:59:09.896 INFO:tasks.workunit.client.0.smithi027.stdout:0/23: dread d0/f2 [0,4194304] 0 2024-04-20T10:59:09.900 INFO:tasks.workunit.client.0.smithi027.stdout:1/18: dwrite f2 [0,4194304] 0 2024-04-20T10:59:09.900 INFO:tasks.workunit.client.0.smithi027.stdout:0/24: dread d0/f2 [0,4194304] 0 2024-04-20T10:59:09.900 INFO:tasks.workunit.client.0.smithi027.stdout:1/19: chown d3 243514159 1 2024-04-20T10:59:09.900 INFO:tasks.workunit.client.0.smithi027.stdout:7/27: mknod d4/d5/c7 0 2024-04-20T10:59:09.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/30: dwrite d2/f5 [0,4194304] 0 2024-04-20T10:59:09.901 INFO:tasks.workunit.client.0.smithi027.stdout:6/19: creat f7 x:0 0 0 2024-04-20T10:59:09.902 INFO:tasks.workunit.client.0.smithi027.stdout:4/25: unlink d0/c3 0 2024-04-20T10:59:09.903 INFO:tasks.workunit.client.0.smithi027.stdout:0/25: dread d0/f2 [0,4194304] 0 2024-04-20T10:59:09.904 INFO:tasks.workunit.client.0.smithi027.stdout:1/20: rmdir d3 39 2024-04-20T10:59:09.906 INFO:tasks.workunit.client.0.smithi027.stdout:7/28: mkdir d4/d5/d8 0 2024-04-20T10:59:09.906 INFO:tasks.workunit.client.0.smithi027.stdout:7/29: dread - d4/d5/f6 zero size 2024-04-20T10:59:09.907 INFO:tasks.workunit.client.0.smithi027.stdout:0/26: dread d0/f2 [0,4194304] 0 2024-04-20T10:59:09.907 INFO:tasks.workunit.client.0.smithi027.stdout:0/27: readlink - no filename 2024-04-20T10:59:09.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/31: rename d2/c8 to d2/cb 0 2024-04-20T10:59:09.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/32: fdatasync d2/f6 0 2024-04-20T10:59:09.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/33: truncate d2/f7 629837 0 2024-04-20T10:59:09.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/34: chown d2/f7 106275 1 2024-04-20T10:59:09.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/35: chown d2/f6 613 1 2024-04-20T10:59:09.910 INFO:tasks.workunit.client.0.smithi027.stdout:3/36: write d2/f7 [743021,23034] 0 2024-04-20T10:59:09.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/20: creat f8 x:0 0 0 2024-04-20T10:59:09.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/21: stat c5 0 2024-04-20T10:59:09.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/22: truncate f7 912418 0 2024-04-20T10:59:09.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/23: read - f4 zero size 2024-04-20T10:59:09.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/28: read d0/f2 [4015806,49573] 0 2024-04-20T10:59:09.911 INFO:tasks.workunit.client.0.smithi027.stdout:4/26: unlink d0/l8 0 2024-04-20T10:59:09.913 INFO:tasks.workunit.client.0.smithi027.stdout:1/21: link d3/c4 d3/c5 0 2024-04-20T10:59:09.914 INFO:tasks.workunit.client.0.smithi027.stdout:7/30: truncate f2 1202487 0 2024-04-20T10:59:09.914 INFO:tasks.workunit.client.0.smithi027.stdout:7/31: fdatasync d4/d5/f6 0 2024-04-20T10:59:09.915 INFO:tasks.workunit.client.0.smithi027.stdout:3/37: creat d2/fc x:0 0 0 2024-04-20T10:59:09.915 INFO:tasks.workunit.client.0.smithi027.stdout:3/38: truncate d2/f9 605218 0 2024-04-20T10:59:09.916 INFO:tasks.workunit.client.0.smithi027.stdout:6/24: mknod c9 0 2024-04-20T10:59:09.916 INFO:tasks.workunit.client.0.smithi027.stdout:6/25: rmdir - no directory 2024-04-20T10:59:09.916 INFO:tasks.workunit.client.0.smithi027.stdout:6/26: chown f7 104131682 1 2024-04-20T10:59:09.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/32: dread f1 [0,4194304] 0 2024-04-20T10:59:09.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/33: readlink l3 0 2024-04-20T10:59:09.917 INFO:tasks.workunit.client.0.smithi027.stdout:0/29: mkdir d0/d7 0 2024-04-20T10:59:09.917 INFO:tasks.workunit.client.0.smithi027.stdout:0/30: write d0/f2 [4639190,25306] 0 2024-04-20T10:59:09.918 INFO:tasks.workunit.client.0.smithi027.stdout:0/31: readlink - no filename 2024-04-20T10:59:09.918 INFO:tasks.workunit.client.0.smithi027.stdout:0/32: chown d0/c5 47 1 2024-04-20T10:59:09.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/27: symlink d0/la 0 2024-04-20T10:59:09.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/28: read - d0/f9 zero size 2024-04-20T10:59:09.918 INFO:tasks.workunit.client.0.smithi027.stdout:4/29: stat d0/f9 0 2024-04-20T10:59:09.925 INFO:tasks.workunit.client.0.smithi027.stdout:1/22: rename f2 to d3/f6 0 2024-04-20T10:59:09.925 INFO:tasks.workunit.client.0.smithi027.stdout:6/27: mkdir da 0 2024-04-20T10:59:09.926 INFO:tasks.workunit.client.0.smithi027.stdout:7/34: mknod d4/d5/c9 0 2024-04-20T10:59:09.926 INFO:tasks.workunit.client.0.smithi027.stdout:7/35: fdatasync d4/d5/f6 0 2024-04-20T10:59:09.929 INFO:tasks.workunit.client.0.smithi027.stdout:7/36: write f1 [1308635,62597] 0 2024-04-20T10:59:09.929 INFO:tasks.workunit.client.0.smithi027.stdout:7/37: truncate f1 1715525 0 2024-04-20T10:59:09.930 INFO:tasks.workunit.client.0.smithi027.stdout:0/33: unlink d0/f4 0 2024-04-20T10:59:09.931 INFO:tasks.workunit.client.0.smithi027.stdout:1/23: stat d3/c5 0 2024-04-20T10:59:09.931 INFO:tasks.workunit.client.0.smithi027.stdout:1/24: chown d3/c4 1682 1 2024-04-20T10:59:09.932 INFO:tasks.workunit.client.0.smithi027.stdout:6/28: rename f8 to da/fb 0 2024-04-20T10:59:09.933 INFO:tasks.workunit.client.0.smithi027.stdout:6/29: read f7 [444597,76433] 0 2024-04-20T10:59:09.933 INFO:tasks.workunit.client.0.smithi027.stdout:6/30: write da/fb [673114,19765] 0 2024-04-20T10:59:09.933 INFO:tasks.workunit.client.0.smithi027.stdout:6/31: stat c9 0 2024-04-20T10:59:09.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/25: mkdir d3/d7 0 2024-04-20T10:59:09.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/26: truncate d3/f6 4824857 0 2024-04-20T10:59:09.943 INFO:tasks.workunit.client.0.smithi027.stdout:3/39: dwrite d2/f7 [0,4194304] 0 2024-04-20T10:59:09.943 INFO:tasks.workunit.client.0.smithi027.stdout:4/30: dwrite d0/f6 [0,4194304] 0 2024-04-20T10:59:09.944 INFO:tasks.workunit.client.0.smithi027.stdout:0/34: dread d0/f2 [0,4194304] 0 2024-04-20T10:59:09.944 INFO:tasks.workunit.client.0.smithi027.stdout:0/35: truncate d0/f2 5440653 0 2024-04-20T10:59:09.944 INFO:tasks.workunit.client.0.smithi027.stdout:6/32: mknod da/cc 0 2024-04-20T10:59:09.946 INFO:tasks.workunit.client.0.smithi027.stdout:1/27: creat d3/d7/f8 x:0 0 0 2024-04-20T10:59:09.946 INFO:tasks.workunit.client.0.smithi027.stdout:1/28: truncate d3/d7/f8 65929 0 2024-04-20T10:59:09.946 INFO:tasks.workunit.client.0.smithi027.stdout:1/29: chown d3/c4 146 1 2024-04-20T10:59:10.024 INFO:tasks.workunit.client.0.smithi027.stdout:8/13: sync 2024-04-20T10:59:10.024 INFO:tasks.workunit.client.0.smithi027.stdout:9/17: sync 2024-04-20T10:59:10.024 INFO:tasks.workunit.client.0.smithi027.stdout:8/14: dread - f2 zero size 2024-04-20T10:59:10.025 INFO:tasks.workunit.client.0.smithi027.stdout:9/18: dwrite - no filename 2024-04-20T10:59:10.025 INFO:tasks.workunit.client.0.smithi027.stdout:9/19: read - no filename 2024-04-20T10:59:10.025 INFO:tasks.workunit.client.0.smithi027.stdout:2/9: sync 2024-04-20T10:59:10.025 INFO:tasks.workunit.client.0.smithi027.stdout:8/15: fdatasync f2 0 2024-04-20T10:59:10.025 INFO:tasks.workunit.client.0.smithi027.stdout:8/16: dread - f2 zero size 2024-04-20T10:59:10.025 INFO:tasks.workunit.client.0.smithi027.stdout:2/10: read - f3 zero size 2024-04-20T10:59:10.025 INFO:tasks.workunit.client.0.smithi027.stdout:2/11: write f3 [23324,4307] 0 2024-04-20T10:59:10.026 INFO:tasks.workunit.client.0.smithi027.stdout:6/33: mkdir da/dd 0 2024-04-20T10:59:10.027 INFO:tasks.workunit.client.0.smithi027.stdout:1/30: rename d3/c5 to d3/c9 0 2024-04-20T10:59:10.028 INFO:tasks.workunit.client.0.smithi027.stdout:8/17: creat f3 x:0 0 0 2024-04-20T10:59:10.028 INFO:tasks.workunit.client.0.smithi027.stdout:8/18: dread - f2 zero size 2024-04-20T10:59:10.028 INFO:tasks.workunit.client.0.smithi027.stdout:2/12: mkdir d5 0 2024-04-20T10:59:10.031 INFO:tasks.workunit.client.0.smithi027.stdout:5/9: sync 2024-04-20T10:59:10.032 INFO:tasks.workunit.client.0.smithi027.stdout:2/13: mknod d5/c6 0 2024-04-20T10:59:10.042 INFO:tasks.workunit.client.0.smithi027.stdout:4/31: getdents d0 0 2024-04-20T10:59:10.043 INFO:tasks.workunit.client.0.smithi027.stdout:6/34: dwrite f4 [0,4194304] 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:5/10: creat f3 x:0 0 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:5/11: fdatasync f3 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:5/12: truncate f3 582980 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:2/14: mknod d5/c7 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:2/15: chown f3 317579 1 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/32: creat d0/fb x:0 0 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:2/16: stat c4 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:7/38: truncate f1 413820 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:7/39: write f2 [359757,56416] 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:0/36: truncate d0/f2 1075277 0 2024-04-20T10:59:10.298 INFO:tasks.workunit.client.0.smithi027.stdout:1/31: truncate d3/f6 3359593 0 2024-04-20T10:59:10.299 INFO:tasks.workunit.client.0.smithi027.stdout:6/35: link f4 da/dd/fe 0 2024-04-20T10:59:10.299 INFO:tasks.workunit.client.0.smithi027.stdout:6/36: write f7 [701893,13873] 0 2024-04-20T10:59:10.299 INFO:tasks.workunit.client.0.smithi027.stdout:5/13: mknod c4 0 2024-04-20T10:59:10.300 INFO:tasks.workunit.client.0.smithi027.stdout:2/17: creat d5/f8 x:0 0 0 2024-04-20T10:59:10.301 INFO:tasks.workunit.client.0.smithi027.stdout:7/40: creat d4/d5/fa x:0 0 0 2024-04-20T10:59:10.301 INFO:tasks.workunit.client.0.smithi027.stdout:7/41: truncate f2 2057000 0 2024-04-20T10:59:10.302 INFO:tasks.workunit.client.0.smithi027.stdout:0/37: unlink d0/c5 0 2024-04-20T10:59:10.304 INFO:tasks.workunit.client.0.smithi027.stdout:1/32: creat d3/d7/fa x:0 0 0 2024-04-20T10:59:10.304 INFO:tasks.workunit.client.0.smithi027.stdout:1/33: write d3/d7/f8 [754301,3641] 0 2024-04-20T10:59:10.305 INFO:tasks.workunit.client.0.smithi027.stdout:1/34: read d3/d7/f8 [170190,70515] 0 2024-04-20T10:59:10.306 INFO:tasks.workunit.client.0.smithi027.stdout:8/19: sync 2024-04-20T10:59:10.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/20: sync 2024-04-20T10:59:10.306 INFO:tasks.workunit.client.0.smithi027.stdout:6/37: mkdir da/df 0 2024-04-20T10:59:10.307 INFO:tasks.workunit.client.0.smithi027.stdout:5/14: mknod c5 0 2024-04-20T10:59:10.308 INFO:tasks.workunit.client.0.smithi027.stdout:2/18: unlink c4 0 2024-04-20T10:59:10.308 INFO:tasks.workunit.client.0.smithi027.stdout:2/19: write f3 [221309,130658] 0 2024-04-20T10:59:10.312 INFO:tasks.workunit.client.0.smithi027.stdout:9/21: rename l2 to l3 0 2024-04-20T10:59:10.312 INFO:tasks.workunit.client.0.smithi027.stdout:5/15: dread f0 [0,4194304] 0 2024-04-20T10:59:10.312 INFO:tasks.workunit.client.0.smithi027.stdout:0/38: mkdir d0/d8 0 2024-04-20T10:59:10.312 INFO:tasks.workunit.client.0.smithi027.stdout:8/20: symlink l4 0 2024-04-20T10:59:10.312 INFO:tasks.workunit.client.0.smithi027.stdout:8/21: stat l4 0 2024-04-20T10:59:10.313 INFO:tasks.workunit.client.0.smithi027.stdout:9/22: mkdir d4 0 2024-04-20T10:59:10.313 INFO:tasks.workunit.client.0.smithi027.stdout:9/23: write - no filename 2024-04-20T10:59:10.314 INFO:tasks.workunit.client.0.smithi027.stdout:0/39: rmdir d0/d8 0 2024-04-20T10:59:10.315 INFO:tasks.workunit.client.0.smithi027.stdout:8/22: creat f5 x:0 0 0 2024-04-20T10:59:10.316 INFO:tasks.workunit.client.0.smithi027.stdout:8/23: stat f3 0 2024-04-20T10:59:10.317 INFO:tasks.workunit.client.0.smithi027.stdout:9/24: creat d4/f5 x:0 0 0 2024-04-20T10:59:10.317 INFO:tasks.workunit.client.0.smithi027.stdout:9/25: chown l3 856881 1 2024-04-20T10:59:10.317 INFO:tasks.workunit.client.0.smithi027.stdout:4/33: dwrite d0/f6 [0,4194304] 0 2024-04-20T10:59:10.318 INFO:tasks.workunit.client.0.smithi027.stdout:8/24: symlink l6 0 2024-04-20T10:59:10.318 INFO:tasks.workunit.client.0.smithi027.stdout:9/26: creat d4/f6 x:0 0 0 2024-04-20T10:59:10.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/34: getdents d0 0 2024-04-20T10:59:10.320 INFO:tasks.workunit.client.0.smithi027.stdout:4/35: readlink d0/l2 0 2024-04-20T10:59:10.320 INFO:tasks.workunit.client.0.smithi027.stdout:8/25: creat f7 x:0 0 0 2024-04-20T10:59:10.321 INFO:tasks.workunit.client.0.smithi027.stdout:9/27: creat d4/f7 x:0 0 0 2024-04-20T10:59:10.322 INFO:tasks.workunit.client.0.smithi027.stdout:8/26: symlink l8 0 2024-04-20T10:59:10.322 INFO:tasks.workunit.client.0.smithi027.stdout:8/27: dread - f7 zero size 2024-04-20T10:59:10.322 INFO:tasks.workunit.client.0.smithi027.stdout:9/28: rmdir d4 39 2024-04-20T10:59:10.322 INFO:tasks.workunit.client.0.smithi027.stdout:9/29: truncate d4/f6 312943 0 2024-04-20T10:59:10.323 INFO:tasks.workunit.client.0.smithi027.stdout:6/38: dread da/dd/fe [0,4194304] 0 2024-04-20T10:59:10.323 INFO:tasks.workunit.client.0.smithi027.stdout:6/39: write f7 [1057505,43295] 0 2024-04-20T10:59:10.323 INFO:tasks.workunit.client.0.smithi027.stdout:3/40: dwrite d2/f7 [4194304,4194304] 0 2024-04-20T10:59:10.324 INFO:tasks.workunit.client.0.smithi027.stdout:3/41: stat d2/fc 0 2024-04-20T10:59:10.324 INFO:tasks.workunit.client.0.smithi027.stdout:7/42: dwrite f1 [0,4194304] 0 2024-04-20T10:59:10.324 INFO:tasks.workunit.client.0.smithi027.stdout:9/30: mknod d4/c8 0 2024-04-20T10:59:10.324 INFO:tasks.workunit.client.0.smithi027.stdout:9/31: rename d4 to d4/d9 22 2024-04-20T10:59:10.324 INFO:tasks.workunit.client.0.smithi027.stdout:9/32: rename d4 to d4/da 22 2024-04-20T10:59:10.326 INFO:tasks.workunit.client.0.smithi027.stdout:6/40: dread da/fb [0,4194304] 0 2024-04-20T10:59:10.326 INFO:tasks.workunit.client.0.smithi027.stdout:3/42: mkdir d2/dd 0 2024-04-20T10:59:10.334 INFO:tasks.workunit.client.0.smithi027.stdout:6/41: creat da/df/f10 x:0 0 0 2024-04-20T10:59:10.337 INFO:tasks.workunit.client.0.smithi027.stdout:0/40: getdents d0 0 2024-04-20T10:59:10.337 INFO:tasks.workunit.client.0.smithi027.stdout:0/41: fsync d0/f2 0 2024-04-20T10:59:10.337 INFO:tasks.workunit.client.0.smithi027.stdout:0/42: write d0/f2 [344379,96286] 0 2024-04-20T10:59:10.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/43: write d2/f5 [3157553,18631] 0 2024-04-20T10:59:10.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/44: truncate d2/fc 357741 0 2024-04-20T10:59:10.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/45: write d2/fc [562680,128737] 0 2024-04-20T10:59:10.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/46: write d2/f6 [455849,2001] 0 2024-04-20T10:59:10.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/47: write d2/fc [1015113,121461] 0 2024-04-20T10:59:10.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/48: write d2/f6 [534413,37471] 0 2024-04-20T10:59:10.341 INFO:tasks.workunit.client.0.smithi027.stdout:6/42: creat da/df/f11 x:0 0 0 2024-04-20T10:59:10.341 INFO:tasks.workunit.client.0.smithi027.stdout:6/43: chown c5 874 1 2024-04-20T10:59:10.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/20: dwrite f1 [0,4194304] 0 2024-04-20T10:59:10.344 INFO:tasks.workunit.client.0.smithi027.stdout:4/36: dwrite d0/fb [0,4194304] 0 2024-04-20T10:59:10.345 INFO:tasks.workunit.client.0.smithi027.stdout:3/49: link d2/f5 d2/fe 0 2024-04-20T10:59:10.345 INFO:tasks.workunit.client.0.smithi027.stdout:9/33: dwrite d4/f5 [0,4194304] 0 2024-04-20T10:59:10.345 INFO:tasks.workunit.client.0.smithi027.stdout:9/34: stat d4/c8 0 2024-04-20T10:59:10.349 INFO:tasks.workunit.client.0.smithi027.stdout:3/50: mkdir d2/dd/df 0 2024-04-20T10:59:10.351 INFO:tasks.workunit.client.0.smithi027.stdout:3/51: creat d2/dd/f10 x:0 0 0 2024-04-20T10:59:10.365 INFO:tasks.workunit.client.0.smithi027.stdout:8/28: dwrite f5 [0,4194304] 0 2024-04-20T10:59:10.365 INFO:tasks.workunit.client.0.smithi027.stdout:8/29: chown l4 43 1 2024-04-20T10:59:10.368 INFO:tasks.workunit.client.0.smithi027.stdout:8/30: mkdir d9 0 2024-04-20T10:59:10.368 INFO:tasks.workunit.client.0.smithi027.stdout:8/31: getdents d9 0 2024-04-20T10:59:10.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/32: rename l4 to d9/la 0 2024-04-20T10:59:10.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/33: stat f3 0 2024-04-20T10:59:10.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/34: dread - f3 zero size 2024-04-20T10:59:10.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/35: rmdir d9 39 2024-04-20T10:59:10.370 INFO:tasks.workunit.client.0.smithi027.stdout:2/21: dwrite d5/f8 [0,4194304] 0 2024-04-20T10:59:10.370 INFO:tasks.workunit.client.0.smithi027.stdout:2/22: chown f3 11 1 2024-04-20T10:59:10.370 INFO:tasks.workunit.client.0.smithi027.stdout:8/36: creat d9/fb x:0 0 0 2024-04-20T10:59:10.372 INFO:tasks.workunit.client.0.smithi027.stdout:1/35: dwrite d3/d7/f8 [0,4194304] 0 2024-04-20T10:59:10.372 INFO:tasks.workunit.client.0.smithi027.stdout:1/36: chown d3/c4 13518943 1 2024-04-20T10:59:10.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/37: unlink l6 0 2024-04-20T10:59:10.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/38: readlink l8 0 2024-04-20T10:59:10.373 INFO:tasks.workunit.client.0.smithi027.stdout:9/35: dwrite d4/f5 [0,4194304] 0 2024-04-20T10:59:10.374 INFO:tasks.workunit.client.0.smithi027.stdout:1/37: symlink d3/d7/lb 0 2024-04-20T10:59:10.374 INFO:tasks.workunit.client.0.smithi027.stdout:7/43: dwrite f1 [0,4194304] 0 2024-04-20T10:59:10.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/52: dwrite d2/f7 [4194304,4194304] 0 2024-04-20T10:59:10.375 INFO:tasks.workunit.client.0.smithi027.stdout:8/39: symlink d9/lc 0 2024-04-20T10:59:10.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/36: getdents d4 0 2024-04-20T10:59:10.376 INFO:tasks.workunit.client.0.smithi027.stdout:0/43: dwrite d0/f2 [0,4194304] 0 2024-04-20T10:59:10.376 INFO:tasks.workunit.client.0.smithi027.stdout:0/44: chown d0/f2 2473183 1 2024-04-20T10:59:10.376 INFO:tasks.workunit.client.0.smithi027.stdout:9/37: write d4/f6 [164456,26880] 0 2024-04-20T10:59:10.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/38: creat d3/d7/fc x:0 0 0 2024-04-20T10:59:10.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/44: mknod d4/cb 0 2024-04-20T10:59:10.379 INFO:tasks.workunit.client.0.smithi027.stdout:3/53: dread d2/f5 [0,4194304] 0 2024-04-20T10:59:10.379 INFO:tasks.workunit.client.0.smithi027.stdout:3/54: readlink d2/la 0 2024-04-20T10:59:10.381 INFO:tasks.workunit.client.0.smithi027.stdout:9/38: unlink d4/f5 0 2024-04-20T10:59:10.383 INFO:tasks.workunit.client.0.smithi027.stdout:3/55: link d2/cb d2/c11 0 2024-04-20T10:59:10.384 INFO:tasks.workunit.client.0.smithi027.stdout:3/56: creat d2/f12 x:0 0 0 2024-04-20T10:59:10.385 INFO:tasks.workunit.client.0.smithi027.stdout:3/57: dread d2/f9 [0,4194304] 0 2024-04-20T10:59:10.391 INFO:tasks.workunit.client.0.smithi027.stdout:3/58: creat d2/dd/f13 x:0 0 0 2024-04-20T10:59:10.393 INFO:tasks.workunit.client.0.smithi027.stdout:3/59: unlink d2/dd/f13 0 2024-04-20T10:59:10.393 INFO:tasks.workunit.client.0.smithi027.stdout:3/60: truncate d2/f7 8444608 0 2024-04-20T10:59:10.393 INFO:tasks.workunit.client.0.smithi027.stdout:3/61: chown d2/dd/df 20 1 2024-04-20T10:59:10.393 INFO:tasks.workunit.client.0.smithi027.stdout:9/39: dread d4/f6 [0,4194304] 0 2024-04-20T10:59:10.393 INFO:tasks.workunit.client.0.smithi027.stdout:3/62: creat d2/f14 x:0 0 0 2024-04-20T10:59:10.393 INFO:tasks.workunit.client.0.smithi027.stdout:3/63: write d2/f5 [3775668,126970] 0 2024-04-20T10:59:10.394 INFO:tasks.workunit.client.0.smithi027.stdout:9/40: rename d4/c8 to d4/cb 0 2024-04-20T10:59:10.395 INFO:tasks.workunit.client.0.smithi027.stdout:3/64: rename d2/cb to d2/dd/df/c15 0 2024-04-20T10:59:10.396 INFO:tasks.workunit.client.0.smithi027.stdout:9/41: truncate d4/f6 214895 0 2024-04-20T10:59:10.396 INFO:tasks.workunit.client.0.smithi027.stdout:0/45: dread d0/f2 [0,4194304] 0 2024-04-20T10:59:10.396 INFO:tasks.workunit.client.0.smithi027.stdout:2/23: dwrite f1 [0,4194304] 0 2024-04-20T10:59:10.397 INFO:tasks.workunit.client.0.smithi027.stdout:3/65: link d2/dd/f10 d2/f16 0 2024-04-20T10:59:10.397 INFO:tasks.workunit.client.0.smithi027.stdout:3/66: write d2/f4 [545478,95502] 0 2024-04-20T10:59:10.397 INFO:tasks.workunit.client.0.smithi027.stdout:2/24: rmdir d5 39 2024-04-20T10:59:10.399 INFO:tasks.workunit.client.0.smithi027.stdout:0/46: write d0/f2 [3220348,86920] 0 2024-04-20T10:59:10.406 INFO:tasks.workunit.client.0.smithi027.stdout:1/39: dwrite d3/f6 [0,4194304] 0 2024-04-20T10:59:10.406 INFO:tasks.workunit.client.0.smithi027.stdout:1/40: read - d3/d7/fa zero size 2024-04-20T10:59:10.407 INFO:tasks.workunit.client.0.smithi027.stdout:1/41: creat d3/d7/fd x:0 0 0 2024-04-20T10:59:10.407 INFO:tasks.workunit.client.0.smithi027.stdout:1/42: write d3/d7/fa [901309,67780] 0 2024-04-20T10:59:10.408 INFO:tasks.workunit.client.0.smithi027.stdout:7/45: dwrite d4/d5/fa [0,4194304] 0 2024-04-20T10:59:10.410 INFO:tasks.workunit.client.0.smithi027.stdout:2/25: dread d5/f8 [0,4194304] 0 2024-04-20T10:59:10.419 INFO:tasks.workunit.client.0.smithi027.stdout:7/46: dread d4/d5/fa [0,4194304] 0 2024-04-20T10:59:10.419 INFO:tasks.workunit.client.0.smithi027.stdout:3/67: dwrite d2/fc [0,4194304] 0 2024-04-20T10:59:10.424 INFO:tasks.workunit.client.0.smithi027.stdout:7/47: dread d4/d5/fa [0,4194304] 0 2024-04-20T10:59:10.426 INFO:tasks.workunit.client.0.smithi027.stdout:5/16: sync 2024-04-20T10:59:10.426 INFO:tasks.workunit.client.0.smithi027.stdout:5/17: rmdir - no directory 2024-04-20T10:59:10.426 INFO:tasks.workunit.client.0.smithi027.stdout:5/18: fdatasync f3 0 2024-04-20T10:59:10.430 INFO:tasks.workunit.client.0.smithi027.stdout:5/19: dread f2 [0,4194304] 0 2024-04-20T10:59:10.430 INFO:tasks.workunit.client.0.smithi027.stdout:5/20: write f3 [1416159,69683] 0 2024-04-20T10:59:10.431 INFO:tasks.workunit.client.0.smithi027.stdout:5/21: dread f2 [4194304,4194304] 0 2024-04-20T10:59:10.434 INFO:tasks.workunit.client.0.smithi027.stdout:4/37: dread d0/fb [0,4194304] 0 2024-04-20T10:59:10.435 INFO:tasks.workunit.client.0.smithi027.stdout:4/38: dread d0/f4 [0,4194304] 0 2024-04-20T10:59:10.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/42: dwrite d4/f6 [0,4194304] 0 2024-04-20T10:59:10.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/43: dread - d4/f7 zero size 2024-04-20T10:59:10.440 INFO:tasks.workunit.client.0.smithi027.stdout:9/44: rename d4/f6 to d4/fc 0 2024-04-20T10:59:10.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/39: rename d0/f4 to d0/fc 0 2024-04-20T10:59:10.442 INFO:tasks.workunit.client.0.smithi027.stdout:4/40: creat d0/fd x:0 0 0 2024-04-20T10:59:10.445 INFO:tasks.workunit.client.0.smithi027.stdout:1/43: dwrite d3/d7/fa [0,4194304] 0 2024-04-20T10:59:10.445 INFO:tasks.workunit.client.0.smithi027.stdout:1/44: fdatasync d3/d7/fd 0 2024-04-20T10:59:10.445 INFO:tasks.workunit.client.0.smithi027.stdout:2/26: dwrite d5/f8 [4194304,4194304] 0 2024-04-20T10:59:10.445 INFO:tasks.workunit.client.0.smithi027.stdout:2/27: fsync f1 0 2024-04-20T10:59:10.447 INFO:tasks.workunit.client.0.smithi027.stdout:3/68: dwrite d2/f9 [0,4194304] 0 2024-04-20T10:59:10.447 INFO:tasks.workunit.client.0.smithi027.stdout:1/45: symlink d3/d7/le 0 2024-04-20T10:59:10.448 INFO:tasks.workunit.client.0.smithi027.stdout:2/28: creat d5/f9 x:0 0 0 2024-04-20T10:59:10.449 INFO:tasks.workunit.client.0.smithi027.stdout:4/41: mknod d0/ce 0 2024-04-20T10:59:10.449 INFO:tasks.workunit.client.0.smithi027.stdout:4/42: read - d0/f9 zero size 2024-04-20T10:59:10.449 INFO:tasks.workunit.client.0.smithi027.stdout:5/22: dwrite f2 [0,4194304] 0 2024-04-20T10:59:10.449 INFO:tasks.workunit.client.0.smithi027.stdout:9/45: truncate d4/fc 1650010 0 2024-04-20T10:59:10.451 INFO:tasks.workunit.client.0.smithi027.stdout:3/69: rename d2/f7 to d2/f17 0 2024-04-20T10:59:10.453 INFO:tasks.workunit.client.0.smithi027.stdout:2/29: unlink d5/f9 0 2024-04-20T10:59:10.454 INFO:tasks.workunit.client.0.smithi027.stdout:4/43: stat d0/l2 0 2024-04-20T10:59:10.457 INFO:tasks.workunit.client.0.smithi027.stdout:5/23: creat f6 x:0 0 0 2024-04-20T10:59:10.458 INFO:tasks.workunit.client.0.smithi027.stdout:1/46: read d3/d7/f8 [1841091,107574] 0 2024-04-20T10:59:10.458 INFO:tasks.workunit.client.0.smithi027.stdout:9/46: mknod d4/cd 0 2024-04-20T10:59:10.458 INFO:tasks.workunit.client.0.smithi027.stdout:1/47: write d3/d7/f8 [2461128,15903] 0 2024-04-20T10:59:10.461 INFO:tasks.workunit.client.0.smithi027.stdout:3/70: mkdir d2/d18 0 2024-04-20T10:59:10.462 INFO:tasks.workunit.client.0.smithi027.stdout:3/71: chown d2/f17 1506491557 1 2024-04-20T10:59:10.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/24: creat f7 x:0 0 0 2024-04-20T10:59:10.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/25: dread - f7 zero size 2024-04-20T10:59:10.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/26: write f0 [5320559,40330] 0 2024-04-20T10:59:10.463 INFO:tasks.workunit.client.0.smithi027.stdout:6/44: sync 2024-04-20T10:59:10.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/48: creat d3/d7/ff x:0 0 0 2024-04-20T10:59:10.464 INFO:tasks.workunit.client.0.smithi027.stdout:4/44: symlink d0/lf 0 2024-04-20T10:59:10.464 INFO:tasks.workunit.client.0.smithi027.stdout:4/45: fdatasync d0/fb 0 2024-04-20T10:59:10.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/27: creat f8 x:0 0 0 2024-04-20T10:59:10.465 INFO:tasks.workunit.client.0.smithi027.stdout:6/45: symlink da/df/l12 0 2024-04-20T10:59:10.466 INFO:tasks.workunit.client.0.smithi027.stdout:3/72: dread d2/fe [0,4194304] 0 2024-04-20T10:59:10.466 INFO:tasks.workunit.client.0.smithi027.stdout:3/73: dread - d2/dd/f10 zero size 2024-04-20T10:59:10.466 INFO:tasks.workunit.client.0.smithi027.stdout:1/49: rename d3/d7/lb to d3/d7/l10 0 2024-04-20T10:59:10.468 INFO:tasks.workunit.client.0.smithi027.stdout:5/28: creat f9 x:0 0 0 2024-04-20T10:59:10.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/29: read f0 [4846023,24230] 0 2024-04-20T10:59:10.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/30: write f8 [617538,98690] 0 2024-04-20T10:59:10.469 INFO:tasks.workunit.client.0.smithi027.stdout:5/31: dread - f7 zero size 2024-04-20T10:59:10.469 INFO:tasks.workunit.client.0.smithi027.stdout:7/48: dread f1 [0,4194304] 0 2024-04-20T10:59:10.470 INFO:tasks.workunit.client.0.smithi027.stdout:6/46: creat da/dd/f13 x:0 0 0 2024-04-20T10:59:10.471 INFO:tasks.workunit.client.0.smithi027.stdout:3/74: creat d2/dd/f19 x:0 0 0 2024-04-20T10:59:10.471 INFO:tasks.workunit.client.0.smithi027.stdout:3/75: chown d2/f16 0 1 2024-04-20T10:59:10.472 INFO:tasks.workunit.client.0.smithi027.stdout:7/49: dread f2 [0,4194304] 0 2024-04-20T10:59:10.472 INFO:tasks.workunit.client.0.smithi027.stdout:4/46: rename d0/l7 to d0/l10 0 2024-04-20T10:59:10.472 INFO:tasks.workunit.client.0.smithi027.stdout:4/47: readlink d0/l2 0 2024-04-20T10:59:10.473 INFO:tasks.workunit.client.0.smithi027.stdout:5/32: mkdir da 0 2024-04-20T10:59:10.474 INFO:tasks.workunit.client.0.smithi027.stdout:6/47: creat da/dd/f14 x:0 0 0 2024-04-20T10:59:10.475 INFO:tasks.workunit.client.0.smithi027.stdout:7/50: creat d4/fc x:0 0 0 2024-04-20T10:59:10.480 INFO:tasks.workunit.client.0.smithi027.stdout:9/47: dwrite d4/f7 [0,4194304] 0 2024-04-20T10:59:10.480 INFO:tasks.workunit.client.0.smithi027.stdout:9/48: fsync d4/fc 0 2024-04-20T10:59:10.480 INFO:tasks.workunit.client.0.smithi027.stdout:9/49: write d4/fc [2350325,83496] 0 2024-04-20T10:59:10.480 INFO:tasks.workunit.client.0.smithi027.stdout:9/50: chown d4 1713503 1 2024-04-20T10:59:10.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/33: rmdir da 0 2024-04-20T10:59:10.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/34: write f6 [505645,76092] 0 2024-04-20T10:59:10.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/35: truncate f8 1688950 0 2024-04-20T10:59:10.482 INFO:tasks.workunit.client.0.smithi027.stdout:1/50: dread d3/d7/f8 [0,4194304] 0 2024-04-20T10:59:10.482 INFO:tasks.workunit.client.0.smithi027.stdout:1/51: chown d3/c4 111 1 2024-04-20T10:59:10.482 INFO:tasks.workunit.client.0.smithi027.stdout:1/52: write d3/d7/f8 [2519773,103550] 0 2024-04-20T10:59:10.482 INFO:tasks.workunit.client.0.smithi027.stdout:6/48: creat da/f15 x:0 0 0 2024-04-20T10:59:10.485 INFO:tasks.workunit.client.0.smithi027.stdout:2/30: dwrite d5/f8 [4194304,4194304] 0 2024-04-20T10:59:10.496 INFO:tasks.workunit.client.0.smithi027.stdout:7/51: dread d4/d5/fa [0,4194304] 0 2024-04-20T10:59:10.496 INFO:tasks.workunit.client.0.smithi027.stdout:3/76: dwrite d2/fc [0,4194304] 0 2024-04-20T10:59:10.496 INFO:tasks.workunit.client.0.smithi027.stdout:7/52: write d4/fc [934729,99379] 0 2024-04-20T10:59:10.497 INFO:tasks.workunit.client.0.smithi027.stdout:7/53: write d4/fc [21377,128109] 0 2024-04-20T10:59:10.497 INFO:tasks.workunit.client.0.smithi027.stdout:7/54: chown l3 363831 1 2024-04-20T10:59:10.498 INFO:tasks.workunit.client.0.smithi027.stdout:5/36: rename f6 to fb 0 2024-04-20T10:59:10.498 INFO:tasks.workunit.client.0.smithi027.stdout:1/53: creat d3/f11 x:0 0 0 2024-04-20T10:59:10.498 INFO:tasks.workunit.client.0.smithi027.stdout:1/54: chown d3/d7/f8 207223 1 2024-04-20T10:59:10.501 INFO:tasks.workunit.client.0.smithi027.stdout:6/49: dwrite da/dd/fe [0,4194304] 0 2024-04-20T10:59:10.501 INFO:tasks.workunit.client.0.smithi027.stdout:6/50: fdatasync da/fb 0 2024-04-20T10:59:10.501 INFO:tasks.workunit.client.0.smithi027.stdout:6/51: stat l6 0 2024-04-20T10:59:10.501 INFO:tasks.workunit.client.0.smithi027.stdout:3/77: link d2/f5 d2/dd/f1a 0 2024-04-20T10:59:10.502 INFO:tasks.workunit.client.0.smithi027.stdout:7/55: mkdir d4/d5/d8/dd 0 2024-04-20T10:59:10.503 INFO:tasks.workunit.client.0.smithi027.stdout:1/55: rename d3/f6 to d3/d7/f12 0 2024-04-20T10:59:10.504 INFO:tasks.workunit.client.0.smithi027.stdout:6/52: creat da/df/f16 x:0 0 0 2024-04-20T10:59:10.505 INFO:tasks.workunit.client.0.smithi027.stdout:3/78: mknod d2/dd/df/c1b 0 2024-04-20T10:59:10.506 INFO:tasks.workunit.client.0.smithi027.stdout:7/56: symlink d4/le 0 2024-04-20T10:59:10.507 INFO:tasks.workunit.client.0.smithi027.stdout:2/31: dwrite f1 [0,4194304] 0 2024-04-20T10:59:10.507 INFO:tasks.workunit.client.0.smithi027.stdout:6/53: rename da/df/f16 to da/df/f17 0 2024-04-20T10:59:10.508 INFO:tasks.workunit.client.0.smithi027.stdout:3/79: mknod d2/dd/c1c 0 2024-04-20T10:59:10.508 INFO:tasks.workunit.client.0.smithi027.stdout:3/80: write d2/fe [1847000,112111] 0 2024-04-20T10:59:10.508 INFO:tasks.workunit.client.0.smithi027.stdout:7/57: rmdir d4/d5/d8 39 2024-04-20T10:59:10.510 INFO:tasks.workunit.client.0.smithi027.stdout:1/56: dread d3/d7/f12 [0,4194304] 0 2024-04-20T10:59:10.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/32: link f1 d5/fa 0 2024-04-20T10:59:10.514 INFO:tasks.workunit.client.0.smithi027.stdout:9/51: write d4/f7 [5240614,120010] 0 2024-04-20T10:59:10.514 INFO:tasks.workunit.client.0.smithi027.stdout:9/52: chown d4/f7 30222037 1 2024-04-20T10:59:10.515 INFO:tasks.workunit.client.0.smithi027.stdout:9/53: chown d4 17 1 2024-04-20T10:59:10.516 INFO:tasks.workunit.client.0.smithi027.stdout:6/54: link da/df/l12 da/l18 0 2024-04-20T10:59:10.516 INFO:tasks.workunit.client.0.smithi027.stdout:3/81: read d2/f17 [1493100,97732] 0 2024-04-20T10:59:10.516 INFO:tasks.workunit.client.0.smithi027.stdout:5/37: link fb fc 0 2024-04-20T10:59:10.518 INFO:tasks.workunit.client.0.smithi027.stdout:1/57: rename d3/d7/l10 to d3/l13 0 2024-04-20T10:59:10.518 INFO:tasks.workunit.client.0.smithi027.stdout:1/58: readlink d3/d7/le 0 2024-04-20T10:59:10.520 INFO:tasks.workunit.client.0.smithi027.stdout:3/82: getdents d2 0 2024-04-20T10:59:10.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/40: sync 2024-04-20T10:59:10.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/41: fdatasync f3 0 2024-04-20T10:59:10.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/42: write f7 [712363,12610] 0 2024-04-20T10:59:10.522 INFO:tasks.workunit.client.0.smithi027.stdout:1/59: symlink d3/d7/l14 0 2024-04-20T10:59:10.531 INFO:tasks.workunit.client.0.smithi027.stdout:2/33: write f1 [1162996,130014] 0 2024-04-20T10:59:10.533 INFO:tasks.workunit.client.0.smithi027.stdout:4/48: dread d0/fb [0,4194304] 0 2024-04-20T10:59:10.533 INFO:tasks.workunit.client.0.smithi027.stdout:8/43: creat d9/fd x:0 0 0 2024-04-20T10:59:10.533 INFO:tasks.workunit.client.0.smithi027.stdout:1/60: mknod d3/d7/c15 0 2024-04-20T10:59:10.534 INFO:tasks.workunit.client.0.smithi027.stdout:7/58: dwrite d4/d5/fa [0,4194304] 0 2024-04-20T10:59:10.536 INFO:tasks.workunit.client.0.smithi027.stdout:0/47: sync 2024-04-20T10:59:10.537 INFO:tasks.workunit.client.0.smithi027.stdout:8/44: unlink d9/la 0 2024-04-20T10:59:10.539 INFO:tasks.workunit.client.0.smithi027.stdout:1/61: mknod d3/d7/c16 0 2024-04-20T10:59:10.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/54: truncate d4/f7 4668830 0 2024-04-20T10:59:10.540 INFO:tasks.workunit.client.0.smithi027.stdout:9/55: fdatasync d4/fc 0 2024-04-20T10:59:10.541 INFO:tasks.workunit.client.0.smithi027.stdout:4/49: mknod d0/c11 0 2024-04-20T10:59:10.541 INFO:tasks.workunit.client.0.smithi027.stdout:4/50: chown d0/lf 146898 1 2024-04-20T10:59:10.541 INFO:tasks.workunit.client.0.smithi027.stdout:1/62: rmdir d3/d7 39 2024-04-20T10:59:10.542 INFO:tasks.workunit.client.0.smithi027.stdout:9/56: rmdir d4 39 2024-04-20T10:59:10.546 INFO:tasks.workunit.client.0.smithi027.stdout:6/55: dwrite da/dd/f14 [0,4194304] 0 2024-04-20T10:59:10.548 INFO:tasks.workunit.client.0.smithi027.stdout:3/83: dwrite d2/f14 [0,4194304] 0 2024-04-20T10:59:10.548 INFO:tasks.workunit.client.0.smithi027.stdout:4/51: creat d0/f12 x:0 0 0 2024-04-20T10:59:10.548 INFO:tasks.workunit.client.0.smithi027.stdout:1/63: creat d3/f17 x:0 0 0 2024-04-20T10:59:10.554 INFO:tasks.workunit.client.0.smithi027.stdout:8/45: getdents d9 0 2024-04-20T10:59:10.554 INFO:tasks.workunit.client.0.smithi027.stdout:8/46: write f2 [989797,46550] 0 2024-04-20T10:59:10.554 INFO:tasks.workunit.client.0.smithi027.stdout:8/47: chown d9 71200773 1 2024-04-20T10:59:10.555 INFO:tasks.workunit.client.0.smithi027.stdout:6/56: rmdir da/dd 39 2024-04-20T10:59:10.555 INFO:tasks.workunit.client.0.smithi027.stdout:3/84: rename d2/f4 to d2/f1d 0 2024-04-20T10:59:10.557 INFO:tasks.workunit.client.0.smithi027.stdout:1/64: rename d3/c9 to d3/c18 0 2024-04-20T10:59:10.557 INFO:tasks.workunit.client.0.smithi027.stdout:1/65: fdatasync d3/d7/f8 0 2024-04-20T10:59:10.558 INFO:tasks.workunit.client.0.smithi027.stdout:6/57: mkdir da/df/d19 0 2024-04-20T10:59:10.561 INFO:tasks.workunit.client.0.smithi027.stdout:3/85: getdents d2/dd 0 2024-04-20T10:59:10.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/57: write d4/f7 [3343640,94874] 0 2024-04-20T10:59:10.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/58: write d4/f7 [5498854,54139] 0 2024-04-20T10:59:10.566 INFO:tasks.workunit.client.0.smithi027.stdout:0/48: dwrite d0/f2 [0,4194304] 0 2024-04-20T10:59:10.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/34: dwrite d5/fa [0,4194304] 0 2024-04-20T10:59:10.566 INFO:tasks.workunit.client.0.smithi027.stdout:0/49: write d0/f2 [4337693,72233] 0 2024-04-20T10:59:10.568 INFO:tasks.workunit.client.0.smithi027.stdout:4/52: link d0/fc d0/f13 0 2024-04-20T10:59:10.568 INFO:tasks.workunit.client.0.smithi027.stdout:4/53: chown d0/f13 206053913 1 2024-04-20T10:59:10.569 INFO:tasks.workunit.client.0.smithi027.stdout:1/66: mknod d3/d7/c19 0 2024-04-20T10:59:10.569 INFO:tasks.workunit.client.0.smithi027.stdout:6/58: dread f3 [0,4194304] 0 2024-04-20T10:59:10.571 INFO:tasks.workunit.client.0.smithi027.stdout:9/59: mknod d4/ce 0 2024-04-20T10:59:10.572 INFO:tasks.workunit.client.0.smithi027.stdout:2/35: mknod d5/cb 0 2024-04-20T10:59:10.572 INFO:tasks.workunit.client.0.smithi027.stdout:2/36: chown d5/c7 1 1 2024-04-20T10:59:10.573 INFO:tasks.workunit.client.0.smithi027.stdout:0/50: mknod d0/c9 0 2024-04-20T10:59:10.573 INFO:tasks.workunit.client.0.smithi027.stdout:4/54: truncate d0/f6 2617369 0 2024-04-20T10:59:10.576 INFO:tasks.workunit.client.0.smithi027.stdout:1/67: creat d3/d7/f1a x:0 0 0 2024-04-20T10:59:10.577 INFO:tasks.workunit.client.0.smithi027.stdout:9/60: rmdir d4 39 2024-04-20T10:59:10.578 INFO:tasks.workunit.client.0.smithi027.stdout:2/37: mkdir d5/dc 0 2024-04-20T10:59:10.579 INFO:tasks.workunit.client.0.smithi027.stdout:1/68: creat d3/f1b x:0 0 0 2024-04-20T10:59:10.581 INFO:tasks.workunit.client.0.smithi027.stdout:3/86: dwrite d2/f17 [0,4194304] 0 2024-04-20T10:59:10.581 INFO:tasks.workunit.client.0.smithi027.stdout:6/59: dread da/dd/f14 [0,4194304] 0 2024-04-20T10:59:10.582 INFO:tasks.workunit.client.0.smithi027.stdout:2/38: creat d5/fd x:0 0 0 2024-04-20T10:59:10.583 INFO:tasks.workunit.client.0.smithi027.stdout:4/55: mkdir d0/d14 0 2024-04-20T10:59:10.584 INFO:tasks.workunit.client.0.smithi027.stdout:3/87: creat d2/f1e x:0 0 0 2024-04-20T10:59:10.584 INFO:tasks.workunit.client.0.smithi027.stdout:3/88: write d2/f16 [474166,60034] 0 2024-04-20T10:59:10.585 INFO:tasks.workunit.client.0.smithi027.stdout:1/69: dread d3/d7/f12 [0,4194304] 0 2024-04-20T10:59:10.585 INFO:tasks.workunit.client.0.smithi027.stdout:1/70: readlink d3/d7/l14 0 2024-04-20T10:59:10.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/51: dwrite d0/f2 [0,4194304] 0 2024-04-20T10:59:10.595 INFO:tasks.workunit.client.0.smithi027.stdout:6/60: dread da/dd/f14 [0,4194304] 0 2024-04-20T10:59:10.597 INFO:tasks.workunit.client.0.smithi027.stdout:9/61: dread d4/fc [0,4194304] 0 2024-04-20T10:59:10.602 INFO:tasks.workunit.client.0.smithi027.stdout:0/52: write d0/f2 [2665010,44070] 0 2024-04-20T10:59:10.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/53: fdatasync d0/f2 0 2024-04-20T10:59:10.610 INFO:tasks.workunit.client.0.smithi027.stdout:0/54: dread d0/f2 [0,4194304] 0 2024-04-20T10:59:10.841 INFO:tasks.workunit.client.0.smithi027.stdout:3/89: creat d2/f1f x:0 0 0 2024-04-20T10:59:10.841 INFO:tasks.workunit.client.0.smithi027.stdout:2/39: getdents d5 0 2024-04-20T10:59:10.842 INFO:tasks.workunit.client.0.smithi027.stdout:1/71: creat d3/f1c x:0 0 0 2024-04-20T10:59:10.842 INFO:tasks.workunit.client.0.smithi027.stdout:1/72: truncate d3/d7/ff 1023468 0 2024-04-20T10:59:10.844 INFO:tasks.workunit.client.0.smithi027.stdout:6/61: rename da/f15 to da/df/f1a 0 2024-04-20T10:59:10.845 INFO:tasks.workunit.client.0.smithi027.stdout:9/62: symlink d4/lf 0 2024-04-20T10:59:10.850 INFO:tasks.workunit.client.0.smithi027.stdout:0/55: creat d0/d7/fa x:0 0 0 2024-04-20T10:59:10.850 INFO:tasks.workunit.client.0.smithi027.stdout:4/56: mkdir d0/d15 0 2024-04-20T10:59:10.850 INFO:tasks.workunit.client.0.smithi027.stdout:2/40: unlink d5/fa 0 2024-04-20T10:59:10.851 INFO:tasks.workunit.client.0.smithi027.stdout:1/73: link d3/d7/ff d3/d7/f1d 0 2024-04-20T10:59:10.851 INFO:tasks.workunit.client.0.smithi027.stdout:5/38: sync 2024-04-20T10:59:10.853 INFO:tasks.workunit.client.0.smithi027.stdout:9/63: rename l1 to d4/l10 0 2024-04-20T10:59:10.853 INFO:tasks.workunit.client.0.smithi027.stdout:8/48: sync 2024-04-20T10:59:10.853 INFO:tasks.workunit.client.0.smithi027.stdout:9/64: truncate d4/fc 2663433 0 2024-04-20T10:59:10.855 INFO:tasks.workunit.client.0.smithi027.stdout:1/74: creat d3/f1e x:0 0 0 2024-04-20T10:59:10.855 INFO:tasks.workunit.client.0.smithi027.stdout:5/39: getdents . 0 2024-04-20T10:59:10.855 INFO:tasks.workunit.client.0.smithi027.stdout:8/49: mkdir d9/de 0 2024-04-20T10:59:10.855 INFO:tasks.workunit.client.0.smithi027.stdout:8/50: write d9/fb [667270,25987] 0 2024-04-20T10:59:10.855 INFO:tasks.workunit.client.0.smithi027.stdout:8/51: write d9/fd [747491,38156] 0 2024-04-20T10:59:10.856 INFO:tasks.workunit.client.0.smithi027.stdout:5/40: rename f7 to fd 0 2024-04-20T10:59:10.856 INFO:tasks.workunit.client.0.smithi027.stdout:5/41: fdatasync f9 0 2024-04-20T10:59:10.856 INFO:tasks.workunit.client.0.smithi027.stdout:5/42: stat f8 0 2024-04-20T10:59:10.856 INFO:tasks.workunit.client.0.smithi027.stdout:5/43: chown c4 60502867 1 2024-04-20T10:59:10.857 INFO:tasks.workunit.client.0.smithi027.stdout:5/44: mknod ce 0 2024-04-20T10:59:10.857 INFO:tasks.workunit.client.0.smithi027.stdout:5/45: chown fd 3 1 2024-04-20T10:59:10.857 INFO:tasks.workunit.client.0.smithi027.stdout:5/46: readlink - no filename 2024-04-20T10:59:10.857 INFO:tasks.workunit.client.0.smithi027.stdout:5/47: chown fc 7820 1 2024-04-20T10:59:10.857 INFO:tasks.workunit.client.0.smithi027.stdout:5/48: mknod cf 0 2024-04-20T10:59:10.864 INFO:tasks.workunit.client.0.smithi027.stdout:3/90: dwrite d2/f1d [0,4194304] 0 2024-04-20T10:59:10.864 INFO:tasks.workunit.client.0.smithi027.stdout:3/91: fdatasync d2/dd/f1a 0 2024-04-20T10:59:10.866 INFO:tasks.workunit.client.0.smithi027.stdout:8/52: write f5 [1697225,126834] 0 2024-04-20T10:59:10.866 INFO:tasks.workunit.client.0.smithi027.stdout:3/92: creat d2/dd/f20 x:0 0 0 2024-04-20T10:59:10.866 INFO:tasks.workunit.client.0.smithi027.stdout:3/93: write d2/f12 [729607,25539] 0 2024-04-20T10:59:10.868 INFO:tasks.workunit.client.0.smithi027.stdout:9/65: dread d4/f7 [0,4194304] 0 2024-04-20T10:59:10.869 INFO:tasks.workunit.client.0.smithi027.stdout:7/59: read f1 [2963798,8234] 0 2024-04-20T10:59:10.869 INFO:tasks.workunit.client.0.smithi027.stdout:8/53: unlink f7 0 2024-04-20T10:59:10.869 INFO:tasks.workunit.client.0.smithi027.stdout:8/54: dread - f3 zero size 2024-04-20T10:59:10.871 INFO:tasks.workunit.client.0.smithi027.stdout:3/94: stat d2/dd/df/c15 0 2024-04-20T10:59:10.871 INFO:tasks.workunit.client.0.smithi027.stdout:3/95: truncate d2/f1e 1019405 0 2024-04-20T10:59:10.871 INFO:tasks.workunit.client.0.smithi027.stdout:3/96: write d2/f12 [604308,112806] 0 2024-04-20T10:59:10.872 INFO:tasks.workunit.client.0.smithi027.stdout:7/60: rmdir d4 39 2024-04-20T10:59:10.872 INFO:tasks.workunit.client.0.smithi027.stdout:7/61: chown d4/d5/d8 378374142 1 2024-04-20T10:59:10.873 INFO:tasks.workunit.client.0.smithi027.stdout:6/62: dwrite da/df/f1a [0,4194304] 0 2024-04-20T10:59:10.874 INFO:tasks.workunit.client.0.smithi027.stdout:3/97: creat d2/f21 x:0 0 0 2024-04-20T10:59:10.874 INFO:tasks.workunit.client.0.smithi027.stdout:3/98: write d2/dd/f20 [124671,55772] 0 2024-04-20T10:59:10.875 INFO:tasks.workunit.client.0.smithi027.stdout:1/75: getdents d3/d7 0 2024-04-20T10:59:10.875 INFO:tasks.workunit.client.0.smithi027.stdout:1/76: chown d3/d7/fa 27 1 2024-04-20T10:59:10.876 INFO:tasks.workunit.client.0.smithi027.stdout:9/66: dread d4/fc [0,4194304] 0 2024-04-20T10:59:10.881 INFO:tasks.workunit.client.0.smithi027.stdout:1/77: dread d3/d7/f12 [0,4194304] 0 2024-04-20T10:59:10.881 INFO:tasks.workunit.client.0.smithi027.stdout:9/67: getdents d4 0 2024-04-20T10:59:10.881 INFO:tasks.workunit.client.0.smithi027.stdout:9/68: chown d4/lf 295 1 2024-04-20T10:59:10.881 INFO:tasks.workunit.client.0.smithi027.stdout:9/69: chown d4 156705986 1 2024-04-20T10:59:10.883 INFO:tasks.workunit.client.0.smithi027.stdout:7/62: dread f1 [0,4194304] 0 2024-04-20T10:59:10.884 INFO:tasks.workunit.client.0.smithi027.stdout:3/99: write d2/dd/f1a [2104063,117142] 0 2024-04-20T10:59:10.885 INFO:tasks.workunit.client.0.smithi027.stdout:3/100: write d2/dd/f20 [222378,82165] 0 2024-04-20T10:59:10.888 INFO:tasks.workunit.client.0.smithi027.stdout:3/101: mkdir d2/dd/d22 0 2024-04-20T10:59:10.893 INFO:tasks.workunit.client.0.smithi027.stdout:3/102: dread d2/f12 [0,4194304] 0 2024-04-20T10:59:10.894 INFO:tasks.workunit.client.0.smithi027.stdout:3/103: mkdir d2/dd/df/d23 0 2024-04-20T10:59:10.894 INFO:tasks.workunit.client.0.smithi027.stdout:3/104: rename d2 to d2/dd/df/d23/d24 22 2024-04-20T10:59:10.906 INFO:tasks.workunit.client.0.smithi027.stdout:4/57: dwrite d0/f12 [0,4194304] 0 2024-04-20T10:59:10.907 INFO:tasks.workunit.client.0.smithi027.stdout:4/58: creat d0/d15/f16 x:0 0 0 2024-04-20T10:59:10.907 INFO:tasks.workunit.client.0.smithi027.stdout:4/59: dread - d0/d15/f16 zero size 2024-04-20T10:59:10.910 INFO:tasks.workunit.client.0.smithi027.stdout:5/49: dwrite f3 [0,4194304] 0 2024-04-20T10:59:10.910 INFO:tasks.workunit.client.0.smithi027.stdout:5/50: stat fb 0 2024-04-20T10:59:10.910 INFO:tasks.workunit.client.0.smithi027.stdout:2/41: dwrite f3 [0,4194304] 0 2024-04-20T10:59:10.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/63: dwrite f4 [0,4194304] 0 2024-04-20T10:59:10.917 INFO:tasks.workunit.client.0.smithi027.stdout:8/55: dwrite f2 [0,4194304] 0 2024-04-20T10:59:10.919 INFO:tasks.workunit.client.0.smithi027.stdout:9/70: dwrite d4/f7 [4194304,4194304] 0 2024-04-20T10:59:10.920 INFO:tasks.workunit.client.0.smithi027.stdout:7/63: dwrite d4/d5/f6 [0,4194304] 0 2024-04-20T10:59:10.920 INFO:tasks.workunit.client.0.smithi027.stdout:7/64: write f2 [15844,17718] 0 2024-04-20T10:59:10.921 INFO:tasks.workunit.client.0.smithi027.stdout:8/56: creat d9/ff x:0 0 0 2024-04-20T10:59:10.923 INFO:tasks.workunit.client.0.smithi027.stdout:2/42: dread d5/f8 [4194304,4194304] 0 2024-04-20T10:59:10.923 INFO:tasks.workunit.client.0.smithi027.stdout:2/43: chown d5 16128 1 2024-04-20T10:59:10.924 INFO:tasks.workunit.client.0.smithi027.stdout:8/57: dread d9/fb [0,4194304] 0 2024-04-20T10:59:10.924 INFO:tasks.workunit.client.0.smithi027.stdout:8/58: fsync f3 0 2024-04-20T10:59:10.925 INFO:tasks.workunit.client.0.smithi027.stdout:9/71: creat d4/f11 x:0 0 0 2024-04-20T10:59:10.925 INFO:tasks.workunit.client.0.smithi027.stdout:9/72: truncate d4/f11 184703 0 2024-04-20T10:59:10.925 INFO:tasks.workunit.client.0.smithi027.stdout:6/64: truncate da/df/f1a 1035441 0 2024-04-20T10:59:10.925 INFO:tasks.workunit.client.0.smithi027.stdout:6/65: readlink da/l18 0 2024-04-20T10:59:10.926 INFO:tasks.workunit.client.0.smithi027.stdout:3/105: truncate d2/dd/f1a 3763619 0 2024-04-20T10:59:10.928 INFO:tasks.workunit.client.0.smithi027.stdout:5/51: dwrite fc [0,4194304] 0 2024-04-20T10:59:10.929 INFO:tasks.workunit.client.0.smithi027.stdout:5/52: read f2 [4285748,37837] 0 2024-04-20T10:59:10.929 INFO:tasks.workunit.client.0.smithi027.stdout:5/53: write f9 [353715,36519] 0 2024-04-20T10:59:10.930 INFO:tasks.workunit.client.0.smithi027.stdout:7/65: creat d4/d5/d8/dd/ff x:0 0 0 2024-04-20T10:59:10.931 INFO:tasks.workunit.client.0.smithi027.stdout:8/59: symlink d9/de/l10 0 2024-04-20T10:59:10.932 INFO:tasks.workunit.client.0.smithi027.stdout:6/66: creat da/f1b x:0 0 0 2024-04-20T10:59:10.933 INFO:tasks.workunit.client.0.smithi027.stdout:4/60: rename d0/d15 to d0/d14/d17 0 2024-04-20T10:59:10.933 INFO:tasks.workunit.client.0.smithi027.stdout:4/61: chown d0/l10 2999 1 2024-04-20T10:59:10.933 INFO:tasks.workunit.client.0.smithi027.stdout:4/62: chown d0 284758 1 2024-04-20T10:59:10.933 INFO:tasks.workunit.client.0.smithi027.stdout:4/63: truncate d0/d14/d17/f16 11369 0 2024-04-20T10:59:10.933 INFO:tasks.workunit.client.0.smithi027.stdout:4/64: dread - d0/f9 zero size 2024-04-20T10:59:10.934 INFO:tasks.workunit.client.0.smithi027.stdout:3/106: creat d2/f25 x:0 0 0 2024-04-20T10:59:10.934 INFO:tasks.workunit.client.0.smithi027.stdout:3/107: fdatasync d2/f16 0 2024-04-20T10:59:10.936 INFO:tasks.workunit.client.0.smithi027.stdout:5/54: symlink l10 0 2024-04-20T10:59:10.937 INFO:tasks.workunit.client.0.smithi027.stdout:8/60: symlink d9/l11 0 2024-04-20T10:59:10.938 INFO:tasks.workunit.client.0.smithi027.stdout:4/65: mkdir d0/d14/d17/d18 0 2024-04-20T10:59:10.938 INFO:tasks.workunit.client.0.smithi027.stdout:4/66: read - d0/fd zero size 2024-04-20T10:59:10.939 INFO:tasks.workunit.client.0.smithi027.stdout:5/55: symlink l11 0 2024-04-20T10:59:10.939 INFO:tasks.workunit.client.0.smithi027.stdout:5/56: fdatasync fb 0 2024-04-20T10:59:10.939 INFO:tasks.workunit.client.0.smithi027.stdout:5/57: stat l10 0 2024-04-20T10:59:10.939 INFO:tasks.workunit.client.0.smithi027.stdout:8/61: write d9/fd [487027,77113] 0 2024-04-20T10:59:10.940 INFO:tasks.workunit.client.0.smithi027.stdout:8/62: rename d9 to d9/d12 22 2024-04-20T10:59:10.941 INFO:tasks.workunit.client.0.smithi027.stdout:4/67: dread d0/fc [0,4194304] 0 2024-04-20T10:59:10.942 INFO:tasks.workunit.client.0.smithi027.stdout:4/68: chown d0/c11 12 1 2024-04-20T10:59:10.945 INFO:tasks.workunit.client.0.smithi027.stdout:8/63: mknod d9/c13 0 2024-04-20T10:59:10.947 INFO:tasks.workunit.client.0.smithi027.stdout:8/64: rename d9/l11 to d9/de/l14 0 2024-04-20T10:59:10.949 INFO:tasks.workunit.client.0.smithi027.stdout:7/66: dwrite d4/d5/fa [0,4194304] 0 2024-04-20T10:59:10.949 INFO:tasks.workunit.client.0.smithi027.stdout:7/67: stat d4/d5/d8 0 2024-04-20T10:59:10.952 INFO:tasks.workunit.client.0.smithi027.stdout:8/65: unlink f5 0 2024-04-20T10:59:10.954 INFO:tasks.workunit.client.0.smithi027.stdout:7/68: mknod d4/d5/d8/c10 0 2024-04-20T10:59:10.962 INFO:tasks.workunit.client.0.smithi027.stdout:7/69: dread f2 [0,4194304] 0 2024-04-20T10:59:10.962 INFO:tasks.workunit.client.0.smithi027.stdout:7/70: stat d4/le 0 2024-04-20T10:59:10.962 INFO:tasks.workunit.client.0.smithi027.stdout:6/67: dwrite da/df/f11 [0,4194304] 0 2024-04-20T10:59:10.974 INFO:tasks.workunit.client.0.smithi027.stdout:3/108: dwrite d2/fc [0,4194304] 0 2024-04-20T10:59:10.979 INFO:tasks.workunit.client.0.smithi027.stdout:5/58: dwrite f9 [0,4194304] 0 2024-04-20T10:59:10.979 INFO:tasks.workunit.client.0.smithi027.stdout:4/69: dwrite d0/fb [0,4194304] 0 2024-04-20T10:59:10.979 INFO:tasks.workunit.client.0.smithi027.stdout:8/66: dwrite d9/fd [0,4194304] 0 2024-04-20T10:59:10.981 INFO:tasks.workunit.client.0.smithi027.stdout:5/59: symlink l12 0 2024-04-20T10:59:10.982 INFO:tasks.workunit.client.0.smithi027.stdout:8/67: creat d9/f15 x:0 0 0 2024-04-20T10:59:10.984 INFO:tasks.workunit.client.0.smithi027.stdout:8/68: mknod d9/de/c16 0 2024-04-20T10:59:10.984 INFO:tasks.workunit.client.0.smithi027.stdout:0/56: sync 2024-04-20T10:59:10.984 INFO:tasks.workunit.client.0.smithi027.stdout:8/69: truncate f2 2719682 0 2024-04-20T10:59:10.986 INFO:tasks.workunit.client.0.smithi027.stdout:0/57: rename d0/f2 to d0/fb 0 2024-04-20T10:59:10.986 INFO:tasks.workunit.client.0.smithi027.stdout:0/58: read d0/fb [921965,4392] 0 2024-04-20T10:59:10.986 INFO:tasks.workunit.client.0.smithi027.stdout:0/59: write d0/fb [3074851,106870] 0 2024-04-20T10:59:10.986 INFO:tasks.workunit.client.0.smithi027.stdout:8/70: symlink d9/l17 0 2024-04-20T10:59:10.987 INFO:tasks.workunit.client.0.smithi027.stdout:0/60: rmdir d0 39 2024-04-20T10:59:10.987 INFO:tasks.workunit.client.0.smithi027.stdout:0/61: truncate d0/fb 4556470 0 2024-04-20T10:59:10.987 INFO:tasks.workunit.client.0.smithi027.stdout:0/62: chown d0/c9 1951871 1 2024-04-20T10:59:10.987 INFO:tasks.workunit.client.0.smithi027.stdout:8/71: rmdir d9/de 39 2024-04-20T10:59:10.987 INFO:tasks.workunit.client.0.smithi027.stdout:0/63: getdents d0/d7 0 2024-04-20T10:59:10.987 INFO:tasks.workunit.client.0.smithi027.stdout:0/64: write d0/d7/fa [783806,21394] 0 2024-04-20T10:59:10.988 INFO:tasks.workunit.client.0.smithi027.stdout:8/72: mkdir d9/de/d18 0 2024-04-20T10:59:10.992 INFO:tasks.workunit.client.0.smithi027.stdout:3/109: dwrite d2/f16 [0,4194304] 0 2024-04-20T10:59:10.992 INFO:tasks.workunit.client.0.smithi027.stdout:3/110: fsync d2/f6 0 2024-04-20T10:59:11.007 INFO:tasks.workunit.client.0.smithi027.stdout:5/60: dwrite fc [4194304,4194304] 0 2024-04-20T10:59:11.008 INFO:tasks.workunit.client.0.smithi027.stdout:3/111: dwrite d2/f6 [0,4194304] 0 2024-04-20T10:59:11.008 INFO:tasks.workunit.client.0.smithi027.stdout:3/112: write d2/f21 [21383,113613] 0 2024-04-20T10:59:11.008 INFO:tasks.workunit.client.0.smithi027.stdout:3/113: fdatasync d2/dd/f19 0 2024-04-20T10:59:11.008 INFO:tasks.workunit.client.0.smithi027.stdout:3/114: truncate d2/dd/f20 793236 0 2024-04-20T10:59:11.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/65: unlink d0/fb 0 2024-04-20T10:59:11.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/66: chown d0/d7/fa 14 1 2024-04-20T10:59:11.015 INFO:tasks.workunit.client.0.smithi027.stdout:5/61: read f2 [4030275,14237] 0 2024-04-20T10:59:11.016 INFO:tasks.workunit.client.0.smithi027.stdout:3/115: creat d2/d18/f26 x:0 0 0 2024-04-20T10:59:11.017 INFO:tasks.workunit.client.0.smithi027.stdout:5/62: symlink l13 0 2024-04-20T10:59:11.018 INFO:tasks.workunit.client.0.smithi027.stdout:3/116: getdents d2 0 2024-04-20T10:59:11.018 INFO:tasks.workunit.client.0.smithi027.stdout:3/117: readlink d2/la 0 2024-04-20T10:59:11.018 INFO:tasks.workunit.client.0.smithi027.stdout:3/118: mknod d2/dd/d22/c27 0 2024-04-20T10:59:11.018 INFO:tasks.workunit.client.0.smithi027.stdout:3/119: write d2/f14 [4430673,45732] 0 2024-04-20T10:59:11.024 INFO:tasks.workunit.client.0.smithi027.stdout:5/63: dread fc [4194304,4194304] 0 2024-04-20T10:59:11.024 INFO:tasks.workunit.client.0.smithi027.stdout:5/64: write fd [304587,54410] 0 2024-04-20T10:59:11.024 INFO:tasks.workunit.client.0.smithi027.stdout:5/65: readlink l12 0 2024-04-20T10:59:11.024 INFO:tasks.workunit.client.0.smithi027.stdout:0/67: dwrite d0/d7/fa [0,4194304] 0 2024-04-20T10:59:11.025 INFO:tasks.workunit.client.0.smithi027.stdout:5/66: creat f14 x:0 0 0 2024-04-20T10:59:11.029 INFO:tasks.workunit.client.0.smithi027.stdout:0/68: creat d0/fc x:0 0 0 2024-04-20T10:59:11.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/120: getdents d2/d18 0 2024-04-20T10:59:11.033 INFO:tasks.workunit.client.0.smithi027.stdout:3/121: creat d2/dd/f28 x:0 0 0 2024-04-20T10:59:11.033 INFO:tasks.workunit.client.0.smithi027.stdout:3/122: readlink d2/la 0 2024-04-20T10:59:11.040 INFO:tasks.workunit.client.0.smithi027.stdout:5/67: dwrite f9 [0,4194304] 0 2024-04-20T10:59:11.044 INFO:tasks.workunit.client.0.smithi027.stdout:5/68: mknod c15 0 2024-04-20T10:59:11.046 INFO:tasks.workunit.client.0.smithi027.stdout:5/69: rename f14 to f16 0 2024-04-20T10:59:11.053 INFO:tasks.workunit.client.0.smithi027.stdout:3/123: dwrite d2/f16 [0,4194304] 0 2024-04-20T10:59:11.053 INFO:tasks.workunit.client.0.smithi027.stdout:5/70: creat f17 x:0 0 0 2024-04-20T10:59:11.053 INFO:tasks.workunit.client.0.smithi027.stdout:0/69: getdents d0 0 2024-04-20T10:59:11.054 INFO:tasks.workunit.client.0.smithi027.stdout:3/124: creat d2/d18/f29 x:0 0 0 2024-04-20T10:59:11.055 INFO:tasks.workunit.client.0.smithi027.stdout:5/71: link f9 f18 0 2024-04-20T10:59:11.056 INFO:tasks.workunit.client.0.smithi027.stdout:3/125: unlink d2/fc 0 2024-04-20T10:59:11.056 INFO:tasks.workunit.client.0.smithi027.stdout:3/126: write d2/f21 [796441,40690] 0 2024-04-20T10:59:11.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/70: dwrite d0/d7/fa [0,4194304] 0 2024-04-20T10:59:11.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/71: write d0/fc [953836,47664] 0 2024-04-20T10:59:11.143 INFO:tasks.workunit.client.0.smithi027.stdout:3/127: truncate d2/dd/f20 616928 0 2024-04-20T10:59:11.143 INFO:tasks.workunit.client.0.smithi027.stdout:3/128: truncate d2/f25 710775 0 2024-04-20T10:59:11.144 INFO:tasks.workunit.client.0.smithi027.stdout:3/129: chown d2/dd/c1c 101146 1 2024-04-20T10:59:11.144 INFO:tasks.workunit.client.0.smithi027.stdout:3/130: creat d2/dd/df/d23/f2a x:0 0 0 2024-04-20T10:59:11.151 INFO:tasks.workunit.client.0.smithi027.stdout:1/78: sync 2024-04-20T10:59:11.153 INFO:tasks.workunit.client.0.smithi027.stdout:1/79: rename d3/d7/fd to d3/d7/f1f 0 2024-04-20T10:59:11.153 INFO:tasks.workunit.client.0.smithi027.stdout:1/80: mkdir d3/d20 0 2024-04-20T10:59:11.154 INFO:tasks.workunit.client.0.smithi027.stdout:1/81: chown d3/d7 874487817 1 2024-04-20T10:59:11.154 INFO:tasks.workunit.client.0.smithi027.stdout:1/82: symlink d3/d7/l21 0 2024-04-20T10:59:11.155 INFO:tasks.workunit.client.0.smithi027.stdout:1/83: unlink d3/d7/c16 0 2024-04-20T10:59:11.156 INFO:tasks.workunit.client.0.smithi027.stdout:1/84: fsync d3/d7/ff 0 2024-04-20T10:59:11.156 INFO:tasks.workunit.client.0.smithi027.stdout:1/85: write d3/d7/ff [1008617,52744] 0 2024-04-20T10:59:11.166 INFO:tasks.workunit.client.0.smithi027.stdout:3/131: dwrite d2/dd/f28 [0,4194304] 0 2024-04-20T10:59:11.168 INFO:tasks.workunit.client.0.smithi027.stdout:3/132: symlink d2/dd/df/l2b 0 2024-04-20T10:59:11.168 INFO:tasks.workunit.client.0.smithi027.stdout:3/133: write d2/f5 [1450015,70033] 0 2024-04-20T10:59:11.168 INFO:tasks.workunit.client.0.smithi027.stdout:3/134: readlink d2/dd/df/l2b 0 2024-04-20T10:59:11.168 INFO:tasks.workunit.client.0.smithi027.stdout:3/135: mkdir d2/d18/d2c 0 2024-04-20T10:59:11.169 INFO:tasks.workunit.client.0.smithi027.stdout:3/136: mknod d2/c2d 0 2024-04-20T10:59:11.169 INFO:tasks.workunit.client.0.smithi027.stdout:3/137: dread - d2/dd/f19 zero size 2024-04-20T10:59:11.169 INFO:tasks.workunit.client.0.smithi027.stdout:3/138: fdatasync d2/f17 0 2024-04-20T10:59:11.169 INFO:tasks.workunit.client.0.smithi027.stdout:3/139: readlink d2/la 0 2024-04-20T10:59:11.177 INFO:tasks.workunit.client.0.smithi027.stdout:1/86: dwrite d3/f17 [0,4194304] 0 2024-04-20T10:59:11.179 INFO:tasks.workunit.client.0.smithi027.stdout:1/87: link d3/c18 d3/c22 0 2024-04-20T10:59:11.179 INFO:tasks.workunit.client.0.smithi027.stdout:1/88: dread - d3/f1c zero size 2024-04-20T10:59:11.181 INFO:tasks.workunit.client.0.smithi027.stdout:7/71: sync 2024-04-20T10:59:11.181 INFO:tasks.workunit.client.0.smithi027.stdout:7/72: stat d4/d5/c9 0 2024-04-20T10:59:11.181 INFO:tasks.workunit.client.0.smithi027.stdout:9/73: sync 2024-04-20T10:59:11.181 INFO:tasks.workunit.client.0.smithi027.stdout:2/44: sync 2024-04-20T10:59:11.181 INFO:tasks.workunit.client.0.smithi027.stdout:2/45: fdatasync f3 0 2024-04-20T10:59:11.182 INFO:tasks.workunit.client.0.smithi027.stdout:7/73: link d4/d5/c9 d4/c11 0 2024-04-20T10:59:11.183 INFO:tasks.workunit.client.0.smithi027.stdout:9/74: rename d4/cb to d4/c12 0 2024-04-20T10:59:11.184 INFO:tasks.workunit.client.0.smithi027.stdout:7/74: rename d4/d5/c7 to d4/d5/d8/dd/c12 0 2024-04-20T10:59:11.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/46: truncate f1 3115461 0 2024-04-20T10:59:11.185 INFO:tasks.workunit.client.0.smithi027.stdout:7/75: link d4/cb d4/c13 0 2024-04-20T10:59:11.186 INFO:tasks.workunit.client.0.smithi027.stdout:1/89: dread d3/f17 [0,4194304] 0 2024-04-20T10:59:11.186 INFO:tasks.workunit.client.0.smithi027.stdout:1/90: stat d3/f1b 0 2024-04-20T10:59:11.186 INFO:tasks.workunit.client.0.smithi027.stdout:9/75: write d4/fc [429896,111012] 0 2024-04-20T10:59:11.186 INFO:tasks.workunit.client.0.smithi027.stdout:3/140: dwrite d2/f21 [0,4194304] 0 2024-04-20T10:59:11.187 INFO:tasks.workunit.client.0.smithi027.stdout:1/91: truncate d3/d7/f12 3637478 0 2024-04-20T10:59:11.188 INFO:tasks.workunit.client.0.smithi027.stdout:9/76: symlink d4/l13 0 2024-04-20T10:59:11.188 INFO:tasks.workunit.client.0.smithi027.stdout:3/141: fsync d2/f1d 0 2024-04-20T10:59:11.189 INFO:tasks.workunit.client.0.smithi027.stdout:3/142: read d2/f1e [1013256,79982] 0 2024-04-20T10:59:11.189 INFO:tasks.workunit.client.0.smithi027.stdout:3/143: write d2/f1e [482220,118178] 0 2024-04-20T10:59:11.195 INFO:tasks.workunit.client.0.smithi027.stdout:2/47: dread f3 [0,4194304] 0 2024-04-20T10:59:11.195 INFO:tasks.workunit.client.0.smithi027.stdout:1/92: creat d3/d7/f23 x:0 0 0 2024-04-20T10:59:11.195 INFO:tasks.workunit.client.0.smithi027.stdout:1/93: chown d3/c22 62562 1 2024-04-20T10:59:11.195 INFO:tasks.workunit.client.0.smithi027.stdout:1/94: truncate d3/d7/fc 494633 0 2024-04-20T10:59:11.197 INFO:tasks.workunit.client.0.smithi027.stdout:3/144: read d2/f14 [829,123631] 0 2024-04-20T10:59:11.197 INFO:tasks.workunit.client.0.smithi027.stdout:3/145: read d2/f14 [910116,9150] 0 2024-04-20T10:59:11.197 INFO:tasks.workunit.client.0.smithi027.stdout:9/77: mkdir d4/d14 0 2024-04-20T10:59:11.197 INFO:tasks.workunit.client.0.smithi027.stdout:9/78: write d4/f7 [845519,63372] 0 2024-04-20T10:59:11.197 INFO:tasks.workunit.client.0.smithi027.stdout:9/79: readlink d4/l13 0 2024-04-20T10:59:11.197 INFO:tasks.workunit.client.0.smithi027.stdout:2/48: creat d5/dc/fe x:0 0 0 2024-04-20T10:59:11.198 INFO:tasks.workunit.client.0.smithi027.stdout:1/95: mknod d3/c24 0 2024-04-20T10:59:11.199 INFO:tasks.workunit.client.0.smithi027.stdout:3/146: mkdir d2/d18/d2e 0 2024-04-20T10:59:11.201 INFO:tasks.workunit.client.0.smithi027.stdout:1/96: unlink d3/c18 0 2024-04-20T10:59:11.203 INFO:tasks.workunit.client.0.smithi027.stdout:3/147: creat d2/dd/d22/f2f x:0 0 0 2024-04-20T10:59:11.204 INFO:tasks.workunit.client.0.smithi027.stdout:7/76: dwrite f1 [0,4194304] 0 2024-04-20T10:59:11.204 INFO:tasks.workunit.client.0.smithi027.stdout:1/97: mknod d3/d7/c25 0 2024-04-20T10:59:11.205 INFO:tasks.workunit.client.0.smithi027.stdout:3/148: creat d2/d18/d2c/f30 x:0 0 0 2024-04-20T10:59:11.206 INFO:tasks.workunit.client.0.smithi027.stdout:7/77: rmdir d4/d5/d8 39 2024-04-20T10:59:11.206 INFO:tasks.workunit.client.0.smithi027.stdout:7/78: fdatasync f2 0 2024-04-20T10:59:11.208 INFO:tasks.workunit.client.0.smithi027.stdout:1/98: creat d3/d7/f26 x:0 0 0 2024-04-20T10:59:11.209 INFO:tasks.workunit.client.0.smithi027.stdout:6/68: sync 2024-04-20T10:59:11.209 INFO:tasks.workunit.client.0.smithi027.stdout:4/70: sync 2024-04-20T10:59:11.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/73: sync 2024-04-20T10:59:11.209 INFO:tasks.workunit.client.0.smithi027.stdout:0/72: sync 2024-04-20T10:59:11.209 INFO:tasks.workunit.client.0.smithi027.stdout:5/72: sync 2024-04-20T10:59:11.213 INFO:tasks.workunit.client.0.smithi027.stdout:9/80: dread d4/f7 [0,4194304] 0 2024-04-20T10:59:11.214 INFO:tasks.workunit.client.0.smithi027.stdout:6/69: unlink f7 0 2024-04-20T10:59:11.214 INFO:tasks.workunit.client.0.smithi027.stdout:6/70: dread - da/dd/f13 zero size 2024-04-20T10:59:11.214 INFO:tasks.workunit.client.0.smithi027.stdout:4/71: symlink d0/d14/d17/d18/l19 0 2024-04-20T10:59:11.214 INFO:tasks.workunit.client.0.smithi027.stdout:4/72: write d0/f13 [619880,25776] 0 2024-04-20T10:59:11.219 INFO:tasks.workunit.client.0.smithi027.stdout:3/149: dread d2/f21 [0,4194304] 0 2024-04-20T10:59:11.220 INFO:tasks.workunit.client.0.smithi027.stdout:4/73: mkdir d0/d14/d17/d1a 0 2024-04-20T10:59:11.221 INFO:tasks.workunit.client.0.smithi027.stdout:9/81: rename d4/l13 to d4/d14/l15 0 2024-04-20T10:59:11.221 INFO:tasks.workunit.client.0.smithi027.stdout:9/82: write d4/fc [3351547,24221] 0 2024-04-20T10:59:11.222 INFO:tasks.workunit.client.0.smithi027.stdout:0/73: dread d0/d7/fa [0,4194304] 0 2024-04-20T10:59:11.225 INFO:tasks.workunit.client.0.smithi027.stdout:0/74: read d0/fc [892171,90244] 0 2024-04-20T10:59:11.225 INFO:tasks.workunit.client.0.smithi027.stdout:0/75: fsync d0/fc 0 2024-04-20T10:59:11.226 INFO:tasks.workunit.client.0.smithi027.stdout:0/76: write d0/fc [1948397,127564] 0 2024-04-20T10:59:11.226 INFO:tasks.workunit.client.0.smithi027.stdout:0/77: rename d0 to d0/d7/dd 22 2024-04-20T10:59:11.226 INFO:tasks.workunit.client.0.smithi027.stdout:4/74: truncate d0/fb 488674 0 2024-04-20T10:59:11.227 INFO:tasks.workunit.client.0.smithi027.stdout:6/71: write f3 [2498372,84825] 0 2024-04-20T10:59:11.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/83: symlink d4/d14/l16 0 2024-04-20T10:59:11.228 INFO:tasks.workunit.client.0.smithi027.stdout:0/78: rename d0/fc to d0/d7/fe 0 2024-04-20T10:59:11.229 INFO:tasks.workunit.client.0.smithi027.stdout:0/79: stat d0/c9 0 2024-04-20T10:59:11.229 INFO:tasks.workunit.client.0.smithi027.stdout:4/75: mkdir d0/d14/d17/d1b 0 2024-04-20T10:59:11.230 INFO:tasks.workunit.client.0.smithi027.stdout:0/80: creat d0/ff x:0 0 0 2024-04-20T10:59:11.230 INFO:tasks.workunit.client.0.smithi027.stdout:0/81: chown d0/c9 588 1 2024-04-20T10:59:11.230 INFO:tasks.workunit.client.0.smithi027.stdout:0/82: fdatasync d0/ff 0 2024-04-20T10:59:11.231 INFO:tasks.workunit.client.0.smithi027.stdout:4/76: creat d0/d14/d17/f1c x:0 0 0 2024-04-20T10:59:11.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/83: unlink d0/c9 0 2024-04-20T10:59:11.233 INFO:tasks.workunit.client.0.smithi027.stdout:4/77: creat d0/d14/d17/f1d x:0 0 0 2024-04-20T10:59:11.235 INFO:tasks.workunit.client.0.smithi027.stdout:4/78: creat d0/d14/d17/f1e x:0 0 0 2024-04-20T10:59:11.236 INFO:tasks.workunit.client.0.smithi027.stdout:4/79: fdatasync d0/fd 0 2024-04-20T10:59:11.236 INFO:tasks.workunit.client.0.smithi027.stdout:4/80: creat d0/d14/d17/f1f x:0 0 0 2024-04-20T10:59:11.236 INFO:tasks.workunit.client.0.smithi027.stdout:4/81: chown d0/d14/d17 591148 1 2024-04-20T10:59:11.238 INFO:tasks.workunit.client.0.smithi027.stdout:8/74: dwrite d9/fd [4194304,4194304] 0 2024-04-20T10:59:11.239 INFO:tasks.workunit.client.0.smithi027.stdout:7/79: dwrite d4/d5/f6 [0,4194304] 0 2024-04-20T10:59:11.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/75: chown d9/fb 128646 1 2024-04-20T10:59:11.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/76: chown d9/l17 213266 1 2024-04-20T10:59:11.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/77: read - f3 zero size 2024-04-20T10:59:11.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/78: readlink d9/lc 0 2024-04-20T10:59:11.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/79: truncate d9/fb 1123294 0 2024-04-20T10:59:11.239 INFO:tasks.workunit.client.0.smithi027.stdout:5/73: dwrite fd [0,4194304] 0 2024-04-20T10:59:11.240 INFO:tasks.workunit.client.0.smithi027.stdout:5/74: fsync f17 0 2024-04-20T10:59:11.240 INFO:tasks.workunit.client.0.smithi027.stdout:5/75: read f0 [913058,46072] 0 2024-04-20T10:59:11.242 INFO:tasks.workunit.client.0.smithi027.stdout:7/80: truncate d4/d5/fa 1092613 0 2024-04-20T10:59:11.243 INFO:tasks.workunit.client.0.smithi027.stdout:8/80: unlink d9/de/c16 0 2024-04-20T10:59:11.244 INFO:tasks.workunit.client.0.smithi027.stdout:8/81: dread - d9/f15 zero size 2024-04-20T10:59:11.244 INFO:tasks.workunit.client.0.smithi027.stdout:8/82: fdatasync d9/f15 0 2024-04-20T10:59:11.250 INFO:tasks.workunit.client.0.smithi027.stdout:8/83: stat d9/de/l14 0 2024-04-20T10:59:11.250 INFO:tasks.workunit.client.0.smithi027.stdout:3/150: dwrite d2/d18/d2c/f30 [0,4194304] 0 2024-04-20T10:59:11.252 INFO:tasks.workunit.client.0.smithi027.stdout:8/84: mknod d9/c19 0 2024-04-20T10:59:11.252 INFO:tasks.workunit.client.0.smithi027.stdout:8/85: fdatasync d9/fb 0 2024-04-20T10:59:11.256 INFO:tasks.workunit.client.0.smithi027.stdout:8/86: creat d9/de/f1a x:0 0 0 2024-04-20T10:59:11.256 INFO:tasks.workunit.client.0.smithi027.stdout:8/87: dread - d9/de/f1a zero size 2024-04-20T10:59:11.257 INFO:tasks.workunit.client.0.smithi027.stdout:8/88: creat d9/de/f1b x:0 0 0 2024-04-20T10:59:11.257 INFO:tasks.workunit.client.0.smithi027.stdout:8/89: write d9/ff [946309,34341] 0 2024-04-20T10:59:11.257 INFO:tasks.workunit.client.0.smithi027.stdout:8/90: readlink d9/de/l10 0 2024-04-20T10:59:11.258 INFO:tasks.workunit.client.0.smithi027.stdout:8/91: write d9/fb [1762466,40299] 0 2024-04-20T10:59:11.258 INFO:tasks.workunit.client.0.smithi027.stdout:8/92: write f3 [473667,96649] 0 2024-04-20T10:59:11.278 INFO:tasks.workunit.client.0.smithi027.stdout:1/99: dwrite d3/d7/f1a [0,4194304] 0 2024-04-20T10:59:11.278 INFO:tasks.workunit.client.0.smithi027.stdout:4/82: dwrite d0/fd [0,4194304] 0 2024-04-20T10:59:11.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/84: dwrite d4/f7 [8388608,4194304] 0 2024-04-20T10:59:11.279 INFO:tasks.workunit.client.0.smithi027.stdout:5/76: dwrite f9 [0,4194304] 0 2024-04-20T10:59:11.279 INFO:tasks.workunit.client.0.smithi027.stdout:5/77: fdatasync f9 0 2024-04-20T10:59:11.280 INFO:tasks.workunit.client.0.smithi027.stdout:1/100: mknod d3/d20/c27 0 2024-04-20T10:59:11.280 INFO:tasks.workunit.client.0.smithi027.stdout:1/101: readlink d3/l13 0 2024-04-20T10:59:11.281 INFO:tasks.workunit.client.0.smithi027.stdout:6/72: dwrite da/dd/f13 [0,4194304] 0 2024-04-20T10:59:11.283 INFO:tasks.workunit.client.0.smithi027.stdout:4/83: getdents d0/d14/d17/d18 0 2024-04-20T10:59:11.284 INFO:tasks.workunit.client.0.smithi027.stdout:3/151: dwrite d2/f21 [0,4194304] 0 2024-04-20T10:59:11.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/85: mknod d4/d14/c17 0 2024-04-20T10:59:11.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/86: truncate d4/fc 4306330 0 2024-04-20T10:59:11.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/87: chown d4/l10 6 1 2024-04-20T10:59:11.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/88: write d4/fc [4811933,85358] 0 2024-04-20T10:59:11.288 INFO:tasks.workunit.client.0.smithi027.stdout:5/78: mkdir d19 0 2024-04-20T10:59:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/73: dread da/dd/fe [0,4194304] 0 2024-04-20T10:59:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/74: chown da/df 16 1 2024-04-20T10:59:11.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/75: read - da/df/f17 zero size 2024-04-20T10:59:11.294 INFO:tasks.workunit.client.0.smithi027.stdout:6/76: dread da/fb [0,4194304] 0 2024-04-20T10:59:11.296 INFO:tasks.workunit.client.0.smithi027.stdout:4/84: creat d0/d14/d17/d18/f20 x:0 0 0 2024-04-20T10:59:11.298 INFO:tasks.workunit.client.0.smithi027.stdout:3/152: symlink d2/d18/l31 0 2024-04-20T10:59:11.298 INFO:tasks.workunit.client.0.smithi027.stdout:3/153: chown d2/f14 1898 1 2024-04-20T10:59:11.300 INFO:tasks.workunit.client.0.smithi027.stdout:5/79: mknod d19/c1a 0 2024-04-20T10:59:11.301 INFO:tasks.workunit.client.0.smithi027.stdout:6/77: mknod da/df/d19/c1c 0 2024-04-20T10:59:11.301 INFO:tasks.workunit.client.0.smithi027.stdout:6/78: write f3 [1867695,79705] 0 2024-04-20T10:59:11.302 INFO:tasks.workunit.client.0.smithi027.stdout:0/84: dwrite d0/d7/fe [0,4194304] 0 2024-04-20T10:59:11.302 INFO:tasks.workunit.client.0.smithi027.stdout:4/85: link d0/d14/d17/f1d d0/d14/d17/d1b/f21 0 2024-04-20T10:59:11.303 INFO:tasks.workunit.client.0.smithi027.stdout:8/93: truncate d9/fd 4199127 0 2024-04-20T10:59:11.303 INFO:tasks.workunit.client.0.smithi027.stdout:8/94: dread - d9/de/f1b zero size 2024-04-20T10:59:11.303 INFO:tasks.workunit.client.0.smithi027.stdout:3/154: rmdir d2/d18/d2c 39 2024-04-20T10:59:11.303 INFO:tasks.workunit.client.0.smithi027.stdout:3/155: fdatasync d2/f25 0 2024-04-20T10:59:11.305 INFO:tasks.workunit.client.0.smithi027.stdout:5/80: mkdir d19/d1b 0 2024-04-20T10:59:11.308 INFO:tasks.workunit.client.0.smithi027.stdout:3/156: dread d2/fe [0,4194304] 0 2024-04-20T10:59:11.308 INFO:tasks.workunit.client.0.smithi027.stdout:3/157: chown d2/fe 619883 1 2024-04-20T10:59:11.311 INFO:tasks.workunit.client.0.smithi027.stdout:1/102: dwrite d3/d7/f8 [0,4194304] 0 2024-04-20T10:59:11.312 INFO:tasks.workunit.client.0.smithi027.stdout:3/158: dread d2/f5 [0,4194304] 0 2024-04-20T10:59:11.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/81: dread d4/d5/fa [0,4194304] 0 2024-04-20T10:59:11.314 INFO:tasks.workunit.client.0.smithi027.stdout:9/89: getdents d4/d14 0 2024-04-20T10:59:11.317 INFO:tasks.workunit.client.0.smithi027.stdout:9/90: rename d4/d14/l15 to d4/l18 0 2024-04-20T10:59:11.317 INFO:tasks.workunit.client.0.smithi027.stdout:0/85: read d0/d7/fa [2574941,74255] 0 2024-04-20T10:59:11.319 INFO:tasks.workunit.client.0.smithi027.stdout:9/91: creat d4/d14/f19 x:0 0 0 2024-04-20T10:59:11.320 INFO:tasks.workunit.client.0.smithi027.stdout:9/92: chown d4/d14/l16 8 1 2024-04-20T10:59:11.320 INFO:tasks.workunit.client.0.smithi027.stdout:9/93: chown l3 85 1 2024-04-20T10:59:11.320 INFO:tasks.workunit.client.0.smithi027.stdout:9/94: chown d4/l18 1048289798 1 2024-04-20T10:59:11.321 INFO:tasks.workunit.client.0.smithi027.stdout:9/95: creat d4/f1a x:0 0 0 2024-04-20T10:59:11.321 INFO:tasks.workunit.client.0.smithi027.stdout:9/96: dread - d4/f1a zero size 2024-04-20T10:59:11.321 INFO:tasks.workunit.client.0.smithi027.stdout:9/97: read - d4/d14/f19 zero size 2024-04-20T10:59:11.322 INFO:tasks.workunit.client.0.smithi027.stdout:9/98: chown d4/lf 13 1 2024-04-20T10:59:11.322 INFO:tasks.workunit.client.0.smithi027.stdout:8/95: dread d9/ff [0,4194304] 0 2024-04-20T10:59:11.322 INFO:tasks.workunit.client.0.smithi027.stdout:6/79: dwrite da/df/f11 [0,4194304] 0 2024-04-20T10:59:11.323 INFO:tasks.workunit.client.0.smithi027.stdout:4/86: dread d0/fd [0,4194304] 0 2024-04-20T10:59:11.323 INFO:tasks.workunit.client.0.smithi027.stdout:9/99: mkdir d4/d1b 0 2024-04-20T10:59:11.323 INFO:tasks.workunit.client.0.smithi027.stdout:9/100: write d4/f1a [640491,50700] 0 2024-04-20T10:59:11.323 INFO:tasks.workunit.client.0.smithi027.stdout:8/96: write f3 [83582,109396] 0 2024-04-20T10:59:11.323 INFO:tasks.workunit.client.0.smithi027.stdout:8/97: fsync f2 0 2024-04-20T10:59:11.323 INFO:tasks.workunit.client.0.smithi027.stdout:8/98: chown l8 114553 1 2024-04-20T10:59:11.325 INFO:tasks.workunit.client.0.smithi027.stdout:6/80: rename da/df/d19/c1c to da/c1d 0 2024-04-20T10:59:11.326 INFO:tasks.workunit.client.0.smithi027.stdout:4/87: mknod d0/d14/d17/d1a/c22 0 2024-04-20T10:59:11.328 INFO:tasks.workunit.client.0.smithi027.stdout:9/101: link d4/f7 d4/f1c 0 2024-04-20T10:59:11.328 INFO:tasks.workunit.client.0.smithi027.stdout:9/102: fdatasync d4/fc 0 2024-04-20T10:59:11.328 INFO:tasks.workunit.client.0.smithi027.stdout:3/159: write d2/f21 [1858141,31600] 0 2024-04-20T10:59:11.328 INFO:tasks.workunit.client.0.smithi027.stdout:3/160: dread - d2/f1f zero size 2024-04-20T10:59:11.328 INFO:tasks.workunit.client.0.smithi027.stdout:3/161: stat d2 0 2024-04-20T10:59:11.328 INFO:tasks.workunit.client.0.smithi027.stdout:3/162: readlink d2/la 0 2024-04-20T10:59:11.329 INFO:tasks.workunit.client.0.smithi027.stdout:8/99: mknod d9/de/c1c 0 2024-04-20T10:59:11.329 INFO:tasks.workunit.client.0.smithi027.stdout:8/100: chown f2 642 1 2024-04-20T10:59:11.329 INFO:tasks.workunit.client.0.smithi027.stdout:8/101: fsync d9/de/f1a 0 2024-04-20T10:59:11.330 INFO:tasks.workunit.client.0.smithi027.stdout:6/81: unlink da/cc 0 2024-04-20T10:59:11.330 INFO:tasks.workunit.client.0.smithi027.stdout:6/82: chown l6 14719551 1 2024-04-20T10:59:11.331 INFO:tasks.workunit.client.0.smithi027.stdout:4/88: creat d0/f23 x:0 0 0 2024-04-20T10:59:11.331 INFO:tasks.workunit.client.0.smithi027.stdout:4/89: read - d0/d14/d17/f1e zero size 2024-04-20T10:59:11.332 INFO:tasks.workunit.client.0.smithi027.stdout:6/83: rename da/df/l12 to da/dd/l1e 0 2024-04-20T10:59:11.334 INFO:tasks.workunit.client.0.smithi027.stdout:3/163: dread d2/f5 [0,4194304] 0 2024-04-20T10:59:11.334 INFO:tasks.workunit.client.0.smithi027.stdout:3/164: write d2/dd/df/d23/f2a [165726,123279] 0 2024-04-20T10:59:11.335 INFO:tasks.workunit.client.0.smithi027.stdout:3/165: link d2/dd/d22/c27 d2/dd/d22/c32 0 2024-04-20T10:59:11.336 INFO:tasks.workunit.client.0.smithi027.stdout:3/166: mkdir d2/dd/df/d33 0 2024-04-20T10:59:11.338 INFO:tasks.workunit.client.0.smithi027.stdout:3/167: symlink d2/d18/l34 0 2024-04-20T10:59:11.340 INFO:tasks.workunit.client.0.smithi027.stdout:7/82: dwrite f2 [0,4194304] 0 2024-04-20T10:59:11.340 INFO:tasks.workunit.client.0.smithi027.stdout:7/83: write d4/d5/fa [1782285,44500] 0 2024-04-20T10:59:11.340 INFO:tasks.workunit.client.0.smithi027.stdout:7/84: write d4/fc [449651,70942] 0 2024-04-20T10:59:11.340 INFO:tasks.workunit.client.0.smithi027.stdout:7/85: fdatasync d4/d5/f6 0 2024-04-20T10:59:11.341 INFO:tasks.workunit.client.0.smithi027.stdout:9/103: write d4/f7 [9749403,44434] 0 2024-04-20T10:59:11.342 INFO:tasks.workunit.client.0.smithi027.stdout:1/103: dwrite d3/d7/ff [0,4194304] 0 2024-04-20T10:59:11.345 INFO:tasks.workunit.client.0.smithi027.stdout:1/104: mkdir d3/d7/d28 0 2024-04-20T10:59:11.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/86: write d4/d5/fa [127358,60670] 0 2024-04-20T10:59:11.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/87: chown d4/d5 2978 1 2024-04-20T10:59:11.345 INFO:tasks.workunit.client.0.smithi027.stdout:1/105: write d3/f1c [634510,20372] 0 2024-04-20T10:59:11.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/86: dwrite d0/d7/fa [0,4194304] 0 2024-04-20T10:59:11.350 INFO:tasks.workunit.client.0.smithi027.stdout:2/49: sync 2024-04-20T10:59:11.350 INFO:tasks.workunit.client.0.smithi027.stdout:2/50: rename d5 to d5/df 22 2024-04-20T10:59:11.350 INFO:tasks.workunit.client.0.smithi027.stdout:2/51: chown d5/c6 756502526 1 2024-04-20T10:59:11.350 INFO:tasks.workunit.client.0.smithi027.stdout:1/106: mknod d3/d20/c29 0 2024-04-20T10:59:11.351 INFO:tasks.workunit.client.0.smithi027.stdout:0/87: getdents d0/d7 0 2024-04-20T10:59:11.352 INFO:tasks.workunit.client.0.smithi027.stdout:2/52: symlink d5/dc/l10 0 2024-04-20T10:59:11.352 INFO:tasks.workunit.client.0.smithi027.stdout:1/107: mknod d3/d7/c2a 0 2024-04-20T10:59:11.353 INFO:tasks.workunit.client.0.smithi027.stdout:0/88: getdents d0 0 2024-04-20T10:59:11.353 INFO:tasks.workunit.client.0.smithi027.stdout:2/53: symlink d5/l11 0 2024-04-20T10:59:11.354 INFO:tasks.workunit.client.0.smithi027.stdout:1/108: mknod d3/d7/c2b 0 2024-04-20T10:59:11.355 INFO:tasks.workunit.client.0.smithi027.stdout:0/89: symlink d0/d7/l10 0 2024-04-20T10:59:11.355 INFO:tasks.workunit.client.0.smithi027.stdout:0/90: chown d0 10201380 1 2024-04-20T10:59:11.355 INFO:tasks.workunit.client.0.smithi027.stdout:1/109: mknod d3/c2c 0 2024-04-20T10:59:11.356 INFO:tasks.workunit.client.0.smithi027.stdout:1/110: mkdir d3/d7/d2d 0 2024-04-20T10:59:11.361 INFO:tasks.workunit.client.0.smithi027.stdout:4/90: dwrite d0/fc [0,4194304] 0 2024-04-20T10:59:11.362 INFO:tasks.workunit.client.0.smithi027.stdout:4/91: fsync d0/f12 0 2024-04-20T10:59:11.362 INFO:tasks.workunit.client.0.smithi027.stdout:7/88: read f2 [238903,120873] 0 2024-04-20T10:59:11.364 INFO:tasks.workunit.client.0.smithi027.stdout:4/92: rename d0/la to d0/d14/d17/d1a/l24 0 2024-04-20T10:59:11.365 INFO:tasks.workunit.client.0.smithi027.stdout:4/93: symlink d0/d14/d17/l25 0 2024-04-20T10:59:11.370 INFO:tasks.workunit.client.0.smithi027.stdout:6/84: dwrite da/df/f11 [0,4194304] 0 2024-04-20T10:59:11.370 INFO:tasks.workunit.client.0.smithi027.stdout:6/85: mkdir da/df/d1f 0 2024-04-20T10:59:11.370 INFO:tasks.workunit.client.0.smithi027.stdout:6/86: symlink da/l20 0 2024-04-20T10:59:11.370 INFO:tasks.workunit.client.0.smithi027.stdout:8/102: dwrite f3 [0,4194304] 0 2024-04-20T10:59:11.371 INFO:tasks.workunit.client.0.smithi027.stdout:3/168: dwrite d2/f12 [0,4194304] 0 2024-04-20T10:59:11.371 INFO:tasks.workunit.client.0.smithi027.stdout:2/54: dwrite f1 [0,4194304] 0 2024-04-20T10:59:11.371 INFO:tasks.workunit.client.0.smithi027.stdout:2/55: truncate d5/dc/fe 702046 0 2024-04-20T10:59:11.371 INFO:tasks.workunit.client.0.smithi027.stdout:2/56: write d5/dc/fe [1603809,22672] 0 2024-04-20T10:59:11.371 INFO:tasks.workunit.client.0.smithi027.stdout:4/94: dread d0/fb [0,4194304] 0 2024-04-20T10:59:11.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/103: creat d9/de/f1d x:0 0 0 2024-04-20T10:59:11.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/104: chown d9/lc 48214 1 2024-04-20T10:59:11.375 INFO:tasks.workunit.client.0.smithi027.stdout:3/169: creat d2/d18/d2e/f35 x:0 0 0 2024-04-20T10:59:11.376 INFO:tasks.workunit.client.0.smithi027.stdout:6/87: dread f4 [0,4194304] 0 2024-04-20T10:59:11.379 INFO:tasks.workunit.client.0.smithi027.stdout:4/95: rmdir d0/d14/d17 39 2024-04-20T10:59:11.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/89: dwrite f1 [0,4194304] 0 2024-04-20T10:59:11.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/90: chown d4/cb 602 1 2024-04-20T10:59:11.380 INFO:tasks.workunit.client.0.smithi027.stdout:8/105: mkdir d9/de/d1e 0 2024-04-20T10:59:11.380 INFO:tasks.workunit.client.0.smithi027.stdout:8/106: write d9/de/f1d [966532,57347] 0 2024-04-20T10:59:11.380 INFO:tasks.workunit.client.0.smithi027.stdout:8/107: chown f3 14814207 1 2024-04-20T10:59:11.382 INFO:tasks.workunit.client.0.smithi027.stdout:3/170: link d2/c2d d2/d18/c36 0 2024-04-20T10:59:11.383 INFO:tasks.workunit.client.0.smithi027.stdout:3/171: symlink d2/d18/d2e/l37 0 2024-04-20T10:59:11.389 INFO:tasks.workunit.client.0.smithi027.stdout:1/111: truncate d3/d7/f8 292588 0 2024-04-20T10:59:11.392 INFO:tasks.workunit.client.0.smithi027.stdout:3/172: write d2/dd/f10 [3797785,7778] 0 2024-04-20T10:59:11.392 INFO:tasks.workunit.client.0.smithi027.stdout:1/112: creat d3/d7/d28/f2e x:0 0 0 2024-04-20T10:59:11.392 INFO:tasks.workunit.client.0.smithi027.stdout:4/96: write d0/f13 [4057765,65469] 0 2024-04-20T10:59:11.393 INFO:tasks.workunit.client.0.smithi027.stdout:0/91: truncate d0/d7/fa 1400210 0 2024-04-20T10:59:11.395 INFO:tasks.workunit.client.0.smithi027.stdout:3/173: rename d2/dd/d22/c27 to d2/dd/df/d33/c38 0 2024-04-20T10:59:11.395 INFO:tasks.workunit.client.0.smithi027.stdout:3/174: chown d2/dd/df/c1b 14 1 2024-04-20T10:59:11.398 INFO:tasks.workunit.client.0.smithi027.stdout:4/97: mknod d0/d14/d17/d18/c26 0 2024-04-20T10:59:11.398 INFO:tasks.workunit.client.0.smithi027.stdout:4/98: fdatasync d0/d14/d17/f1e 0 2024-04-20T10:59:11.398 INFO:tasks.workunit.client.0.smithi027.stdout:4/99: truncate d0/d14/d17/f16 705065 0 2024-04-20T10:59:11.400 INFO:tasks.workunit.client.0.smithi027.stdout:6/88: dwrite da/dd/f14 [0,4194304] 0 2024-04-20T10:59:11.401 INFO:tasks.workunit.client.0.smithi027.stdout:0/92: unlink d0/d7/fe 0 2024-04-20T10:59:11.401 INFO:tasks.workunit.client.0.smithi027.stdout:0/93: write d0/ff [887851,119259] 0 2024-04-20T10:59:11.403 INFO:tasks.workunit.client.0.smithi027.stdout:7/91: dwrite f2 [0,4194304] 0 2024-04-20T10:59:11.404 INFO:tasks.workunit.client.0.smithi027.stdout:7/92: creat d4/d5/f14 x:0 0 0 2024-04-20T10:59:11.405 INFO:tasks.workunit.client.0.smithi027.stdout:6/89: truncate da/df/f11 3837150 0 2024-04-20T10:59:11.407 INFO:tasks.workunit.client.0.smithi027.stdout:7/93: truncate d4/d5/fa 496305 0 2024-04-20T10:59:11.407 INFO:tasks.workunit.client.0.smithi027.stdout:8/108: dwrite d9/fb [0,4194304] 0 2024-04-20T10:59:11.409 INFO:tasks.workunit.client.0.smithi027.stdout:6/90: mkdir da/d21 0 2024-04-20T10:59:11.410 INFO:tasks.workunit.client.0.smithi027.stdout:0/94: dread d0/ff [0,4194304] 0 2024-04-20T10:59:11.411 INFO:tasks.workunit.client.0.smithi027.stdout:8/109: creat d9/de/d1e/f1f x:0 0 0 2024-04-20T10:59:11.412 INFO:tasks.workunit.client.0.smithi027.stdout:6/91: mknod da/df/d19/c22 0 2024-04-20T10:59:11.413 INFO:tasks.workunit.client.0.smithi027.stdout:0/95: mkdir d0/d11 0 2024-04-20T10:59:11.414 INFO:tasks.workunit.client.0.smithi027.stdout:8/110: mkdir d9/de/d20 0 2024-04-20T10:59:11.414 INFO:tasks.workunit.client.0.smithi027.stdout:8/111: truncate d9/de/f1b 779898 0 2024-04-20T10:59:11.415 INFO:tasks.workunit.client.0.smithi027.stdout:1/113: dwrite d3/d7/f1a [0,4194304] 0 2024-04-20T10:59:11.415 INFO:tasks.workunit.client.0.smithi027.stdout:1/114: chown d3/d7/f1a 1571471 1 2024-04-20T10:59:11.416 INFO:tasks.workunit.client.0.smithi027.stdout:4/100: dwrite d0/fb [0,4194304] 0 2024-04-20T10:59:11.416 INFO:tasks.workunit.client.0.smithi027.stdout:4/101: chown d0 2160 1 2024-04-20T10:59:11.421 INFO:tasks.workunit.client.0.smithi027.stdout:6/92: rename da/df/d19/c22 to da/df/c23 0 2024-04-20T10:59:11.426 INFO:tasks.workunit.client.0.smithi027.stdout:4/102: creat d0/f27 x:0 0 0 2024-04-20T10:59:11.426 INFO:tasks.workunit.client.0.smithi027.stdout:6/93: rename l6 to da/df/d19/l24 0 2024-04-20T10:59:11.426 INFO:tasks.workunit.client.0.smithi027.stdout:7/94: dwrite d4/fc [0,4194304] 0 2024-04-20T10:59:11.426 INFO:tasks.workunit.client.0.smithi027.stdout:7/95: write d4/d5/fa [1015805,109684] 0 2024-04-20T10:59:11.427 INFO:tasks.workunit.client.0.smithi027.stdout:0/96: dread d0/d7/fa [0,4194304] 0 2024-04-20T10:59:11.427 INFO:tasks.workunit.client.0.smithi027.stdout:0/97: stat d0/d7 0 2024-04-20T10:59:11.430 INFO:tasks.workunit.client.0.smithi027.stdout:6/94: creat da/dd/f25 x:0 0 0 2024-04-20T10:59:11.433 INFO:tasks.workunit.client.0.smithi027.stdout:0/98: mkdir d0/d7/d12 0 2024-04-20T10:59:11.435 INFO:tasks.workunit.client.0.smithi027.stdout:8/112: dwrite d9/fd [4194304,4194304] 0 2024-04-20T10:59:11.435 INFO:tasks.workunit.client.0.smithi027.stdout:4/103: rename d0/f12 to d0/f28 0 2024-04-20T10:59:11.436 INFO:tasks.workunit.client.0.smithi027.stdout:0/99: symlink d0/l13 0 2024-04-20T10:59:11.439 INFO:tasks.workunit.client.0.smithi027.stdout:0/100: dread d0/ff [0,4194304] 0 2024-04-20T10:59:11.439 INFO:tasks.workunit.client.0.smithi027.stdout:0/101: write d0/d7/fa [1932657,53773] 0 2024-04-20T10:59:11.443 INFO:tasks.workunit.client.0.smithi027.stdout:0/102: truncate d0/ff 433336 0 2024-04-20T10:59:11.444 INFO:tasks.workunit.client.0.smithi027.stdout:1/115: dwrite d3/d7/f1a [4194304,4194304] 0 2024-04-20T10:59:11.449 INFO:tasks.workunit.client.0.smithi027.stdout:7/96: dwrite d4/fc [0,4194304] 0 2024-04-20T10:59:11.449 INFO:tasks.workunit.client.0.smithi027.stdout:1/116: dread d3/d7/f8 [0,4194304] 0 2024-04-20T10:59:11.450 INFO:tasks.workunit.client.0.smithi027.stdout:7/97: mknod d4/d5/d8/c15 0 2024-04-20T10:59:11.452 INFO:tasks.workunit.client.0.smithi027.stdout:7/98: unlink f1 0 2024-04-20T10:59:11.453 INFO:tasks.workunit.client.0.smithi027.stdout:7/99: mknod d4/d5/d8/dd/c16 0 2024-04-20T10:59:11.453 INFO:tasks.workunit.client.0.smithi027.stdout:7/100: write d4/d5/f14 [9477,43972] 0 2024-04-20T10:59:11.454 INFO:tasks.workunit.client.0.smithi027.stdout:7/101: unlink f2 0 2024-04-20T10:59:11.454 INFO:tasks.workunit.client.0.smithi027.stdout:7/102: write d4/d5/d8/dd/ff [330261,103169] 0 2024-04-20T10:59:11.455 INFO:tasks.workunit.client.0.smithi027.stdout:7/103: creat d4/f17 x:0 0 0 2024-04-20T10:59:11.459 INFO:tasks.workunit.client.0.smithi027.stdout:0/103: dread d0/d7/fa [0,4194304] 0 2024-04-20T10:59:11.460 INFO:tasks.workunit.client.0.smithi027.stdout:0/104: mknod d0/d7/c14 0 2024-04-20T10:59:11.461 INFO:tasks.workunit.client.0.smithi027.stdout:1/117: write d3/d7/f1d [1001091,32770] 0 2024-04-20T10:59:11.463 INFO:tasks.workunit.client.0.smithi027.stdout:1/118: creat d3/d7/d2d/f2f x:0 0 0 2024-04-20T10:59:11.463 INFO:tasks.workunit.client.0.smithi027.stdout:1/119: chown d3/d7/c2b 187439291 1 2024-04-20T10:59:11.463 INFO:tasks.workunit.client.0.smithi027.stdout:1/120: read - d3/d7/d2d/f2f zero size 2024-04-20T10:59:11.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/121: mknod d3/d20/c30 0 2024-04-20T10:59:11.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/122: stat d3/f1b 0 2024-04-20T10:59:11.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/123: symlink d3/d7/d28/l31 0 2024-04-20T10:59:11.465 INFO:tasks.workunit.client.0.smithi027.stdout:1/124: symlink d3/l32 0 2024-04-20T10:59:11.466 INFO:tasks.workunit.client.0.smithi027.stdout:1/125: link d3/d7/f26 d3/d20/f33 0 2024-04-20T10:59:11.467 INFO:tasks.workunit.client.0.smithi027.stdout:1/126: rename d3/d7/f12 to d3/d7/f34 0 2024-04-20T10:59:11.468 INFO:tasks.workunit.client.0.smithi027.stdout:6/95: dwrite da/dd/fe [0,4194304] 0 2024-04-20T10:59:11.468 INFO:tasks.workunit.client.0.smithi027.stdout:1/127: unlink d3/d7/d28/f2e 0 2024-04-20T10:59:11.469 INFO:tasks.workunit.client.0.smithi027.stdout:8/113: dwrite f2 [0,4194304] 0 2024-04-20T10:59:11.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/104: dwrite d0/f13 [0,4194304] 0 2024-04-20T10:59:11.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/105: dread - d0/d14/d17/f1f zero size 2024-04-20T10:59:11.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/106: write d0/d14/d17/d18/f20 [418811,13419] 0 2024-04-20T10:59:11.469 INFO:tasks.workunit.client.0.smithi027.stdout:1/128: dread d3/d7/fc [0,4194304] 0 2024-04-20T10:59:11.470 INFO:tasks.workunit.client.0.smithi027.stdout:7/104: dread d4/d5/d8/dd/ff [0,4194304] 0 2024-04-20T10:59:11.471 INFO:tasks.workunit.client.0.smithi027.stdout:6/96: rename da/c1d to da/df/c26 0 2024-04-20T10:59:11.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/105: dread d4/d5/f14 [0,4194304] 0 2024-04-20T10:59:11.478 INFO:tasks.workunit.client.0.smithi027.stdout:0/105: link d0/ff d0/f15 0 2024-04-20T10:59:11.479 INFO:tasks.workunit.client.0.smithi027.stdout:8/114: dread f2 [0,4194304] 0 2024-04-20T10:59:11.479 INFO:tasks.workunit.client.0.smithi027.stdout:6/97: symlink da/df/d19/l27 0 2024-04-20T10:59:11.479 INFO:tasks.workunit.client.0.smithi027.stdout:6/98: fdatasync f3 0 2024-04-20T10:59:11.479 INFO:tasks.workunit.client.0.smithi027.stdout:6/99: read - da/f1b zero size 2024-04-20T10:59:11.479 INFO:tasks.workunit.client.0.smithi027.stdout:6/100: truncate f3 4971636 0 2024-04-20T10:59:11.480 INFO:tasks.workunit.client.0.smithi027.stdout:0/106: dread d0/f15 [0,4194304] 0 2024-04-20T10:59:11.480 INFO:tasks.workunit.client.0.smithi027.stdout:4/107: mknod d0/c29 0 2024-04-20T10:59:11.480 INFO:tasks.workunit.client.0.smithi027.stdout:4/108: truncate d0/f9 432422 0 2024-04-20T10:59:11.480 INFO:tasks.workunit.client.0.smithi027.stdout:4/109: stat d0/d14/d17/d1a/c22 0 2024-04-20T10:59:11.481 INFO:tasks.workunit.client.0.smithi027.stdout:1/129: truncate d3/d7/ff 1356608 0 2024-04-20T10:59:11.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/81: sync 2024-04-20T10:59:11.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/82: fdatasync f9 0 2024-04-20T10:59:11.488 INFO:tasks.workunit.client.0.smithi027.stdout:8/115: dread d9/fb [0,4194304] 0 2024-04-20T10:59:11.489 INFO:tasks.workunit.client.0.smithi027.stdout:5/83: mkdir d19/d1b/d1c 0 2024-04-20T10:59:11.490 INFO:tasks.workunit.client.0.smithi027.stdout:5/84: dread - f16 zero size 2024-04-20T10:59:11.490 INFO:tasks.workunit.client.0.smithi027.stdout:8/116: dread d9/de/f1b [0,4194304] 0 2024-04-20T10:59:11.490 INFO:tasks.workunit.client.0.smithi027.stdout:8/117: dread - d9/de/d1e/f1f zero size 2024-04-20T10:59:11.490 INFO:tasks.workunit.client.0.smithi027.stdout:7/106: write d4/d5/f6 [2234614,29831] 0 2024-04-20T10:59:11.490 INFO:tasks.workunit.client.0.smithi027.stdout:7/107: readlink l3 0 2024-04-20T10:59:11.490 INFO:tasks.workunit.client.0.smithi027.stdout:6/101: read f3 [4888487,98048] 0 2024-04-20T10:59:11.492 INFO:tasks.workunit.client.0.smithi027.stdout:7/108: creat d4/d5/d8/f18 x:0 0 0 2024-04-20T10:59:11.492 INFO:tasks.workunit.client.0.smithi027.stdout:6/102: mkdir da/d21/d28 0 2024-04-20T10:59:11.494 INFO:tasks.workunit.client.0.smithi027.stdout:7/109: creat d4/d5/f19 x:0 0 0 2024-04-20T10:59:11.494 INFO:tasks.workunit.client.0.smithi027.stdout:7/110: chown l3 173941 1 2024-04-20T10:59:11.503 INFO:tasks.workunit.client.0.smithi027.stdout:8/118: write f3 [2485788,28260] 0 2024-04-20T10:59:11.503 INFO:tasks.workunit.client.0.smithi027.stdout:4/110: dwrite d0/d14/d17/f1d [0,4194304] 0 2024-04-20T10:59:11.505 INFO:tasks.workunit.client.0.smithi027.stdout:8/119: creat d9/de/d18/f21 x:0 0 0 2024-04-20T10:59:11.506 INFO:tasks.workunit.client.0.smithi027.stdout:6/103: dread da/dd/f13 [0,4194304] 0 2024-04-20T10:59:11.506 INFO:tasks.workunit.client.0.smithi027.stdout:4/111: getdents d0/d14/d17/d1b 0 2024-04-20T10:59:11.506 INFO:tasks.workunit.client.0.smithi027.stdout:8/120: truncate d9/de/f1d 559520 0 2024-04-20T10:59:11.507 INFO:tasks.workunit.client.0.smithi027.stdout:6/104: write da/fb [398431,39133] 0 2024-04-20T10:59:11.507 INFO:tasks.workunit.client.0.smithi027.stdout:8/121: creat d9/de/d18/f22 x:0 0 0 2024-04-20T10:59:11.508 INFO:tasks.workunit.client.0.smithi027.stdout:8/122: mknod d9/de/d18/c23 0 2024-04-20T10:59:11.509 INFO:tasks.workunit.client.0.smithi027.stdout:8/123: link d9/de/f1a d9/f24 0 2024-04-20T10:59:11.510 INFO:tasks.workunit.client.0.smithi027.stdout:8/124: dread - d9/f24 zero size 2024-04-20T10:59:11.510 INFO:tasks.workunit.client.0.smithi027.stdout:8/125: stat d9/de/c1c 0 2024-04-20T10:59:11.510 INFO:tasks.workunit.client.0.smithi027.stdout:8/126: write d9/f24 [357807,86322] 0 2024-04-20T10:59:11.510 INFO:tasks.workunit.client.0.smithi027.stdout:8/127: creat d9/de/d20/f25 x:0 0 0 2024-04-20T10:59:11.510 INFO:tasks.workunit.client.0.smithi027.stdout:8/128: mkdir d9/de/d20/d26 0 2024-04-20T10:59:11.511 INFO:tasks.workunit.client.0.smithi027.stdout:8/129: fsync d9/de/f1b 0 2024-04-20T10:59:11.511 INFO:tasks.workunit.client.0.smithi027.stdout:8/130: write d9/de/f1b [461180,99188] 0 2024-04-20T10:59:11.512 INFO:tasks.workunit.client.0.smithi027.stdout:8/131: link d9/ff d9/de/d20/f27 0 2024-04-20T10:59:11.517 INFO:tasks.workunit.client.0.smithi027.stdout:4/112: write d0/fc [4032243,113505] 0 2024-04-20T10:59:11.518 INFO:tasks.workunit.client.0.smithi027.stdout:4/113: mknod d0/d14/d17/d18/c2a 0 2024-04-20T10:59:11.518 INFO:tasks.workunit.client.0.smithi027.stdout:1/130: dwrite d3/f17 [0,4194304] 0 2024-04-20T10:59:11.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/85: dwrite f0 [0,4194304] 0 2024-04-20T10:59:11.520 INFO:tasks.workunit.client.0.smithi027.stdout:5/86: chown l10 128 1 2024-04-20T10:59:11.523 INFO:tasks.workunit.client.0.smithi027.stdout:1/131: mkdir d3/d7/d28/d35 0 2024-04-20T10:59:11.523 INFO:tasks.workunit.client.0.smithi027.stdout:8/132: truncate d9/de/f1d 1215027 0 2024-04-20T10:59:11.525 INFO:tasks.workunit.client.0.smithi027.stdout:5/87: symlink d19/l1d 0 2024-04-20T10:59:11.526 INFO:tasks.workunit.client.0.smithi027.stdout:1/132: rename d3/d7/c25 to d3/d7/c36 0 2024-04-20T10:59:11.528 INFO:tasks.workunit.client.0.smithi027.stdout:4/114: write d0/fb [3555064,56283] 0 2024-04-20T10:59:11.528 INFO:tasks.workunit.client.0.smithi027.stdout:4/115: write d0/d14/d17/f1d [4821117,5064] 0 2024-04-20T10:59:11.528 INFO:tasks.workunit.client.0.smithi027.stdout:4/116: truncate d0/d14/d17/d18/f20 1023471 0 2024-04-20T10:59:11.528 INFO:tasks.workunit.client.0.smithi027.stdout:4/117: dread - d0/d14/d17/f1f zero size 2024-04-20T10:59:11.529 INFO:tasks.workunit.client.0.smithi027.stdout:8/133: creat d9/de/d20/f28 x:0 0 0 2024-04-20T10:59:11.530 INFO:tasks.workunit.client.0.smithi027.stdout:5/88: mkdir d19/d1b/d1c/d1e 0 2024-04-20T10:59:11.530 INFO:tasks.workunit.client.0.smithi027.stdout:1/133: link d3/d20/c30 d3/d20/c37 0 2024-04-20T10:59:11.530 INFO:tasks.workunit.client.0.smithi027.stdout:1/134: fsync d3/d7/f8 0 2024-04-20T10:59:11.531 INFO:tasks.workunit.client.0.smithi027.stdout:1/135: dread d3/d7/f8 [0,4194304] 0 2024-04-20T10:59:11.531 INFO:tasks.workunit.client.0.smithi027.stdout:4/118: symlink d0/d14/l2b 0 2024-04-20T10:59:11.531 INFO:tasks.workunit.client.0.smithi027.stdout:4/119: fsync d0/d14/d17/f1c 0 2024-04-20T10:59:11.532 INFO:tasks.workunit.client.0.smithi027.stdout:8/134: link d9/de/l14 d9/de/d1e/l29 0 2024-04-20T10:59:11.532 INFO:tasks.workunit.client.0.smithi027.stdout:1/136: dread d3/f1c [0,4194304] 0 2024-04-20T10:59:11.533 INFO:tasks.workunit.client.0.smithi027.stdout:5/89: creat d19/d1b/f1f x:0 0 0 2024-04-20T10:59:11.533 INFO:tasks.workunit.client.0.smithi027.stdout:5/90: truncate fc 9073263 0 2024-04-20T10:59:11.533 INFO:tasks.workunit.client.0.smithi027.stdout:5/91: truncate d19/d1b/f1f 735556 0 2024-04-20T10:59:11.534 INFO:tasks.workunit.client.0.smithi027.stdout:8/135: truncate d9/fb 127535 0 2024-04-20T10:59:11.534 INFO:tasks.workunit.client.0.smithi027.stdout:6/105: dwrite da/fb [0,4194304] 0 2024-04-20T10:59:11.534 INFO:tasks.workunit.client.0.smithi027.stdout:6/106: fsync da/df/f1a 0 2024-04-20T10:59:11.535 INFO:tasks.workunit.client.0.smithi027.stdout:1/137: mkdir d3/d7/d38 0 2024-04-20T10:59:11.535 INFO:tasks.workunit.client.0.smithi027.stdout:5/92: truncate f18 1829377 0 2024-04-20T10:59:11.536 INFO:tasks.workunit.client.0.smithi027.stdout:4/120: mknod d0/c2c 0 2024-04-20T10:59:11.538 INFO:tasks.workunit.client.0.smithi027.stdout:8/136: mknod d9/de/d20/d26/c2a 0 2024-04-20T10:59:11.541 INFO:tasks.workunit.client.0.smithi027.stdout:1/138: symlink d3/d7/d28/d35/l39 0 2024-04-20T10:59:11.541 INFO:tasks.workunit.client.0.smithi027.stdout:1/139: write d3/d7/d2d/f2f [697772,82192] 0 2024-04-20T10:59:11.542 INFO:tasks.workunit.client.0.smithi027.stdout:4/121: creat d0/d14/d17/d18/f2d x:0 0 0 2024-04-20T10:59:11.543 INFO:tasks.workunit.client.0.smithi027.stdout:8/137: creat d9/de/f2b x:0 0 0 2024-04-20T10:59:11.544 INFO:tasks.workunit.client.0.smithi027.stdout:6/107: read da/dd/fe [1514540,58093] 0 2024-04-20T10:59:11.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/104: sync 2024-04-20T10:59:11.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/105: dread - d4/d14/f19 zero size 2024-04-20T10:59:11.544 INFO:tasks.workunit.client.0.smithi027.stdout:1/140: link d3/d7/c2a d3/d7/c3a 0 2024-04-20T10:59:11.544 INFO:tasks.workunit.client.0.smithi027.stdout:1/141: truncate d3/d7/f26 471519 0 2024-04-20T10:59:11.545 INFO:tasks.workunit.client.0.smithi027.stdout:9/106: dread d4/f11 [0,4194304] 0 2024-04-20T10:59:11.545 INFO:tasks.workunit.client.0.smithi027.stdout:4/122: rename d0/f6 to d0/d14/d17/f2e 0 2024-04-20T10:59:11.546 INFO:tasks.workunit.client.0.smithi027.stdout:4/123: chown d0/d14/d17/f16 264668275 1 2024-04-20T10:59:11.546 INFO:tasks.workunit.client.0.smithi027.stdout:6/108: creat da/df/f29 x:0 0 0 2024-04-20T10:59:11.548 INFO:tasks.workunit.client.0.smithi027.stdout:9/107: stat d4/c12 0 2024-04-20T10:59:11.548 INFO:tasks.workunit.client.0.smithi027.stdout:1/142: rename d3/d20/c37 to d3/d7/d2d/c3b 0 2024-04-20T10:59:11.548 INFO:tasks.workunit.client.0.smithi027.stdout:9/108: fsync d4/fc 0 2024-04-20T10:59:11.559 INFO:tasks.workunit.client.0.smithi027.stdout:4/124: mkdir d0/d14/d17/d1b/d2f 0 2024-04-20T10:59:11.559 INFO:tasks.workunit.client.0.smithi027.stdout:6/109: creat da/f2a x:0 0 0 2024-04-20T10:59:11.559 INFO:tasks.workunit.client.0.smithi027.stdout:6/110: chown f4 26913965 1 2024-04-20T10:59:11.559 INFO:tasks.workunit.client.0.smithi027.stdout:6/111: dread - da/df/f10 zero size 2024-04-20T10:59:11.560 INFO:tasks.workunit.client.0.smithi027.stdout:4/125: truncate d0/fb 4026701 0 2024-04-20T10:59:11.561 INFO:tasks.workunit.client.0.smithi027.stdout:4/126: creat d0/d14/d17/d1b/f30 x:0 0 0 2024-04-20T10:59:11.563 INFO:tasks.workunit.client.0.smithi027.stdout:4/127: creat d0/d14/d17/f31 x:0 0 0 2024-04-20T10:59:11.563 INFO:tasks.workunit.client.0.smithi027.stdout:4/128: fdatasync d0/f13 0 2024-04-20T10:59:11.563 INFO:tasks.workunit.client.0.smithi027.stdout:9/109: read d4/f7 [8345626,67572] 0 2024-04-20T10:59:11.563 INFO:tasks.workunit.client.0.smithi027.stdout:8/138: dwrite f2 [0,4194304] 0 2024-04-20T10:59:11.564 INFO:tasks.workunit.client.0.smithi027.stdout:8/139: write d9/f15 [318766,90407] 0 2024-04-20T10:59:11.564 INFO:tasks.workunit.client.0.smithi027.stdout:4/129: symlink d0/d14/d17/d1b/d2f/l32 0 2024-04-20T10:59:11.565 INFO:tasks.workunit.client.0.smithi027.stdout:9/110: symlink d4/d14/l1d 0 2024-04-20T10:59:11.566 INFO:tasks.workunit.client.0.smithi027.stdout:4/130: mkdir d0/d14/d17/d1b/d33 0 2024-04-20T10:59:11.567 INFO:tasks.workunit.client.0.smithi027.stdout:4/131: symlink d0/d14/l34 0 2024-04-20T10:59:11.567 INFO:tasks.workunit.client.0.smithi027.stdout:4/132: chown d0/d14/d17/d1b 51 1 2024-04-20T10:59:11.568 INFO:tasks.workunit.client.0.smithi027.stdout:4/133: creat d0/d14/d17/d1b/d33/f35 x:0 0 0 2024-04-20T10:59:11.569 INFO:tasks.workunit.client.0.smithi027.stdout:4/134: creat d0/d14/d17/d18/f36 x:0 0 0 2024-04-20T10:59:11.570 INFO:tasks.workunit.client.0.smithi027.stdout:1/143: read d3/d7/f26 [252947,42130] 0 2024-04-20T10:59:11.572 INFO:tasks.workunit.client.0.smithi027.stdout:4/135: creat d0/f37 x:0 0 0 2024-04-20T10:59:11.579 INFO:tasks.workunit.client.0.smithi027.stdout:9/111: dread d4/fc [0,4194304] 0 2024-04-20T10:59:11.579 INFO:tasks.workunit.client.0.smithi027.stdout:6/112: dwrite da/df/f11 [0,4194304] 0 2024-04-20T10:59:11.579 INFO:tasks.workunit.client.0.smithi027.stdout:6/113: truncate da/dd/f13 5028987 0 2024-04-20T10:59:11.579 INFO:tasks.workunit.client.0.smithi027.stdout:8/140: write f2 [1406928,66500] 0 2024-04-20T10:59:11.579 INFO:tasks.workunit.client.0.smithi027.stdout:8/141: chown d9/f24 4826838 1 2024-04-20T10:59:11.579 INFO:tasks.workunit.client.0.smithi027.stdout:8/142: chown d9/de/d20/f28 19659388 1 2024-04-20T10:59:11.579 INFO:tasks.workunit.client.0.smithi027.stdout:9/112: dread d4/f11 [0,4194304] 0 2024-04-20T10:59:11.579 INFO:tasks.workunit.client.0.smithi027.stdout:6/114: symlink da/df/d19/l2b 0 2024-04-20T10:59:11.580 INFO:tasks.workunit.client.0.smithi027.stdout:8/143: mkdir d9/de/d18/d2c 0 2024-04-20T10:59:11.580 INFO:tasks.workunit.client.0.smithi027.stdout:8/144: write d9/de/f2b [970220,40652] 0 2024-04-20T10:59:11.580 INFO:tasks.workunit.client.0.smithi027.stdout:8/145: truncate d9/de/d1e/f1f 655237 0 2024-04-20T10:59:11.581 INFO:tasks.workunit.client.0.smithi027.stdout:9/113: link d4/d14/f19 d4/f1e 0 2024-04-20T10:59:11.582 INFO:tasks.workunit.client.0.smithi027.stdout:9/114: read - d4/d14/f19 zero size 2024-04-20T10:59:11.583 INFO:tasks.workunit.client.0.smithi027.stdout:6/115: mknod da/df/d1f/c2c 0 2024-04-20T10:59:11.583 INFO:tasks.workunit.client.0.smithi027.stdout:6/116: write f3 [3240039,12285] 0 2024-04-20T10:59:11.584 INFO:tasks.workunit.client.0.smithi027.stdout:8/146: link d9/de/d18/f22 d9/de/d18/f2d 0 2024-04-20T10:59:11.584 INFO:tasks.workunit.client.0.smithi027.stdout:8/147: write d9/de/f1a [13522,128604] 0 2024-04-20T10:59:11.585 INFO:tasks.workunit.client.0.smithi027.stdout:9/115: mkdir d4/d1b/d1f 0 2024-04-20T10:59:11.586 INFO:tasks.workunit.client.0.smithi027.stdout:7/111: dwrite d4/d5/f6 [0,4194304] 0 2024-04-20T10:59:11.586 INFO:tasks.workunit.client.0.smithi027.stdout:6/117: unlink da/dd/f14 0 2024-04-20T10:59:11.587 INFO:tasks.workunit.client.0.smithi027.stdout:8/148: symlink d9/de/d18/d2c/l2e 0 2024-04-20T10:59:11.587 INFO:tasks.workunit.client.0.smithi027.stdout:9/116: mkdir d4/d1b/d1f/d20 0 2024-04-20T10:59:11.588 INFO:tasks.workunit.client.0.smithi027.stdout:9/117: readlink d4/l18 0 2024-04-20T10:59:11.588 INFO:tasks.workunit.client.0.smithi027.stdout:2/57: sync 2024-04-20T10:59:11.588 INFO:tasks.workunit.client.0.smithi027.stdout:3/175: sync 2024-04-20T10:59:11.588 INFO:tasks.workunit.client.0.smithi027.stdout:2/58: fdatasync d5/f8 0 2024-04-20T10:59:11.589 INFO:tasks.workunit.client.0.smithi027.stdout:4/136: dwrite d0/d14/d17/f1d [0,4194304] 0 2024-04-20T10:59:11.594 INFO:tasks.workunit.client.0.smithi027.stdout:6/118: mkdir da/d21/d28/d2d 0 2024-04-20T10:59:11.594 INFO:tasks.workunit.client.0.smithi027.stdout:6/119: stat da/f2a 0 2024-04-20T10:59:11.595 INFO:tasks.workunit.client.0.smithi027.stdout:9/118: creat d4/d14/f21 x:0 0 0 2024-04-20T10:59:11.597 INFO:tasks.workunit.client.0.smithi027.stdout:3/176: creat d2/dd/f39 x:0 0 0 2024-04-20T10:59:11.597 INFO:tasks.workunit.client.0.smithi027.stdout:3/177: stat d2/dd/df/c1b 0 2024-04-20T10:59:11.598 INFO:tasks.workunit.client.0.smithi027.stdout:2/59: symlink d5/dc/l12 0 2024-04-20T10:59:11.599 INFO:tasks.workunit.client.0.smithi027.stdout:4/137: dread d0/fd [0,4194304] 0 2024-04-20T10:59:11.600 INFO:tasks.workunit.client.0.smithi027.stdout:2/60: mkdir d5/dc/d13 0 2024-04-20T10:59:11.602 INFO:tasks.workunit.client.0.smithi027.stdout:4/138: rename d0/f23 to d0/d14/d17/d18/f38 0 2024-04-20T10:59:11.603 INFO:tasks.workunit.client.0.smithi027.stdout:2/61: mknod d5/dc/d13/c14 0 2024-04-20T10:59:11.603 INFO:tasks.workunit.client.0.smithi027.stdout:2/62: chown f3 9 1 2024-04-20T10:59:11.604 INFO:tasks.workunit.client.0.smithi027.stdout:4/139: creat d0/d14/d17/d18/f39 x:0 0 0 2024-04-20T10:59:11.606 INFO:tasks.workunit.client.0.smithi027.stdout:2/63: truncate d5/f8 7535023 0 2024-04-20T10:59:11.606 INFO:tasks.workunit.client.0.smithi027.stdout:2/64: dread - d5/fd zero size 2024-04-20T10:59:11.607 INFO:tasks.workunit.client.0.smithi027.stdout:2/65: chown d5/dc 842400 1 2024-04-20T10:59:11.608 INFO:tasks.workunit.client.0.smithi027.stdout:8/149: getdents d9/de/d18 0 2024-04-20T10:59:11.609 INFO:tasks.workunit.client.0.smithi027.stdout:4/140: link d0/d14/d17/d18/f36 d0/f3a 0 2024-04-20T10:59:11.610 INFO:tasks.workunit.client.0.smithi027.stdout:4/141: write d0/d14/d17/d1b/f21 [4795834,60883] 0 2024-04-20T10:59:11.611 INFO:tasks.workunit.client.0.smithi027.stdout:8/150: creat d9/de/f2f x:0 0 0 2024-04-20T10:59:11.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/142: mkdir d0/d14/d17/d1a/d3b 0 2024-04-20T10:59:11.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/143: dread - d0/d14/d17/f1c zero size 2024-04-20T10:59:11.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/144: truncate d0/d14/d17/d18/f2d 144746 0 2024-04-20T10:59:11.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/145: dread - d0/d14/d17/d18/f36 zero size 2024-04-20T10:59:11.614 INFO:tasks.workunit.client.0.smithi027.stdout:8/151: rmdir d9/de/d1e 39 2024-04-20T10:59:11.615 INFO:tasks.workunit.client.0.smithi027.stdout:4/146: symlink d0/d14/d17/d1a/d3b/l3c 0 2024-04-20T10:59:11.615 INFO:tasks.workunit.client.0.smithi027.stdout:4/147: write d0/d14/d17/d1b/f21 [5269397,4947] 0 2024-04-20T10:59:11.616 INFO:tasks.workunit.client.0.smithi027.stdout:4/148: creat d0/d14/d17/d1b/f3d x:0 0 0 2024-04-20T10:59:11.616 INFO:tasks.workunit.client.0.smithi027.stdout:0/107: sync 2024-04-20T10:59:11.617 INFO:tasks.workunit.client.0.smithi027.stdout:4/149: truncate d0/f28 2364017 0 2024-04-20T10:59:11.619 INFO:tasks.workunit.client.0.smithi027.stdout:7/112: dwrite d4/d5/fa [0,4194304] 0 2024-04-20T10:59:11.619 INFO:tasks.workunit.client.0.smithi027.stdout:7/113: chown d4/d5/d8/dd 51 1 2024-04-20T10:59:11.619 INFO:tasks.workunit.client.0.smithi027.stdout:7/114: chown d4/d5/d8/dd/ff 48081860 1 2024-04-20T10:59:11.619 INFO:tasks.workunit.client.0.smithi027.stdout:3/178: dwrite d2/f16 [4194304,4194304] 0 2024-04-20T10:59:11.620 INFO:tasks.workunit.client.0.smithi027.stdout:3/179: read - d2/d18/f26 zero size 2024-04-20T10:59:11.620 INFO:tasks.workunit.client.0.smithi027.stdout:3/180: write d2/d18/d2e/f35 [536430,55662] 0 2024-04-20T10:59:11.620 INFO:tasks.workunit.client.0.smithi027.stdout:7/115: link d4/d5/d8/dd/c16 d4/d5/d8/c1a 0 2024-04-20T10:59:11.623 INFO:tasks.workunit.client.0.smithi027.stdout:3/181: dread d2/dd/f10 [0,4194304] 0 2024-04-20T10:59:11.632 INFO:tasks.workunit.client.0.smithi027.stdout:3/182: dread d2/f21 [0,4194304] 0 2024-04-20T10:59:11.632 INFO:tasks.workunit.client.0.smithi027.stdout:2/66: rename d5/f8 to d5/dc/f15 0 2024-04-20T10:59:11.633 INFO:tasks.workunit.client.0.smithi027.stdout:3/183: truncate d2/fe 3667311 0 2024-04-20T10:59:11.634 INFO:tasks.workunit.client.0.smithi027.stdout:3/184: write d2/dd/df/d23/f2a [425656,93028] 0 2024-04-20T10:59:11.634 INFO:tasks.workunit.client.0.smithi027.stdout:3/185: rmdir d2/d18/d2c 39 2024-04-20T10:59:11.635 INFO:tasks.workunit.client.0.smithi027.stdout:3/186: creat d2/dd/df/f3a x:0 0 0 2024-04-20T10:59:11.639 INFO:tasks.workunit.client.0.smithi027.stdout:4/150: dwrite d0/d14/d17/d1b/f21 [0,4194304] 0 2024-04-20T10:59:11.639 INFO:tasks.workunit.client.0.smithi027.stdout:4/151: rename d0/d14/d17/d1b to d0/d14/d17/d1b/d3e 22 2024-04-20T10:59:11.640 INFO:tasks.workunit.client.0.smithi027.stdout:0/108: dwrite d0/f15 [0,4194304] 0 2024-04-20T10:59:11.644 INFO:tasks.workunit.client.0.smithi027.stdout:2/67: read f3 [3656784,56104] 0 2024-04-20T10:59:11.644 INFO:tasks.workunit.client.0.smithi027.stdout:2/68: readlink d5/l11 0 2024-04-20T10:59:11.644 INFO:tasks.workunit.client.0.smithi027.stdout:0/109: creat d0/d7/f16 x:0 0 0 2024-04-20T10:59:11.646 INFO:tasks.workunit.client.0.smithi027.stdout:0/110: symlink d0/d7/d12/l17 0 2024-04-20T10:59:11.647 INFO:tasks.workunit.client.0.smithi027.stdout:2/69: read d5/dc/f15 [925118,118598] 0 2024-04-20T10:59:11.647 INFO:tasks.workunit.client.0.smithi027.stdout:2/70: fdatasync d5/dc/fe 0 2024-04-20T10:59:11.649 INFO:tasks.workunit.client.0.smithi027.stdout:2/71: rename d5/dc/d13 to d5/dc/d16 0 2024-04-20T10:59:11.667 INFO:tasks.workunit.client.0.smithi027.stdout:3/187: dwrite d2/f9 [4194304,4194304] 0 2024-04-20T10:59:11.667 INFO:tasks.workunit.client.0.smithi027.stdout:3/188: dread - d2/f1f zero size 2024-04-20T10:59:11.669 INFO:tasks.workunit.client.0.smithi027.stdout:9/119: dwrite d4/f7 [0,4194304] 0 2024-04-20T10:59:11.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/111: dwrite d0/d7/fa [0,4194304] 0 2024-04-20T10:59:11.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/112: write d0/d7/f16 [398061,81987] 0 2024-04-20T10:59:11.673 INFO:tasks.workunit.client.0.smithi027.stdout:4/152: dwrite d0/d14/d17/f1f [0,4194304] 0 2024-04-20T10:59:11.684 INFO:tasks.workunit.client.0.smithi027.stdout:3/189: dwrite d2/f14 [0,4194304] 0 2024-04-20T10:59:11.684 INFO:tasks.workunit.client.0.smithi027.stdout:3/190: chown d2/d18/l31 240929 1 2024-04-20T10:59:11.921 INFO:tasks.workunit.client.0.smithi027.stdout:0/113: mknod d0/d7/c18 0 2024-04-20T10:59:11.921 INFO:tasks.workunit.client.0.smithi027.stdout:9/120: creat d4/d1b/f22 x:0 0 0 2024-04-20T10:59:11.921 INFO:tasks.workunit.client.0.smithi027.stdout:0/114: truncate d0/d7/f16 841305 0 2024-04-20T10:59:11.922 INFO:tasks.workunit.client.0.smithi027.stdout:4/153: rmdir d0/d14/d17/d1b/d33 39 2024-04-20T10:59:11.922 INFO:tasks.workunit.client.0.smithi027.stdout:4/154: write d0/d14/d17/d18/f2d [875170,88248] 0 2024-04-20T10:59:11.922 INFO:tasks.workunit.client.0.smithi027.stdout:4/155: write d0/d14/d17/d1b/f3d [246399,40858] 0 2024-04-20T10:59:11.923 INFO:tasks.workunit.client.0.smithi027.stdout:3/191: rename d2/c11 to d2/c3b 0 2024-04-20T10:59:11.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/156: rename d0/d14/d17/f1f to d0/d14/d17/d1b/d2f/f3f 0 2024-04-20T10:59:11.925 INFO:tasks.workunit.client.0.smithi027.stdout:4/157: rename d0/d14/d17/d18/c26 to d0/d14/d17/d1b/d33/c40 0 2024-04-20T10:59:11.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/158: symlink d0/d14/d17/d1b/l41 0 2024-04-20T10:59:11.928 INFO:tasks.workunit.client.0.smithi027.stdout:4/159: link d0/fb d0/d14/d17/d1b/d2f/f42 0 2024-04-20T10:59:11.928 INFO:tasks.workunit.client.0.smithi027.stdout:4/160: readlink d0/l10 0 2024-04-20T10:59:11.929 INFO:tasks.workunit.client.0.smithi027.stdout:4/161: link d0/c2c d0/d14/d17/d1a/c43 0 2024-04-20T10:59:11.931 INFO:tasks.workunit.client.0.smithi027.stdout:3/192: dread d2/f14 [0,4194304] 0 2024-04-20T10:59:11.931 INFO:tasks.workunit.client.0.smithi027.stdout:4/162: rename d0/d14/d17/d18/l19 to d0/d14/l44 0 2024-04-20T10:59:11.931 INFO:tasks.workunit.client.0.smithi027.stdout:5/93: sync 2024-04-20T10:59:11.931 INFO:tasks.workunit.client.0.smithi027.stdout:5/94: mkdir d19/d1b/d20 0 2024-04-20T10:59:11.932 INFO:tasks.workunit.client.0.smithi027.stdout:5/95: truncate f9 119174 0 2024-04-20T10:59:11.932 INFO:tasks.workunit.client.0.smithi027.stdout:5/96: chown c4 18 1 2024-04-20T10:59:11.933 INFO:tasks.workunit.client.0.smithi027.stdout:5/97: symlink d19/d1b/d20/l21 0 2024-04-20T10:59:11.934 INFO:tasks.workunit.client.0.smithi027.stdout:8/152: sync 2024-04-20T10:59:11.934 INFO:tasks.workunit.client.0.smithi027.stdout:6/120: sync 2024-04-20T10:59:11.935 INFO:tasks.workunit.client.0.smithi027.stdout:1/144: sync 2024-04-20T10:59:11.935 INFO:tasks.workunit.client.0.smithi027.stdout:8/153: fsync d9/f15 0 2024-04-20T10:59:11.935 INFO:tasks.workunit.client.0.smithi027.stdout:7/116: sync 2024-04-20T10:59:11.935 INFO:tasks.workunit.client.0.smithi027.stdout:6/121: mkdir da/df/d19/d2e 0 2024-04-20T10:59:11.936 INFO:tasks.workunit.client.0.smithi027.stdout:1/145: creat d3/d20/f3c x:0 0 0 2024-04-20T10:59:11.936 INFO:tasks.workunit.client.0.smithi027.stdout:1/146: stat d3/d20 0 2024-04-20T10:59:11.936 INFO:tasks.workunit.client.0.smithi027.stdout:4/163: write d0/d14/d17/f2e [227635,7238] 0 2024-04-20T10:59:11.937 INFO:tasks.workunit.client.0.smithi027.stdout:8/154: creat d9/de/d20/d26/f30 x:0 0 0 2024-04-20T10:59:11.937 INFO:tasks.workunit.client.0.smithi027.stdout:7/117: unlink d4/d5/d8/dd/c12 0 2024-04-20T10:59:11.938 INFO:tasks.workunit.client.0.smithi027.stdout:8/155: chown d9/de/d1e 48 1 2024-04-20T10:59:11.938 INFO:tasks.workunit.client.0.smithi027.stdout:7/118: chown d4/d5/f6 0 1 2024-04-20T10:59:11.938 INFO:tasks.workunit.client.0.smithi027.stdout:8/156: write d9/de/f1a [159851,36066] 0 2024-04-20T10:59:11.938 INFO:tasks.workunit.client.0.smithi027.stdout:8/157: fsync d9/de/f1b 0 2024-04-20T10:59:11.938 INFO:tasks.workunit.client.0.smithi027.stdout:6/122: creat da/d21/d28/d2d/f2f x:0 0 0 2024-04-20T10:59:11.939 INFO:tasks.workunit.client.0.smithi027.stdout:6/123: chown da/df/d1f/c2c 643076847 1 2024-04-20T10:59:11.939 INFO:tasks.workunit.client.0.smithi027.stdout:6/124: stat da/df/f10 0 2024-04-20T10:59:11.939 INFO:tasks.workunit.client.0.smithi027.stdout:6/125: write da/df/f10 [427753,7944] 0 2024-04-20T10:59:11.939 INFO:tasks.workunit.client.0.smithi027.stdout:6/126: chown da/df/f1a 77 1 2024-04-20T10:59:11.939 INFO:tasks.workunit.client.0.smithi027.stdout:6/127: stat da/df/f17 0 2024-04-20T10:59:11.939 INFO:tasks.workunit.client.0.smithi027.stdout:1/147: rename d3/d7/f26 to d3/d7/d28/f3d 0 2024-04-20T10:59:11.941 INFO:tasks.workunit.client.0.smithi027.stdout:8/158: rmdir d9/de/d18 39 2024-04-20T10:59:11.941 INFO:tasks.workunit.client.0.smithi027.stdout:6/128: creat da/f30 x:0 0 0 2024-04-20T10:59:11.942 INFO:tasks.workunit.client.0.smithi027.stdout:8/159: symlink d9/de/d20/l31 0 2024-04-20T10:59:11.943 INFO:tasks.workunit.client.0.smithi027.stdout:6/129: symlink da/d21/d28/l31 0 2024-04-20T10:59:11.943 INFO:tasks.workunit.client.0.smithi027.stdout:5/98: dread f3 [0,4194304] 0 2024-04-20T10:59:11.943 INFO:tasks.workunit.client.0.smithi027.stdout:6/130: mkdir da/df/d19/d2e/d32 0 2024-04-20T10:59:11.944 INFO:tasks.workunit.client.0.smithi027.stdout:5/99: mkdir d19/d1b/d1c/d1e/d22 0 2024-04-20T10:59:11.945 INFO:tasks.workunit.client.0.smithi027.stdout:6/131: creat da/df/f33 x:0 0 0 2024-04-20T10:59:11.945 INFO:tasks.workunit.client.0.smithi027.stdout:6/132: chown l0 16 1 2024-04-20T10:59:11.945 INFO:tasks.workunit.client.0.smithi027.stdout:6/133: dread - da/df/f17 zero size 2024-04-20T10:59:11.945 INFO:tasks.workunit.client.0.smithi027.stdout:6/134: write da/f2a [112283,34567] 0 2024-04-20T10:59:11.945 INFO:tasks.workunit.client.0.smithi027.stdout:6/135: dread - da/dd/f25 zero size 2024-04-20T10:59:11.946 INFO:tasks.workunit.client.0.smithi027.stdout:5/100: link c4 d19/c23 0 2024-04-20T10:59:11.947 INFO:tasks.workunit.client.0.smithi027.stdout:8/160: dread d9/de/f2b [0,4194304] 0 2024-04-20T10:59:11.948 INFO:tasks.workunit.client.0.smithi027.stdout:8/161: symlink d9/l32 0 2024-04-20T10:59:11.950 INFO:tasks.workunit.client.0.smithi027.stdout:2/72: dread f1 [0,4194304] 0 2024-04-20T10:59:11.951 INFO:tasks.workunit.client.0.smithi027.stdout:2/73: creat d5/f17 x:0 0 0 2024-04-20T10:59:11.955 INFO:tasks.workunit.client.0.smithi027.stdout:4/164: dwrite d0/d14/d17/f1e [0,4194304] 0 2024-04-20T10:59:11.955 INFO:tasks.workunit.client.0.smithi027.stdout:3/193: dwrite d2/fe [0,4194304] 0 2024-04-20T10:59:11.958 INFO:tasks.workunit.client.0.smithi027.stdout:7/119: rename d4/d5/d8/dd to d4/d1b 0 2024-04-20T10:59:11.962 INFO:tasks.workunit.client.0.smithi027.stdout:2/74: rename d5/cb to d5/dc/d16/c18 0 2024-04-20T10:59:11.962 INFO:tasks.workunit.client.0.smithi027.stdout:2/75: fdatasync d5/dc/f15 0 2024-04-20T10:59:11.965 INFO:tasks.workunit.client.0.smithi027.stdout:3/194: dread d2/f25 [0,4194304] 0 2024-04-20T10:59:11.965 INFO:tasks.workunit.client.0.smithi027.stdout:5/101: dread f18 [0,4194304] 0 2024-04-20T10:59:11.965 INFO:tasks.workunit.client.0.smithi027.stdout:5/102: dread - f16 zero size 2024-04-20T10:59:11.967 INFO:tasks.workunit.client.0.smithi027.stdout:5/103: rename f9 to d19/d1b/d20/f24 0 2024-04-20T10:59:11.968 INFO:tasks.workunit.client.0.smithi027.stdout:5/104: mkdir d19/d1b/d1c/d1e/d25 0 2024-04-20T10:59:11.971 INFO:tasks.workunit.client.0.smithi027.stdout:1/148: dread d3/f17 [0,4194304] 0 2024-04-20T10:59:11.971 INFO:tasks.workunit.client.0.smithi027.stdout:1/149: chown d3/f17 653 1 2024-04-20T10:59:11.976 INFO:tasks.workunit.client.0.smithi027.stdout:1/150: mknod d3/d20/c3e 0 2024-04-20T10:59:11.976 INFO:tasks.workunit.client.0.smithi027.stdout:7/120: write d4/d5/f6 [1815548,58371] 0 2024-04-20T10:59:11.988 INFO:tasks.workunit.client.0.smithi027.stdout:7/121: dread - d4/d5/f19 zero size 2024-04-20T10:59:11.988 INFO:tasks.workunit.client.0.smithi027.stdout:4/165: dwrite d0/d14/d17/d1b/f30 [0,4194304] 0 2024-04-20T10:59:11.988 INFO:tasks.workunit.client.0.smithi027.stdout:6/136: fsync da/df/f33 0 2024-04-20T10:59:11.989 INFO:tasks.workunit.client.0.smithi027.stdout:8/162: dwrite d9/de/d18/f21 [0,4194304] 0 2024-04-20T10:59:11.989 INFO:tasks.workunit.client.0.smithi027.stdout:5/105: dread d19/d1b/d20/f24 [0,4194304] 0 2024-04-20T10:59:11.989 INFO:tasks.workunit.client.0.smithi027.stdout:4/166: mknod d0/d14/d17/d18/c45 0 2024-04-20T10:59:11.989 INFO:tasks.workunit.client.0.smithi027.stdout:6/137: rmdir da/df/d19 39 2024-04-20T10:59:11.989 INFO:tasks.workunit.client.0.smithi027.stdout:8/163: creat d9/f33 x:0 0 0 2024-04-20T10:59:11.989 INFO:tasks.workunit.client.0.smithi027.stdout:8/164: truncate d9/de/f2f 271431 0 2024-04-20T10:59:11.989 INFO:tasks.workunit.client.0.smithi027.stdout:5/106: link l11 d19/d1b/d1c/d1e/d25/l26 0 2024-04-20T10:59:11.990 INFO:tasks.workunit.client.0.smithi027.stdout:5/107: creat d19/f27 x:0 0 0 2024-04-20T10:59:11.990 INFO:tasks.workunit.client.0.smithi027.stdout:8/165: creat d9/f34 x:0 0 0 2024-04-20T10:59:11.991 INFO:tasks.workunit.client.0.smithi027.stdout:8/166: chown d9/de/d20/f28 509737 1 2024-04-20T10:59:11.991 INFO:tasks.workunit.client.0.smithi027.stdout:8/167: dread - d9/de/d18/f22 zero size 2024-04-20T10:59:11.991 INFO:tasks.workunit.client.0.smithi027.stdout:4/167: read d0/f28 [1893953,68705] 0 2024-04-20T10:59:11.991 INFO:tasks.workunit.client.0.smithi027.stdout:4/168: write d0/f28 [2702053,66686] 0 2024-04-20T10:59:11.993 INFO:tasks.workunit.client.0.smithi027.stdout:5/108: unlink d19/d1b/d1c/d1e/d25/l26 0 2024-04-20T10:59:11.994 INFO:tasks.workunit.client.0.smithi027.stdout:8/168: creat d9/de/d18/d2c/f35 x:0 0 0 2024-04-20T10:59:11.997 INFO:tasks.workunit.client.0.smithi027.stdout:5/109: mknod d19/d1b/d20/c28 0 2024-04-20T10:59:11.999 INFO:tasks.workunit.client.0.smithi027.stdout:3/195: dwrite d2/f6 [0,4194304] 0 2024-04-20T10:59:12.001 INFO:tasks.workunit.client.0.smithi027.stdout:3/196: creat d2/dd/df/d33/f3c x:0 0 0 2024-04-20T10:59:12.001 INFO:tasks.workunit.client.0.smithi027.stdout:6/138: dwrite da/df/f33 [0,4194304] 0 2024-04-20T10:59:12.001 INFO:tasks.workunit.client.0.smithi027.stdout:4/169: dread d0/f28 [0,4194304] 0 2024-04-20T10:59:12.001 INFO:tasks.workunit.client.0.smithi027.stdout:4/170: truncate d0/d14/d17/d1b/d33/f35 481065 0 2024-04-20T10:59:12.001 INFO:tasks.workunit.client.0.smithi027.stdout:3/197: link d2/d18/f26 d2/dd/f3d 0 2024-04-20T10:59:12.003 INFO:tasks.workunit.client.0.smithi027.stdout:6/139: rename da/f30 to da/d21/d28/f34 0 2024-04-20T10:59:12.004 INFO:tasks.workunit.client.0.smithi027.stdout:4/171: symlink d0/l46 0 2024-04-20T10:59:12.005 INFO:tasks.workunit.client.0.smithi027.stdout:4/172: mknod d0/d14/d17/c47 0 2024-04-20T10:59:12.005 INFO:tasks.workunit.client.0.smithi027.stdout:7/122: dwrite d4/d1b/ff [0,4194304] 0 2024-04-20T10:59:12.006 INFO:tasks.workunit.client.0.smithi027.stdout:7/123: dread d4/d5/f14 [0,4194304] 0 2024-04-20T10:59:12.008 INFO:tasks.workunit.client.0.smithi027.stdout:4/173: mkdir d0/d48 0 2024-04-20T10:59:12.008 INFO:tasks.workunit.client.0.smithi027.stdout:7/124: symlink d4/d1b/l1c 0 2024-04-20T10:59:12.009 INFO:tasks.workunit.client.0.smithi027.stdout:4/174: symlink d0/d14/d17/d1b/d2f/l49 0 2024-04-20T10:59:12.012 INFO:tasks.workunit.client.0.smithi027.stdout:8/169: dwrite d9/ff [0,4194304] 0 2024-04-20T10:59:12.012 INFO:tasks.workunit.client.0.smithi027.stdout:1/151: write d3/f17 [3702449,106420] 0 2024-04-20T10:59:12.012 INFO:tasks.workunit.client.0.smithi027.stdout:3/198: dread d2/dd/f10 [4194304,4194304] 0 2024-04-20T10:59:12.015 INFO:tasks.workunit.client.0.smithi027.stdout:8/170: rename d9/de/d20/d26/c2a to d9/de/d18/d2c/c36 0 2024-04-20T10:59:12.017 INFO:tasks.workunit.client.0.smithi027.stdout:3/199: mkdir d2/d18/d3e 0 2024-04-20T10:59:12.017 INFO:tasks.workunit.client.0.smithi027.stdout:5/110: getdents d19/d1b/d1c/d1e/d25 0 2024-04-20T10:59:12.018 INFO:tasks.workunit.client.0.smithi027.stdout:8/171: rmdir d9/de/d18/d2c 39 2024-04-20T10:59:12.019 INFO:tasks.workunit.client.0.smithi027.stdout:5/111: creat d19/d1b/d1c/d1e/f29 x:0 0 0 2024-04-20T10:59:12.021 INFO:tasks.workunit.client.0.smithi027.stdout:8/172: link d9/fd d9/de/f37 0 2024-04-20T10:59:12.029 INFO:tasks.workunit.client.0.smithi027.stdout:8/173: write d9/de/f1d [286312,6922] 0 2024-04-20T10:59:12.030 INFO:tasks.workunit.client.0.smithi027.stdout:8/174: truncate d9/de/d1e/f1f 1444617 0 2024-04-20T10:59:12.031 INFO:tasks.workunit.client.0.smithi027.stdout:8/175: mknod d9/de/d1e/c38 0 2024-04-20T10:59:12.031 INFO:tasks.workunit.client.0.smithi027.stdout:3/200: write d2/f6 [1188747,7618] 0 2024-04-20T10:59:12.032 INFO:tasks.workunit.client.0.smithi027.stdout:7/125: dwrite d4/d5/f14 [0,4194304] 0 2024-04-20T10:59:12.032 INFO:tasks.workunit.client.0.smithi027.stdout:4/175: dwrite d0/d14/d17/d1b/d2f/f3f [0,4194304] 0 2024-04-20T10:59:12.032 INFO:tasks.workunit.client.0.smithi027.stdout:1/152: dwrite d3/d7/f8 [0,4194304] 0 2024-04-20T10:59:12.032 INFO:tasks.workunit.client.0.smithi027.stdout:1/153: dread - d3/d20/f3c zero size 2024-04-20T10:59:12.033 INFO:tasks.workunit.client.0.smithi027.stdout:8/176: creat d9/f39 x:0 0 0 2024-04-20T10:59:12.038 INFO:tasks.workunit.client.0.smithi027.stdout:1/154: rmdir d3/d7 39 2024-04-20T10:59:12.038 INFO:tasks.workunit.client.0.smithi027.stdout:3/201: link d2/f1f d2/d18/f3f 0 2024-04-20T10:59:12.038 INFO:tasks.workunit.client.0.smithi027.stdout:1/155: fdatasync d3/d7/d2d/f2f 0 2024-04-20T10:59:12.038 INFO:tasks.workunit.client.0.smithi027.stdout:1/156: write d3/d7/f8 [4574418,236] 0 2024-04-20T10:59:12.038 INFO:tasks.workunit.client.0.smithi027.stdout:1/157: dread - d3/f11 zero size 2024-04-20T10:59:12.041 INFO:tasks.workunit.client.0.smithi027.stdout:1/158: truncate d3/d7/fa 3808360 0 2024-04-20T10:59:12.041 INFO:tasks.workunit.client.0.smithi027.stdout:1/159: read d3/d7/fc [442193,51901] 0 2024-04-20T10:59:12.044 INFO:tasks.workunit.client.0.smithi027.stdout:1/160: dread d3/f1c [0,4194304] 0 2024-04-20T10:59:12.046 INFO:tasks.workunit.client.0.smithi027.stdout:5/112: dwrite fb [0,4194304] 0 2024-04-20T10:59:12.046 INFO:tasks.workunit.client.0.smithi027.stdout:1/161: truncate d3/d20/f33 304550 0 2024-04-20T10:59:12.046 INFO:tasks.workunit.client.0.smithi027.stdout:6/140: dwrite da/df/f10 [0,4194304] 0 2024-04-20T10:59:12.048 INFO:tasks.workunit.client.0.smithi027.stdout:1/162: creat d3/d7/d2d/f3f x:0 0 0 2024-04-20T10:59:12.049 INFO:tasks.workunit.client.0.smithi027.stdout:1/163: fsync d3/f17 0 2024-04-20T10:59:12.050 INFO:tasks.workunit.client.0.smithi027.stdout:6/141: dread da/df/f1a [0,4194304] 0 2024-04-20T10:59:12.051 INFO:tasks.workunit.client.0.smithi027.stdout:6/142: chown da/d21/d28/l31 364811 1 2024-04-20T10:59:12.051 INFO:tasks.workunit.client.0.smithi027.stdout:6/143: stat da/d21/d28/d2d/f2f 0 2024-04-20T10:59:12.051 INFO:tasks.workunit.client.0.smithi027.stdout:6/144: mknod da/dd/c35 0 2024-04-20T10:59:12.052 INFO:tasks.workunit.client.0.smithi027.stdout:6/145: truncate da/df/f17 821988 0 2024-04-20T10:59:12.053 INFO:tasks.workunit.client.0.smithi027.stdout:6/146: unlink da/dd/l1e 0 2024-04-20T10:59:12.053 INFO:tasks.workunit.client.0.smithi027.stdout:3/202: dread d2/f9 [4194304,4194304] 0 2024-04-20T10:59:12.053 INFO:tasks.workunit.client.0.smithi027.stdout:3/203: write d2/d18/f29 [568288,53356] 0 2024-04-20T10:59:12.053 INFO:tasks.workunit.client.0.smithi027.stdout:3/204: write d2/f16 [8746014,92966] 0 2024-04-20T10:59:12.053 INFO:tasks.workunit.client.0.smithi027.stdout:4/176: write d0/d14/d17/d1b/d33/f35 [331297,18437] 0 2024-04-20T10:59:12.053 INFO:tasks.workunit.client.0.smithi027.stdout:4/177: fdatasync d0/d14/d17/f1d 0 2024-04-20T10:59:12.053 INFO:tasks.workunit.client.0.smithi027.stdout:3/205: write d2/d18/f29 [516260,58003] 0 2024-04-20T10:59:12.054 INFO:tasks.workunit.client.0.smithi027.stdout:6/147: creat da/d21/f36 x:0 0 0 2024-04-20T10:59:12.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/148: symlink da/df/l37 0 2024-04-20T10:59:12.059 INFO:tasks.workunit.client.0.smithi027.stdout:1/164: dread d3/d7/f34 [0,4194304] 0 2024-04-20T10:59:12.059 INFO:tasks.workunit.client.0.smithi027.stdout:1/165: chown d3/d7/d2d/f3f 22692 1 2024-04-20T10:59:12.059 INFO:tasks.workunit.client.0.smithi027.stdout:1/166: dread - d3/d20/f3c zero size 2024-04-20T10:59:12.059 INFO:tasks.workunit.client.0.smithi027.stdout:1/167: fdatasync d3/d7/f34 0 2024-04-20T10:59:12.060 INFO:tasks.workunit.client.0.smithi027.stdout:1/168: mknod d3/d7/c40 0 2024-04-20T10:59:12.061 INFO:tasks.workunit.client.0.smithi027.stdout:1/169: truncate d3/d7/fc 111930 0 2024-04-20T10:59:12.061 INFO:tasks.workunit.client.0.smithi027.stdout:1/170: mkdir d3/d41 0 2024-04-20T10:59:12.062 INFO:tasks.workunit.client.0.smithi027.stdout:1/171: rmdir d3/d20 39 2024-04-20T10:59:12.063 INFO:tasks.workunit.client.0.smithi027.stdout:3/206: read d2/dd/f1a [2135323,36670] 0 2024-04-20T10:59:12.063 INFO:tasks.workunit.client.0.smithi027.stdout:3/207: chown d2/f1d 437502 1 2024-04-20T10:59:12.063 INFO:tasks.workunit.client.0.smithi027.stdout:3/208: write d2/fe [471408,36693] 0 2024-04-20T10:59:12.063 INFO:tasks.workunit.client.0.smithi027.stdout:3/209: chown d2/d18 1582 1 2024-04-20T10:59:12.064 INFO:tasks.workunit.client.0.smithi027.stdout:3/210: symlink d2/dd/d22/l40 0 2024-04-20T10:59:12.064 INFO:tasks.workunit.client.0.smithi027.stdout:3/211: stat d2/f1f 0 2024-04-20T10:59:12.065 INFO:tasks.workunit.client.0.smithi027.stdout:3/212: unlink d2/dd/f20 0 2024-04-20T10:59:12.077 INFO:tasks.workunit.client.0.smithi027.stdout:7/126: dwrite d4/d5/f14 [0,4194304] 0 2024-04-20T10:59:12.078 INFO:tasks.workunit.client.0.smithi027.stdout:8/177: dwrite d9/de/d18/d2c/f35 [0,4194304] 0 2024-04-20T10:59:12.078 INFO:tasks.workunit.client.0.smithi027.stdout:5/113: dwrite fc [0,4194304] 0 2024-04-20T10:59:12.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/127: rmdir d4/d5/d8 39 2024-04-20T10:59:12.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/128: write d4/d5/f19 [446016,103515] 0 2024-04-20T10:59:12.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/129: write d4/f17 [322054,105436] 0 2024-04-20T10:59:12.081 INFO:tasks.workunit.client.0.smithi027.stdout:5/114: mknod d19/d1b/d1c/c2a 0 2024-04-20T10:59:12.081 INFO:tasks.workunit.client.0.smithi027.stdout:5/115: write d19/d1b/d1c/d1e/f29 [204665,38948] 0 2024-04-20T10:59:12.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/178: creat d9/de/f3a x:0 0 0 2024-04-20T10:59:12.083 INFO:tasks.workunit.client.0.smithi027.stdout:5/116: rename d19/l1d to d19/d1b/d1c/d1e/d22/l2b 0 2024-04-20T10:59:12.083 INFO:tasks.workunit.client.0.smithi027.stdout:5/117: symlink d19/d1b/d1c/d1e/l2c 0 2024-04-20T10:59:12.086 INFO:tasks.workunit.client.0.smithi027.stdout:6/149: dwrite f4 [0,4194304] 0 2024-04-20T10:59:12.087 INFO:tasks.workunit.client.0.smithi027.stdout:6/150: write da/d21/d28/d2d/f2f [583279,94301] 0 2024-04-20T10:59:12.087 INFO:tasks.workunit.client.0.smithi027.stdout:6/151: chown da/df/f33 2967 1 2024-04-20T10:59:12.089 INFO:tasks.workunit.client.0.smithi027.stdout:8/179: write d9/de/d18/d2c/f35 [2949340,122625] 0 2024-04-20T10:59:12.090 INFO:tasks.workunit.client.0.smithi027.stdout:6/152: link da/df/f1a da/df/d19/d2e/f38 0 2024-04-20T10:59:12.090 INFO:tasks.workunit.client.0.smithi027.stdout:6/153: readlink da/d21/d28/l31 0 2024-04-20T10:59:12.091 INFO:tasks.workunit.client.0.smithi027.stdout:3/213: dwrite d2/d18/f3f [0,4194304] 0 2024-04-20T10:59:12.091 INFO:tasks.workunit.client.0.smithi027.stdout:1/172: dread d3/d7/f8 [0,4194304] 0 2024-04-20T10:59:12.092 INFO:tasks.workunit.client.0.smithi027.stdout:1/173: read d3/d20/f33 [266860,50415] 0 2024-04-20T10:59:12.092 INFO:tasks.workunit.client.0.smithi027.stdout:6/154: mknod da/df/d19/d2e/c39 0 2024-04-20T10:59:12.095 INFO:tasks.workunit.client.0.smithi027.stdout:1/174: creat d3/d7/f42 x:0 0 0 2024-04-20T10:59:12.096 INFO:tasks.workunit.client.0.smithi027.stdout:6/155: creat da/d21/d28/f3a x:0 0 0 2024-04-20T10:59:12.098 INFO:tasks.workunit.client.0.smithi027.stdout:3/214: dread d2/dd/f1a [0,4194304] 0 2024-04-20T10:59:12.102 INFO:tasks.workunit.client.0.smithi027.stdout:6/156: fdatasync da/dd/f25 0 2024-04-20T10:59:12.104 INFO:tasks.workunit.client.0.smithi027.stdout:6/157: write da/df/f17 [231165,130770] 0 2024-04-20T10:59:12.105 INFO:tasks.workunit.client.0.smithi027.stdout:6/158: write da/f1b [972388,111886] 0 2024-04-20T10:59:12.109 INFO:tasks.workunit.client.0.smithi027.stdout:5/118: rename d19/d1b/d1c/d1e/d22/l2b to d19/l2d 0 2024-04-20T10:59:12.109 INFO:tasks.workunit.client.0.smithi027.stdout:3/215: write d2/f17 [7436558,38326] 0 2024-04-20T10:59:12.118 INFO:tasks.workunit.client.0.smithi027.stdout:7/130: dwrite d4/d5/f19 [0,4194304] 0 2024-04-20T10:59:12.123 INFO:tasks.workunit.client.0.smithi027.stdout:1/175: dwrite d3/d7/f1a [0,4194304] 0 2024-04-20T10:59:12.124 INFO:tasks.workunit.client.0.smithi027.stdout:8/180: dwrite f3 [4194304,4194304] 0 2024-04-20T10:59:12.124 INFO:tasks.workunit.client.0.smithi027.stdout:8/181: dread - d9/de/d18/f2d zero size 2024-04-20T10:59:12.124 INFO:tasks.workunit.client.0.smithi027.stdout:1/176: creat d3/d20/f43 x:0 0 0 2024-04-20T10:59:12.124 INFO:tasks.workunit.client.0.smithi027.stdout:8/182: rename d9/de/d1e/l29 to d9/de/l3b 0 2024-04-20T10:59:12.125 INFO:tasks.workunit.client.0.smithi027.stdout:1/177: mknod d3/d7/c44 0 2024-04-20T10:59:12.133 INFO:tasks.workunit.client.0.smithi027.stdout:6/159: chown da/l18 14616505 1 2024-04-20T10:59:12.140 INFO:tasks.workunit.client.0.smithi027.stdout:5/119: dwrite f3 [0,4194304] 0 2024-04-20T10:59:12.144 INFO:tasks.workunit.client.0.smithi027.stdout:3/216: dwrite d2/dd/f19 [0,4194304] 0 2024-04-20T10:59:12.144 INFO:tasks.workunit.client.0.smithi027.stdout:7/131: dwrite d4/d5/fa [0,4194304] 0 2024-04-20T10:59:12.146 INFO:tasks.workunit.client.0.smithi027.stdout:3/217: mkdir d2/d18/d2e/d41 0 2024-04-20T10:59:12.146 INFO:tasks.workunit.client.0.smithi027.stdout:3/218: chown d2/la 20849 1 2024-04-20T10:59:12.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/219: creat d2/dd/df/f42 x:0 0 0 2024-04-20T10:59:12.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/220: chown d2/d18/l31 19 1 2024-04-20T10:59:12.149 INFO:tasks.workunit.client.0.smithi027.stdout:4/178: dread d0/d14/d17/d1b/d33/f35 [0,4194304] 0 2024-04-20T10:59:12.149 INFO:tasks.workunit.client.0.smithi027.stdout:6/160: write da/df/f11 [3730390,82861] 0 2024-04-20T10:59:12.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/179: link d0/d14/d17/f16 d0/d14/f4a 0 2024-04-20T10:59:12.158 INFO:tasks.workunit.client.0.smithi027.stdout:6/161: link da/d21/d28/d2d/f2f da/f3b 0 2024-04-20T10:59:12.160 INFO:tasks.workunit.client.0.smithi027.stdout:9/121: sync 2024-04-20T10:59:12.160 INFO:tasks.workunit.client.0.smithi027.stdout:0/115: sync 2024-04-20T10:59:12.160 INFO:tasks.workunit.client.0.smithi027.stdout:2/76: sync 2024-04-20T10:59:12.160 INFO:tasks.workunit.client.0.smithi027.stdout:9/122: write d4/f1c [8951540,79740] 0 2024-04-20T10:59:12.161 INFO:tasks.workunit.client.0.smithi027.stdout:2/77: unlink d5/c6 0 2024-04-20T10:59:12.162 INFO:tasks.workunit.client.0.smithi027.stdout:0/116: mkdir d0/d19 0 2024-04-20T10:59:12.163 INFO:tasks.workunit.client.0.smithi027.stdout:0/117: read d0/d7/f16 [498729,103066] 0 2024-04-20T10:59:12.163 INFO:tasks.workunit.client.0.smithi027.stdout:9/123: unlink d4/f11 0 2024-04-20T10:59:12.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/118: mkdir d0/d7/d12/d1a 0 2024-04-20T10:59:12.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/119: write d0/d7/fa [4575024,17201] 0 2024-04-20T10:59:12.166 INFO:tasks.workunit.client.0.smithi027.stdout:0/120: mknod d0/d19/c1b 0 2024-04-20T10:59:12.169 INFO:tasks.workunit.client.0.smithi027.stdout:1/178: dwrite d3/d7/f42 [0,4194304] 0 2024-04-20T10:59:12.169 INFO:tasks.workunit.client.0.smithi027.stdout:1/179: stat d3/f17 0 2024-04-20T10:59:12.171 INFO:tasks.workunit.client.0.smithi027.stdout:1/180: unlink d3/d7/d2d/f2f 0 2024-04-20T10:59:12.172 INFO:tasks.workunit.client.0.smithi027.stdout:1/181: mkdir d3/d45 0 2024-04-20T10:59:12.172 INFO:tasks.workunit.client.0.smithi027.stdout:1/182: write d3/d20/f33 [123042,1885] 0 2024-04-20T10:59:12.173 INFO:tasks.workunit.client.0.smithi027.stdout:1/183: creat d3/d20/f46 x:0 0 0 2024-04-20T10:59:12.173 INFO:tasks.workunit.client.0.smithi027.stdout:1/184: getdents d3/d7/d38 0 2024-04-20T10:59:12.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/185: truncate d3/d7/f8 838760 0 2024-04-20T10:59:12.175 INFO:tasks.workunit.client.0.smithi027.stdout:1/186: mkdir d3/d7/d28/d35/d47 0 2024-04-20T10:59:12.177 INFO:tasks.workunit.client.0.smithi027.stdout:7/132: dwrite d4/fc [4194304,4194304] 0 2024-04-20T10:59:12.177 INFO:tasks.workunit.client.0.smithi027.stdout:8/183: dwrite d9/de/d20/f25 [0,4194304] 0 2024-04-20T10:59:12.177 INFO:tasks.workunit.client.0.smithi027.stdout:5/120: dwrite f17 [0,4194304] 0 2024-04-20T10:59:12.177 INFO:tasks.workunit.client.0.smithi027.stdout:5/121: getdents d19/d1b/d1c/d1e/d25 0 2024-04-20T10:59:12.180 INFO:tasks.workunit.client.0.smithi027.stdout:7/133: getdents d4/d1b 0 2024-04-20T10:59:12.180 INFO:tasks.workunit.client.0.smithi027.stdout:7/134: chown d4 1 1 2024-04-20T10:59:12.181 INFO:tasks.workunit.client.0.smithi027.stdout:8/184: mkdir d9/de/d20/d26/d3c 0 2024-04-20T10:59:12.181 INFO:tasks.workunit.client.0.smithi027.stdout:8/185: write d9/de/f1b [1215689,9429] 0 2024-04-20T10:59:12.183 INFO:tasks.workunit.client.0.smithi027.stdout:5/122: truncate fc 6615013 0 2024-04-20T10:59:12.184 INFO:tasks.workunit.client.0.smithi027.stdout:6/162: dwrite da/f2a [0,4194304] 0 2024-04-20T10:59:12.185 INFO:tasks.workunit.client.0.smithi027.stdout:7/135: getdents d4/d5/d8 0 2024-04-20T10:59:12.185 INFO:tasks.workunit.client.0.smithi027.stdout:7/136: truncate d4/d5/d8/f18 637486 0 2024-04-20T10:59:12.186 INFO:tasks.workunit.client.0.smithi027.stdout:3/221: dwrite d2/f16 [4194304,4194304] 0 2024-04-20T10:59:12.187 INFO:tasks.workunit.client.0.smithi027.stdout:8/186: link d9/de/d20/l31 d9/de/d18/d2c/l3d 0 2024-04-20T10:59:12.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/78: dread f1 [0,4194304] 0 2024-04-20T10:59:12.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/79: fdatasync d5/dc/f15 0 2024-04-20T10:59:12.190 INFO:tasks.workunit.client.0.smithi027.stdout:9/124: truncate d4/f7 1094208 0 2024-04-20T10:59:12.191 INFO:tasks.workunit.client.0.smithi027.stdout:0/121: dwrite d0/d7/f16 [0,4194304] 0 2024-04-20T10:59:12.191 INFO:tasks.workunit.client.0.smithi027.stdout:0/122: chown d0/ff 161781 1 2024-04-20T10:59:12.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/187: write d3/d7/f1a [3504728,127514] 0 2024-04-20T10:59:12.191 INFO:tasks.workunit.client.0.smithi027.stdout:6/163: mkdir da/dd/d3c 0 2024-04-20T10:59:12.191 INFO:tasks.workunit.client.0.smithi027.stdout:7/137: unlink d4/c11 0 2024-04-20T10:59:12.192 INFO:tasks.workunit.client.0.smithi027.stdout:5/123: write f2 [3073894,45461] 0 2024-04-20T10:59:12.192 INFO:tasks.workunit.client.0.smithi027.stdout:5/124: readlink l13 0 2024-04-20T10:59:12.193 INFO:tasks.workunit.client.0.smithi027.stdout:8/187: creat d9/de/f3e x:0 0 0 2024-04-20T10:59:12.194 INFO:tasks.workunit.client.0.smithi027.stdout:2/80: creat d5/dc/f19 x:0 0 0 2024-04-20T10:59:12.194 INFO:tasks.workunit.client.0.smithi027.stdout:2/81: write d5/fd [614523,47424] 0 2024-04-20T10:59:12.196 INFO:tasks.workunit.client.0.smithi027.stdout:1/188: rmdir d3/d20 39 2024-04-20T10:59:12.198 INFO:tasks.workunit.client.0.smithi027.stdout:7/138: rename c0 to d4/c1d 0 2024-04-20T10:59:12.198 INFO:tasks.workunit.client.0.smithi027.stdout:7/139: fdatasync d4/d5/f6 0 2024-04-20T10:59:12.199 INFO:tasks.workunit.client.0.smithi027.stdout:6/164: getdents da/d21 0 2024-04-20T10:59:12.199 INFO:tasks.workunit.client.0.smithi027.stdout:6/165: dread - da/dd/f25 zero size 2024-04-20T10:59:12.201 INFO:tasks.workunit.client.0.smithi027.stdout:8/188: link d9/ff d9/de/d20/d26/f3f 0 2024-04-20T10:59:12.202 INFO:tasks.workunit.client.0.smithi027.stdout:2/82: symlink d5/dc/l1a 0 2024-04-20T10:59:12.202 INFO:tasks.workunit.client.0.smithi027.stdout:2/83: chown d5/dc/f19 24 1 2024-04-20T10:59:12.203 INFO:tasks.workunit.client.0.smithi027.stdout:1/189: creat d3/d7/d28/f48 x:0 0 0 2024-04-20T10:59:12.204 INFO:tasks.workunit.client.0.smithi027.stdout:5/125: rename l11 to d19/d1b/d1c/l2e 0 2024-04-20T10:59:12.205 INFO:tasks.workunit.client.0.smithi027.stdout:7/140: mknod d4/d1b/c1e 0 2024-04-20T10:59:12.206 INFO:tasks.workunit.client.0.smithi027.stdout:8/189: unlink d9/l17 0 2024-04-20T10:59:12.207 INFO:tasks.workunit.client.0.smithi027.stdout:1/190: stat d3/d7/c36 0 2024-04-20T10:59:12.207 INFO:tasks.workunit.client.0.smithi027.stdout:1/191: getdents d3/d7/d28/d35/d47 0 2024-04-20T10:59:12.207 INFO:tasks.workunit.client.0.smithi027.stdout:1/192: write d3/d7/d28/f3d [355173,116895] 0 2024-04-20T10:59:12.210 INFO:tasks.workunit.client.0.smithi027.stdout:8/190: rename d9/de/d18/d2c/l2e to d9/de/d18/d2c/l40 0 2024-04-20T10:59:12.212 INFO:tasks.workunit.client.0.smithi027.stdout:1/193: creat d3/d41/f49 x:0 0 0 2024-04-20T10:59:12.218 INFO:tasks.workunit.client.0.smithi027.stdout:9/125: dwrite d4/d14/f19 [0,4194304] 0 2024-04-20T10:59:12.218 INFO:tasks.workunit.client.0.smithi027.stdout:0/123: dwrite d0/d7/fa [0,4194304] 0 2024-04-20T10:59:12.220 INFO:tasks.workunit.client.0.smithi027.stdout:9/126: mkdir d4/d1b/d1f/d20/d23 0 2024-04-20T10:59:12.220 INFO:tasks.workunit.client.0.smithi027.stdout:1/194: write d3/d7/f34 [3298136,104321] 0 2024-04-20T10:59:12.221 INFO:tasks.workunit.client.0.smithi027.stdout:9/127: getdents d4/d1b/d1f 0 2024-04-20T10:59:12.222 INFO:tasks.workunit.client.0.smithi027.stdout:1/195: creat d3/d7/d2d/f4a x:0 0 0 2024-04-20T10:59:12.222 INFO:tasks.workunit.client.0.smithi027.stdout:1/196: stat d3/d7/l14 0 2024-04-20T10:59:12.222 INFO:tasks.workunit.client.0.smithi027.stdout:1/197: write d3/d41/f49 [102498,82665] 0 2024-04-20T10:59:12.223 INFO:tasks.workunit.client.0.smithi027.stdout:9/128: creat d4/d1b/f24 x:0 0 0 2024-04-20T10:59:12.223 INFO:tasks.workunit.client.0.smithi027.stdout:8/191: write d9/de/d20/f25 [1638191,110964] 0 2024-04-20T10:59:12.224 INFO:tasks.workunit.client.0.smithi027.stdout:9/129: unlink d4/c12 0 2024-04-20T10:59:12.224 INFO:tasks.workunit.client.0.smithi027.stdout:9/130: write d4/f7 [2048597,19156] 0 2024-04-20T10:59:12.225 INFO:tasks.workunit.client.0.smithi027.stdout:8/192: symlink d9/de/d20/d26/l41 0 2024-04-20T10:59:12.226 INFO:tasks.workunit.client.0.smithi027.stdout:8/193: symlink d9/de/d1e/l42 0 2024-04-20T10:59:12.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/131: mknod d4/d14/c25 0 2024-04-20T10:59:12.232 INFO:tasks.workunit.client.0.smithi027.stdout:7/141: dwrite d4/d5/f19 [0,4194304] 0 2024-04-20T10:59:12.234 INFO:tasks.workunit.client.0.smithi027.stdout:2/84: dwrite d5/dc/f15 [8388608,4194304] 0 2024-04-20T10:59:12.234 INFO:tasks.workunit.client.0.smithi027.stdout:0/124: dread d0/d7/fa [0,4194304] 0 2024-04-20T10:59:12.234 INFO:tasks.workunit.client.0.smithi027.stdout:0/125: write d0/ff [4259051,46141] 0 2024-04-20T10:59:12.234 INFO:tasks.workunit.client.0.smithi027.stdout:0/126: fdatasync d0/d7/fa 0 2024-04-20T10:59:12.239 INFO:tasks.workunit.client.0.smithi027.stdout:7/142: unlink d4/d5/d8/f18 0 2024-04-20T10:59:12.239 INFO:tasks.workunit.client.0.smithi027.stdout:0/127: creat d0/d7/f1c x:0 0 0 2024-04-20T10:59:12.241 INFO:tasks.workunit.client.0.smithi027.stdout:7/143: link d4/d5/f6 d4/d5/f1f 0 2024-04-20T10:59:12.241 INFO:tasks.workunit.client.0.smithi027.stdout:7/144: readlink l3 0 2024-04-20T10:59:12.242 INFO:tasks.workunit.client.0.smithi027.stdout:0/128: creat d0/d19/f1d x:0 0 0 2024-04-20T10:59:12.242 INFO:tasks.workunit.client.0.smithi027.stdout:0/129: chown d0/d19/c1b 1833 1 2024-04-20T10:59:12.242 INFO:tasks.workunit.client.0.smithi027.stdout:0/130: stat d0/ff 0 2024-04-20T10:59:12.242 INFO:tasks.workunit.client.0.smithi027.stdout:0/131: write d0/d19/f1d [870433,118400] 0 2024-04-20T10:59:12.243 INFO:tasks.workunit.client.0.smithi027.stdout:7/145: link d4/d1b/ff d4/d1b/f20 0 2024-04-20T10:59:12.244 INFO:tasks.workunit.client.0.smithi027.stdout:0/132: unlink d0/d19/c1b 0 2024-04-20T10:59:12.247 INFO:tasks.workunit.client.0.smithi027.stdout:5/126: dwrite f2 [0,4194304] 0 2024-04-20T10:59:12.247 INFO:tasks.workunit.client.0.smithi027.stdout:5/127: write f18 [554632,59785] 0 2024-04-20T10:59:12.247 INFO:tasks.workunit.client.0.smithi027.stdout:5/128: readlink d19/d1b/d1c/d1e/l2c 0 2024-04-20T10:59:12.248 INFO:tasks.workunit.client.0.smithi027.stdout:5/129: creat d19/d1b/d1c/d1e/d25/f2f x:0 0 0 2024-04-20T10:59:12.249 INFO:tasks.workunit.client.0.smithi027.stdout:5/130: read d19/d1b/f1f [186036,23937] 0 2024-04-20T10:59:12.249 INFO:tasks.workunit.client.0.smithi027.stdout:5/131: chown f0 0 1 2024-04-20T10:59:12.249 INFO:tasks.workunit.client.0.smithi027.stdout:5/132: creat d19/d1b/d1c/d1e/d22/f30 x:0 0 0 2024-04-20T10:59:12.250 INFO:tasks.workunit.client.0.smithi027.stdout:5/133: write d19/d1b/d1c/d1e/d22/f30 [1030274,75543] 0 2024-04-20T10:59:12.250 INFO:tasks.workunit.client.0.smithi027.stdout:1/198: dwrite d3/d7/f42 [4194304,4194304] 0 2024-04-20T10:59:12.251 INFO:tasks.workunit.client.0.smithi027.stdout:1/199: getdents d3/d7/d28/d35/d47 0 2024-04-20T10:59:12.251 INFO:tasks.workunit.client.0.smithi027.stdout:2/85: dwrite d5/fd [0,4194304] 0 2024-04-20T10:59:12.252 INFO:tasks.workunit.client.0.smithi027.stdout:1/200: symlink d3/d7/d2d/l4b 0 2024-04-20T10:59:12.252 INFO:tasks.workunit.client.0.smithi027.stdout:1/201: fsync d3/d7/d28/f48 0 2024-04-20T10:59:12.252 INFO:tasks.workunit.client.0.smithi027.stdout:8/194: dwrite d9/de/d20/d26/f3f [0,4194304] 0 2024-04-20T10:59:12.254 INFO:tasks.workunit.client.0.smithi027.stdout:5/134: dread f18 [0,4194304] 0 2024-04-20T10:59:12.254 INFO:tasks.workunit.client.0.smithi027.stdout:8/195: link d9/de/d20/d26/l41 d9/de/d18/d2c/l43 0 2024-04-20T10:59:12.256 INFO:tasks.workunit.client.0.smithi027.stdout:7/146: dread d4/fc [4194304,4194304] 0 2024-04-20T10:59:12.257 INFO:tasks.workunit.client.0.smithi027.stdout:8/196: creat d9/de/d20/d26/d3c/f44 x:0 0 0 2024-04-20T10:59:12.258 INFO:tasks.workunit.client.0.smithi027.stdout:8/197: creat d9/de/d18/d2c/f45 x:0 0 0 2024-04-20T10:59:12.259 INFO:tasks.workunit.client.0.smithi027.stdout:8/198: mknod d9/c46 0 2024-04-20T10:59:12.261 INFO:tasks.workunit.client.0.smithi027.stdout:0/133: write d0/f15 [2665290,102449] 0 2024-04-20T10:59:12.262 INFO:tasks.workunit.client.0.smithi027.stdout:1/202: dread d3/d7/f1a [4194304,4194304] 0 2024-04-20T10:59:12.262 INFO:tasks.workunit.client.0.smithi027.stdout:1/203: dread - d3/d7/d28/f48 zero size 2024-04-20T10:59:12.266 INFO:tasks.workunit.client.0.smithi027.stdout:5/135: dread f17 [0,4194304] 0 2024-04-20T10:59:12.267 INFO:tasks.workunit.client.0.smithi027.stdout:8/199: write d9/de/f37 [3662521,18453] 0 2024-04-20T10:59:12.267 INFO:tasks.workunit.client.0.smithi027.stdout:8/200: write d9/fb [943270,68900] 0 2024-04-20T10:59:12.271 INFO:tasks.workunit.client.0.smithi027.stdout:8/201: rename d9/de/d20/f28 to d9/de/f47 0 2024-04-20T10:59:12.272 INFO:tasks.workunit.client.0.smithi027.stdout:8/202: mkdir d9/de/d20/d48 0 2024-04-20T10:59:12.274 INFO:tasks.workunit.client.0.smithi027.stdout:7/147: dwrite d4/d5/f19 [0,4194304] 0 2024-04-20T10:59:12.274 INFO:tasks.workunit.client.0.smithi027.stdout:8/203: mkdir d9/de/d1e/d49 0 2024-04-20T10:59:12.275 INFO:tasks.workunit.client.0.smithi027.stdout:8/204: dread d9/de/f1d [0,4194304] 0 2024-04-20T10:59:12.277 INFO:tasks.workunit.client.0.smithi027.stdout:8/205: dread d9/de/f2f [0,4194304] 0 2024-04-20T10:59:12.277 INFO:tasks.workunit.client.0.smithi027.stdout:7/148: mkdir d4/d5/d8/d21 0 2024-04-20T10:59:12.277 INFO:tasks.workunit.client.0.smithi027.stdout:7/149: chown d4/d5/c9 16694642 1 2024-04-20T10:59:12.277 INFO:tasks.workunit.client.0.smithi027.stdout:8/206: write d9/de/d18/f22 [863244,74329] 0 2024-04-20T10:59:12.277 INFO:tasks.workunit.client.0.smithi027.stdout:9/132: truncate d4/f7 1628386 0 2024-04-20T10:59:12.278 INFO:tasks.workunit.client.0.smithi027.stdout:7/150: mknod d4/d5/d8/d21/c22 0 2024-04-20T10:59:12.278 INFO:tasks.workunit.client.0.smithi027.stdout:9/133: symlink d4/d1b/d1f/l26 0 2024-04-20T10:59:12.284 INFO:tasks.workunit.client.0.smithi027.stdout:1/204: dwrite d3/d20/f3c [0,4194304] 0 2024-04-20T10:59:12.286 INFO:tasks.workunit.client.0.smithi027.stdout:1/205: dread d3/d7/fc [0,4194304] 0 2024-04-20T10:59:12.288 INFO:tasks.workunit.client.0.smithi027.stdout:5/136: dwrite d19/d1b/d1c/d1e/f29 [0,4194304] 0 2024-04-20T10:59:12.290 INFO:tasks.workunit.client.0.smithi027.stdout:1/206: getdents d3/d7/d2d 0 2024-04-20T10:59:12.290 INFO:tasks.workunit.client.0.smithi027.stdout:1/207: stat d3/d7/d2d/f4a 0 2024-04-20T10:59:12.293 INFO:tasks.workunit.client.0.smithi027.stdout:5/137: link d19/f27 d19/d1b/d20/f31 0 2024-04-20T10:59:12.294 INFO:tasks.workunit.client.0.smithi027.stdout:1/208: link d3/c22 d3/d7/d28/c4c 0 2024-04-20T10:59:12.294 INFO:tasks.workunit.client.0.smithi027.stdout:1/209: write d3/f1e [896657,129492] 0 2024-04-20T10:59:12.299 INFO:tasks.workunit.client.0.smithi027.stdout:1/210: rmdir d3/d7/d28/d35/d47 0 2024-04-20T10:59:12.300 INFO:tasks.workunit.client.0.smithi027.stdout:8/207: dwrite d9/de/d18/d2c/f35 [0,4194304] 0 2024-04-20T10:59:12.300 INFO:tasks.workunit.client.0.smithi027.stdout:8/208: write d9/f24 [841146,3636] 0 2024-04-20T10:59:12.300 INFO:tasks.workunit.client.0.smithi027.stdout:1/211: dread d3/d7/fc [0,4194304] 0 2024-04-20T10:59:12.302 INFO:tasks.workunit.client.0.smithi027.stdout:4/180: sync 2024-04-20T10:59:12.302 INFO:tasks.workunit.client.0.smithi027.stdout:8/209: unlink d9/c19 0 2024-04-20T10:59:12.303 INFO:tasks.workunit.client.0.smithi027.stdout:4/181: creat d0/d14/d17/d18/f4b x:0 0 0 2024-04-20T10:59:12.304 INFO:tasks.workunit.client.0.smithi027.stdout:8/210: rename d9/de/d18/d2c to d9/de/d1e/d4a 0 2024-04-20T10:59:12.305 INFO:tasks.workunit.client.0.smithi027.stdout:4/182: rename d0/fc to d0/d14/d17/d1b/f4c 0 2024-04-20T10:59:12.306 INFO:tasks.workunit.client.0.smithi027.stdout:8/211: mkdir d9/de/d20/d26/d3c/d4b 0 2024-04-20T10:59:12.306 INFO:tasks.workunit.client.0.smithi027.stdout:4/183: rmdir d0/d14/d17/d18 39 2024-04-20T10:59:12.307 INFO:tasks.workunit.client.0.smithi027.stdout:8/212: creat d9/de/d1e/d49/f4c x:0 0 0 2024-04-20T10:59:12.307 INFO:tasks.workunit.client.0.smithi027.stdout:8/213: readlink d9/de/d20/d26/l41 0 2024-04-20T10:59:12.308 INFO:tasks.workunit.client.0.smithi027.stdout:4/184: symlink d0/d14/d17/d1a/l4d 0 2024-04-20T10:59:12.308 INFO:tasks.workunit.client.0.smithi027.stdout:8/214: dread d9/de/d1e/f1f [0,4194304] 0 2024-04-20T10:59:12.310 INFO:tasks.workunit.client.0.smithi027.stdout:1/212: dread d3/d7/f34 [0,4194304] 0 2024-04-20T10:59:12.310 INFO:tasks.workunit.client.0.smithi027.stdout:8/215: rename d9/c13 to d9/de/c4d 0 2024-04-20T10:59:12.310 INFO:tasks.workunit.client.0.smithi027.stdout:8/216: read d9/de/f37 [396378,97610] 0 2024-04-20T10:59:12.310 INFO:tasks.workunit.client.0.smithi027.stdout:8/217: readlink d9/lc 0 2024-04-20T10:59:12.311 INFO:tasks.workunit.client.0.smithi027.stdout:7/151: dwrite d4/d5/fa [0,4194304] 0 2024-04-20T10:59:12.311 INFO:tasks.workunit.client.0.smithi027.stdout:7/152: readlink d4/le 0 2024-04-20T10:59:12.311 INFO:tasks.workunit.client.0.smithi027.stdout:9/134: dwrite d4/fc [0,4194304] 0 2024-04-20T10:59:12.311 INFO:tasks.workunit.client.0.smithi027.stdout:1/213: unlink d3/d7/f1a 0 2024-04-20T10:59:12.312 INFO:tasks.workunit.client.0.smithi027.stdout:5/138: dwrite f0 [4194304,4194304] 0 2024-04-20T10:59:12.313 INFO:tasks.workunit.client.0.smithi027.stdout:5/139: read f18 [577567,62863] 0 2024-04-20T10:59:12.313 INFO:tasks.workunit.client.0.smithi027.stdout:5/140: stat cf 0 2024-04-20T10:59:12.313 INFO:tasks.workunit.client.0.smithi027.stdout:8/218: rename d9/de/d20/f27 to d9/de/d1e/d4a/f4e 0 2024-04-20T10:59:12.318 INFO:tasks.workunit.client.0.smithi027.stdout:1/214: creat d3/d20/f4d x:0 0 0 2024-04-20T10:59:12.318 INFO:tasks.workunit.client.0.smithi027.stdout:1/215: fsync d3/d7/d2d/f3f 0 2024-04-20T10:59:12.320 INFO:tasks.workunit.client.0.smithi027.stdout:7/153: write d4/d1b/ff [2062649,66528] 0 2024-04-20T10:59:12.320 INFO:tasks.workunit.client.0.smithi027.stdout:8/219: truncate d9/de/f1b 383383 0 2024-04-20T10:59:12.321 INFO:tasks.workunit.client.0.smithi027.stdout:8/220: write f2 [2706280,7431] 0 2024-04-20T10:59:12.321 INFO:tasks.workunit.client.0.smithi027.stdout:1/216: unlink d3/d20/f43 0 2024-04-20T10:59:12.322 INFO:tasks.workunit.client.0.smithi027.stdout:5/141: rename d19/d1b/d1c/d1e/f29 to d19/d1b/d1c/d1e/d22/f32 0 2024-04-20T10:59:12.322 INFO:tasks.workunit.client.0.smithi027.stdout:5/142: chown d19/d1b/d1c/d1e/d25 3 1 2024-04-20T10:59:12.323 INFO:tasks.workunit.client.0.smithi027.stdout:7/154: stat d4/c1d 0 2024-04-20T10:59:12.323 INFO:tasks.workunit.client.0.smithi027.stdout:8/221: rmdir d9 39 2024-04-20T10:59:12.323 INFO:tasks.workunit.client.0.smithi027.stdout:1/217: rmdir d3 39 2024-04-20T10:59:12.324 INFO:tasks.workunit.client.0.smithi027.stdout:7/155: rename d4/d5/d8/c1a to d4/d1b/c23 0 2024-04-20T10:59:12.325 INFO:tasks.workunit.client.0.smithi027.stdout:1/218: creat d3/d41/f4e x:0 0 0 2024-04-20T10:59:12.325 INFO:tasks.workunit.client.0.smithi027.stdout:7/156: getdents d4/d5/d8/d21 0 2024-04-20T10:59:12.326 INFO:tasks.workunit.client.0.smithi027.stdout:1/219: rename d3/d7/d28/d35/l39 to d3/d7/d2d/l4f 0 2024-04-20T10:59:12.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/220: fdatasync d3/d7/f8 0 2024-04-20T10:59:12.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/221: chown d3/d7/d2d/c3b 1660 1 2024-04-20T10:59:12.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/222: symlink d3/d7/d38/l50 0 2024-04-20T10:59:12.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/223: chown d3/d7/d2d/f3f 53016 1 2024-04-20T10:59:12.327 INFO:tasks.workunit.client.0.smithi027.stdout:1/224: rmdir d3/d20 39 2024-04-20T10:59:12.328 INFO:tasks.workunit.client.0.smithi027.stdout:1/225: mknod d3/d45/c51 0 2024-04-20T10:59:12.330 INFO:tasks.workunit.client.0.smithi027.stdout:4/185: dwrite d0/fd [0,4194304] 0 2024-04-20T10:59:12.331 INFO:tasks.workunit.client.0.smithi027.stdout:1/226: write d3/f1e [489136,85135] 0 2024-04-20T10:59:12.332 INFO:tasks.workunit.client.0.smithi027.stdout:4/186: symlink d0/d14/d17/d1b/l4e 0 2024-04-20T10:59:12.333 INFO:tasks.workunit.client.0.smithi027.stdout:1/227: creat d3/d7/d28/d35/f52 x:0 0 0 2024-04-20T10:59:12.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/135: dwrite d4/f1c [0,4194304] 0 2024-04-20T10:59:12.334 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:59:12.335 INFO:tasks.workunit.client.0.smithi027.stdout:4/187: symlink d0/d48/l4f 0 2024-04-20T10:59:12.335 INFO:tasks.workunit.client.0.smithi027.stdout:4/188: write d0/d14/d17/d18/f38 [346943,789] 0 2024-04-20T10:59:12.335 INFO:tasks.workunit.client.0.smithi027.stdout:1/228: truncate d3/d7/f34 323619 0 2024-04-20T10:59:12.337 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:59:12.344 INFO:tasks.workunit.client.0.smithi027.stdout:9/136: read d4/fc [3349150,42309] 0 2024-04-20T10:59:12.344 INFO:tasks.workunit.client.0.smithi027.stdout:5/143: dwrite fc [0,4194304] 0 2024-04-20T10:59:12.351 INFO:tasks.workunit.client.0.smithi027.stdout:4/189: write d0/d14/d17/d1b/d2f/f3f [1933411,66595] 0 2024-04-20T10:59:12.352 INFO:tasks.workunit.client.0.smithi027.stdout:7/157: dwrite d4/d1b/ff [0,4194304] 0 2024-04-20T10:59:12.359 INFO:tasks.workunit.client.0.smithi027.stdout:8/222: dwrite d9/de/f47 [0,4194304] 0 2024-04-20T10:59:12.593 INFO:tasks.workunit.client.0.smithi027.stdout:1/229: creat d3/d7/d2d/f53 x:0 0 0 2024-04-20T10:59:12.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/137: creat d4/d14/f27 x:0 0 0 2024-04-20T10:59:12.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/144: rename d19/d1b/f1f to d19/d1b/f33 0 2024-04-20T10:59:12.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/145: write f17 [4400475,81549] 0 2024-04-20T10:59:12.597 INFO:tasks.workunit.client.0.smithi027.stdout:4/190: unlink d0/f9 0 2024-04-20T10:59:12.598 INFO:tasks.workunit.client.0.smithi027.stdout:7/158: creat d4/d5/f24 x:0 0 0 2024-04-20T10:59:12.598 INFO:tasks.workunit.client.0.smithi027.stdout:7/159: truncate d4/d5/fa 4408169 0 2024-04-20T10:59:12.600 INFO:tasks.workunit.client.0.smithi027.stdout:8/223: mkdir d9/de/d20/d4f 0 2024-04-20T10:59:12.600 INFO:tasks.workunit.client.0.smithi027.stdout:1/230: mknod d3/d7/d38/c54 0 2024-04-20T10:59:12.602 INFO:tasks.workunit.client.0.smithi027.stdout:5/146: write f17 [3927793,123262] 0 2024-04-20T10:59:12.602 INFO:tasks.workunit.client.0.smithi027.stdout:4/191: symlink d0/d14/d17/d1b/d2f/l50 0 2024-04-20T10:59:12.603 INFO:tasks.workunit.client.0.smithi027.stdout:7/160: mknod d4/d5/d8/c25 0 2024-04-20T10:59:12.604 INFO:tasks.workunit.client.0.smithi027.stdout:1/231: creat d3/d7/f55 x:0 0 0 2024-04-20T10:59:12.604 INFO:tasks.workunit.client.0.smithi027.stdout:1/232: write d3/d20/f46 [148832,61742] 0 2024-04-20T10:59:12.605 INFO:tasks.workunit.client.0.smithi027.stdout:5/147: creat d19/d1b/d1c/d1e/d25/f34 x:0 0 0 2024-04-20T10:59:12.606 INFO:tasks.workunit.client.0.smithi027.stdout:4/192: getdents d0/d48 0 2024-04-20T10:59:12.606 INFO:tasks.workunit.client.0.smithi027.stdout:4/193: chown d0/d14/d17/l25 7903 1 2024-04-20T10:59:12.607 INFO:tasks.workunit.client.0.smithi027.stdout:1/233: rename d3/d41/f49 to d3/d7/d38/f56 0 2024-04-20T10:59:12.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/194: mknod d0/d14/d17/d1b/d33/c51 0 2024-04-20T10:59:12.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/195: readlink d0/d14/d17/d1b/d2f/l49 0 2024-04-20T10:59:12.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/196: getdents d0/d48 0 2024-04-20T10:59:12.609 INFO:tasks.workunit.client.0.smithi027.stdout:1/234: creat d3/d7/f57 x:0 0 0 2024-04-20T10:59:12.610 INFO:tasks.workunit.client.0.smithi027.stdout:4/197: mknod d0/d14/d17/c52 0 2024-04-20T10:59:12.610 INFO:tasks.workunit.client.0.smithi027.stdout:1/235: mknod d3/d20/c58 0 2024-04-20T10:59:12.612 INFO:tasks.workunit.client.0.smithi027.stdout:4/198: mkdir d0/d14/d17/d53 0 2024-04-20T10:59:12.613 INFO:tasks.workunit.client.0.smithi027.stdout:1/236: creat d3/d7/d28/f59 x:0 0 0 2024-04-20T10:59:12.613 INFO:tasks.workunit.client.0.smithi027.stdout:4/199: creat d0/d14/d17/d18/f54 x:0 0 0 2024-04-20T10:59:12.613 INFO:tasks.workunit.client.0.smithi027.stdout:5/148: read fc [1061136,35950] 0 2024-04-20T10:59:12.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/237: creat d3/d7/d2d/f5a x:0 0 0 2024-04-20T10:59:12.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/238: chown d3/d20/c27 0 1 2024-04-20T10:59:12.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/239: dread - d3/d7/f23 zero size 2024-04-20T10:59:12.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/240: read - d3/d7/d2d/f4a zero size 2024-04-20T10:59:12.615 INFO:tasks.workunit.client.0.smithi027.stdout:5/149: symlink d19/d1b/d20/l35 0 2024-04-20T10:59:12.615 INFO:tasks.workunit.client.0.smithi027.stdout:1/241: mknod d3/d20/c5b 0 2024-04-20T10:59:12.615 INFO:tasks.workunit.client.0.smithi027.stdout:1/242: write d3/d7/d28/f59 [182115,115585] 0 2024-04-20T10:59:12.615 INFO:tasks.workunit.client.0.smithi027.stdout:1/243: truncate d3/d7/d2d/f53 101062 0 2024-04-20T10:59:12.616 INFO:tasks.workunit.client.0.smithi027.stdout:5/150: link d19/l2d d19/d1b/d1c/d1e/l36 0 2024-04-20T10:59:12.616 INFO:tasks.workunit.client.0.smithi027.stdout:1/244: creat d3/d7/d28/f5c x:0 0 0 2024-04-20T10:59:12.616 INFO:tasks.workunit.client.0.smithi027.stdout:1/245: fsync d3/d7/f34 0 2024-04-20T10:59:12.617 INFO:tasks.workunit.client.0.smithi027.stdout:5/151: symlink d19/d1b/d1c/d1e/d25/l37 0 2024-04-20T10:59:12.617 INFO:tasks.workunit.client.0.smithi027.stdout:5/152: write fc [2388145,6674] 0 2024-04-20T10:59:12.617 INFO:tasks.workunit.client.0.smithi027.stdout:5/153: write f16 [631073,78937] 0 2024-04-20T10:59:12.617 INFO:tasks.workunit.client.0.smithi027.stdout:1/246: rmdir d3/d7 39 2024-04-20T10:59:12.618 INFO:tasks.workunit.client.0.smithi027.stdout:1/247: dread d3/d7/fc [0,4194304] 0 2024-04-20T10:59:12.623 INFO:tasks.workunit.client.0.smithi027.stdout:8/224: dwrite d9/f24 [0,4194304] 0 2024-04-20T10:59:12.625 INFO:tasks.workunit.client.0.smithi027.stdout:5/154: getdents d19/d1b/d1c/d1e 0 2024-04-20T10:59:12.626 INFO:tasks.workunit.client.0.smithi027.stdout:6/166: sync 2024-04-20T10:59:12.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/222: sync 2024-04-20T10:59:12.626 INFO:tasks.workunit.client.0.smithi027.stdout:2/86: sync 2024-04-20T10:59:12.626 INFO:tasks.workunit.client.0.smithi027.stdout:0/134: sync 2024-04-20T10:59:12.626 INFO:tasks.workunit.client.0.smithi027.stdout:2/87: write f3 [2381620,23104] 0 2024-04-20T10:59:12.626 INFO:tasks.workunit.client.0.smithi027.stdout:2/88: chown d5/dc/d16/c14 28900860 1 2024-04-20T10:59:12.626 INFO:tasks.workunit.client.0.smithi027.stdout:2/89: rename d5/dc to d5/dc/d16/d1b 22 2024-04-20T10:59:12.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/248: write d3/d7/fa [744797,69917] 0 2024-04-20T10:59:12.627 INFO:tasks.workunit.client.0.smithi027.stdout:8/225: creat d9/de/f50 x:0 0 0 2024-04-20T10:59:12.628 INFO:tasks.workunit.client.0.smithi027.stdout:6/167: readlink da/l20 0 2024-04-20T10:59:12.628 INFO:tasks.workunit.client.0.smithi027.stdout:6/168: stat da/l20 0 2024-04-20T10:59:12.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/249: dread d3/d7/d38/f56 [0,4194304] 0 2024-04-20T10:59:12.630 INFO:tasks.workunit.client.0.smithi027.stdout:3/223: symlink d2/dd/l43 0 2024-04-20T10:59:12.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/250: chown d3/d7/le 185983 1 2024-04-20T10:59:12.630 INFO:tasks.workunit.client.0.smithi027.stdout:3/224: fdatasync d2/dd/f3d 0 2024-04-20T10:59:12.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/251: chown d3/d20/c3e 0 1 2024-04-20T10:59:12.631 INFO:tasks.workunit.client.0.smithi027.stdout:2/90: link d5/dc/fe d5/dc/f1c 0 2024-04-20T10:59:12.631 INFO:tasks.workunit.client.0.smithi027.stdout:0/135: dread d0/ff [0,4194304] 0 2024-04-20T10:59:12.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/169: mkdir da/dd/d3d 0 2024-04-20T10:59:12.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/170: readlink da/df/l37 0 2024-04-20T10:59:12.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/171: fsync da/dd/f25 0 2024-04-20T10:59:12.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/172: dread - da/d21/f36 zero size 2024-04-20T10:59:12.635 INFO:tasks.workunit.client.0.smithi027.stdout:8/226: write d9/de/d18/f21 [2212788,26756] 0 2024-04-20T10:59:12.636 INFO:tasks.workunit.client.0.smithi027.stdout:3/225: unlink d2/f1e 0 2024-04-20T10:59:12.636 INFO:tasks.workunit.client.0.smithi027.stdout:1/252: mkdir d3/d45/d5d 0 2024-04-20T10:59:12.636 INFO:tasks.workunit.client.0.smithi027.stdout:0/136: getdents d0/d7/d12 0 2024-04-20T10:59:12.637 INFO:tasks.workunit.client.0.smithi027.stdout:6/173: creat da/d21/d28/f3e x:0 0 0 2024-04-20T10:59:12.638 INFO:tasks.workunit.client.0.smithi027.stdout:0/137: mknod d0/d19/c1e 0 2024-04-20T10:59:12.638 INFO:tasks.workunit.client.0.smithi027.stdout:1/253: dread d3/d7/f8 [0,4194304] 0 2024-04-20T10:59:12.639 INFO:tasks.workunit.client.0.smithi027.stdout:6/174: creat da/df/d1f/f3f x:0 0 0 2024-04-20T10:59:12.639 INFO:tasks.workunit.client.0.smithi027.stdout:3/226: dread d2/f5 [0,4194304] 0 2024-04-20T10:59:12.641 INFO:tasks.workunit.client.0.smithi027.stdout:2/91: dread d5/dc/f15 [4194304,4194304] 0 2024-04-20T10:59:12.641 INFO:tasks.workunit.client.0.smithi027.stdout:0/138: creat d0/f1f x:0 0 0 2024-04-20T10:59:12.641 INFO:tasks.workunit.client.0.smithi027.stdout:1/254: creat d3/d7/d28/d35/f5e x:0 0 0 2024-04-20T10:59:12.642 INFO:tasks.workunit.client.0.smithi027.stdout:3/227: mknod d2/d18/d2c/c44 0 2024-04-20T10:59:12.642 INFO:tasks.workunit.client.0.smithi027.stdout:3/228: truncate d2/dd/d22/f2f 373601 0 2024-04-20T10:59:12.644 INFO:tasks.workunit.client.0.smithi027.stdout:2/92: rename d5/dc/l1a to d5/dc/l1d 0 2024-04-20T10:59:12.644 INFO:tasks.workunit.client.0.smithi027.stdout:8/227: write d9/f24 [4099550,16416] 0 2024-04-20T10:59:12.644 INFO:tasks.workunit.client.0.smithi027.stdout:8/228: truncate d9/f39 182813 0 2024-04-20T10:59:12.645 INFO:tasks.workunit.client.0.smithi027.stdout:2/93: getdents d5 0 2024-04-20T10:59:12.647 INFO:tasks.workunit.client.0.smithi027.stdout:8/229: symlink d9/de/d20/d26/d3c/d4b/l51 0 2024-04-20T10:59:12.647 INFO:tasks.workunit.client.0.smithi027.stdout:8/230: chown d9/de/f1d 3260 1 2024-04-20T10:59:12.650 INFO:tasks.workunit.client.0.smithi027.stdout:2/94: unlink d5/dc/l10 0 2024-04-20T10:59:12.650 INFO:tasks.workunit.client.0.smithi027.stdout:2/95: fsync d5/f17 0 2024-04-20T10:59:12.650 INFO:tasks.workunit.client.0.smithi027.stdout:8/231: rmdir d9/de/d18 39 2024-04-20T10:59:12.651 INFO:tasks.workunit.client.0.smithi027.stdout:2/96: mkdir d5/d1e 0 2024-04-20T10:59:12.652 INFO:tasks.workunit.client.0.smithi027.stdout:8/232: mknod d9/c52 0 2024-04-20T10:59:12.652 INFO:tasks.workunit.client.0.smithi027.stdout:8/233: write d9/de/d20/d26/d3c/f44 [837827,54125] 0 2024-04-20T10:59:12.653 INFO:tasks.workunit.client.0.smithi027.stdout:8/234: fdatasync d9/de/f1d 0 2024-04-20T10:59:12.653 INFO:tasks.workunit.client.0.smithi027.stdout:8/235: fdatasync d9/de/d1e/d49/f4c 0 2024-04-20T10:59:12.653 INFO:tasks.workunit.client.0.smithi027.stdout:2/97: rmdir d5/dc 39 2024-04-20T10:59:12.658 INFO:tasks.workunit.client.0.smithi027.stdout:2/98: mknod d5/dc/c1f 0 2024-04-20T10:59:12.658 INFO:tasks.workunit.client.0.smithi027.stdout:2/99: truncate d5/f17 59460 0 2024-04-20T10:59:12.662 INFO:tasks.workunit.client.0.smithi027.stdout:2/100: symlink d5/dc/l20 0 2024-04-20T10:59:12.663 INFO:tasks.workunit.client.0.smithi027.stdout:0/139: dread d0/d7/f16 [0,4194304] 0 2024-04-20T10:59:12.668 INFO:tasks.workunit.client.0.smithi027.stdout:0/140: dread d0/ff [0,4194304] 0 2024-04-20T10:59:12.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/141: link d0/d7/f1c d0/f20 0 2024-04-20T10:59:12.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/142: fsync d0/d7/fa 0 2024-04-20T10:59:12.670 INFO:tasks.workunit.client.0.smithi027.stdout:0/143: link d0/d7/f16 d0/d11/f21 0 2024-04-20T10:59:12.674 INFO:tasks.workunit.client.0.smithi027.stdout:2/101: dread d5/fd [0,4194304] 0 2024-04-20T10:59:12.675 INFO:tasks.workunit.client.0.smithi027.stdout:1/255: dwrite d3/d7/d2d/f3f [0,4194304] 0 2024-04-20T10:59:12.675 INFO:tasks.workunit.client.0.smithi027.stdout:3/229: dwrite d2/f16 [4194304,4194304] 0 2024-04-20T10:59:12.675 INFO:tasks.workunit.client.0.smithi027.stdout:2/102: mkdir d5/d1e/d21 0 2024-04-20T10:59:12.677 INFO:tasks.workunit.client.0.smithi027.stdout:8/236: dwrite d9/de/f3a [0,4194304] 0 2024-04-20T10:59:12.677 INFO:tasks.workunit.client.0.smithi027.stdout:1/256: link d3/d7/c2b d3/d7/d28/d35/c5f 0 2024-04-20T10:59:12.678 INFO:tasks.workunit.client.0.smithi027.stdout:6/175: dwrite da/d21/d28/f34 [0,4194304] 0 2024-04-20T10:59:12.678 INFO:tasks.workunit.client.0.smithi027.stdout:1/257: dread d3/d7/fc [0,4194304] 0 2024-04-20T10:59:12.678 INFO:tasks.workunit.client.0.smithi027.stdout:1/258: write d3/d20/f46 [737297,38762] 0 2024-04-20T10:59:12.679 INFO:tasks.workunit.client.0.smithi027.stdout:3/230: getdents d2/dd/d22 0 2024-04-20T10:59:12.681 INFO:tasks.workunit.client.0.smithi027.stdout:2/103: creat d5/d1e/f22 x:0 0 0 2024-04-20T10:59:12.684 INFO:tasks.workunit.client.0.smithi027.stdout:8/237: mkdir d9/de/d20/d53 0 2024-04-20T10:59:12.684 INFO:tasks.workunit.client.0.smithi027.stdout:8/238: chown d9/de/d20/l31 38515586 1 2024-04-20T10:59:12.685 INFO:tasks.workunit.client.0.smithi027.stdout:1/259: symlink d3/d45/l60 0 2024-04-20T10:59:12.687 INFO:tasks.workunit.client.0.smithi027.stdout:3/231: link d2/dd/f28 d2/d18/d2e/d41/f45 0 2024-04-20T10:59:12.688 INFO:tasks.workunit.client.0.smithi027.stdout:8/239: creat d9/f54 x:0 0 0 2024-04-20T10:59:12.689 INFO:tasks.workunit.client.0.smithi027.stdout:1/260: truncate d3/d7/f1d 221956 0 2024-04-20T10:59:12.690 INFO:tasks.workunit.client.0.smithi027.stdout:0/144: dwrite d0/f20 [0,4194304] 0 2024-04-20T10:59:12.692 INFO:tasks.workunit.client.0.smithi027.stdout:8/240: truncate d9/de/f1a 636381 0 2024-04-20T10:59:12.692 INFO:tasks.workunit.client.0.smithi027.stdout:8/241: write d9/de/d20/d26/f3f [4904478,24985] 0 2024-04-20T10:59:12.696 INFO:tasks.workunit.client.0.smithi027.stdout:6/176: dwrite da/df/f17 [0,4194304] 0 2024-04-20T10:59:12.696 INFO:tasks.workunit.client.0.smithi027.stdout:6/177: readlink l0 0 2024-04-20T10:59:12.698 INFO:tasks.workunit.client.0.smithi027.stdout:6/178: symlink da/dd/d3d/l40 0 2024-04-20T10:59:12.701 INFO:tasks.workunit.client.0.smithi027.stdout:2/104: dwrite d5/fd [0,4194304] 0 2024-04-20T10:59:12.701 INFO:tasks.workunit.client.0.smithi027.stdout:6/179: creat da/df/d19/d2e/f41 x:0 0 0 2024-04-20T10:59:12.702 INFO:tasks.workunit.client.0.smithi027.stdout:2/105: mknod d5/c23 0 2024-04-20T10:59:12.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/180: mkdir da/df/d19/d2e/d42 0 2024-04-20T10:59:12.705 INFO:tasks.workunit.client.0.smithi027.stdout:6/181: creat da/d21/d28/d2d/f43 x:0 0 0 2024-04-20T10:59:12.708 INFO:tasks.workunit.client.0.smithi027.stdout:3/232: dwrite d2/dd/d22/f2f [0,4194304] 0 2024-04-20T10:59:12.709 INFO:tasks.workunit.client.0.smithi027.stdout:2/106: read d5/fd [3716066,130644] 0 2024-04-20T10:59:12.711 INFO:tasks.workunit.client.0.smithi027.stdout:8/242: truncate d9/de/d18/f21 1795595 0 2024-04-20T10:59:12.713 INFO:tasks.workunit.client.0.smithi027.stdout:3/233: rename d2/f6 to d2/dd/df/f46 0 2024-04-20T10:59:12.714 INFO:tasks.workunit.client.0.smithi027.stdout:2/107: symlink d5/l24 0 2024-04-20T10:59:12.714 INFO:tasks.workunit.client.0.smithi027.stdout:1/261: dwrite d3/d7/f1f [0,4194304] 0 2024-04-20T10:59:12.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/243: link d9/de/d20/d26/f30 d9/de/d1e/d4a/f55 0 2024-04-20T10:59:12.718 INFO:tasks.workunit.client.0.smithi027.stdout:0/145: dwrite d0/d11/f21 [0,4194304] 0 2024-04-20T10:59:12.718 INFO:tasks.workunit.client.0.smithi027.stdout:1/262: rmdir d3/d45 39 2024-04-20T10:59:12.719 INFO:tasks.workunit.client.0.smithi027.stdout:8/244: mknod d9/de/d20/d4f/c56 0 2024-04-20T10:59:12.719 INFO:tasks.workunit.client.0.smithi027.stdout:8/245: readlink d9/de/d1e/l42 0 2024-04-20T10:59:12.719 INFO:tasks.workunit.client.0.smithi027.stdout:0/146: symlink d0/d7/d12/l22 0 2024-04-20T10:59:12.720 INFO:tasks.workunit.client.0.smithi027.stdout:1/263: dread d3/d7/f34 [0,4194304] 0 2024-04-20T10:59:12.721 INFO:tasks.workunit.client.0.smithi027.stdout:0/147: rename d0/d19/c1e to d0/d7/d12/d1a/c23 0 2024-04-20T10:59:12.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/246: dread f3 [0,4194304] 0 2024-04-20T10:59:12.722 INFO:tasks.workunit.client.0.smithi027.stdout:1/264: truncate d3/d7/f42 6391558 0 2024-04-20T10:59:12.724 INFO:tasks.workunit.client.0.smithi027.stdout:0/148: mkdir d0/d24 0 2024-04-20T10:59:12.724 INFO:tasks.workunit.client.0.smithi027.stdout:0/149: stat d0/d7/d12 0 2024-04-20T10:59:12.735 INFO:tasks.workunit.client.0.smithi027.stdout:3/234: dread d2/d18/f3f [0,4194304] 0 2024-04-20T10:59:12.737 INFO:tasks.workunit.client.0.smithi027.stdout:3/235: unlink d2/dd/f28 0 2024-04-20T10:59:12.737 INFO:tasks.workunit.client.0.smithi027.stdout:3/236: chown d2/d18/d2e/f35 0 1 2024-04-20T10:59:12.737 INFO:tasks.workunit.client.0.smithi027.stdout:3/237: write d2/dd/f19 [4445256,61066] 0 2024-04-20T10:59:12.740 INFO:tasks.workunit.client.0.smithi027.stdout:8/247: dwrite d9/de/d20/d26/f30 [0,4194304] 0 2024-04-20T10:59:12.740 INFO:tasks.workunit.client.0.smithi027.stdout:8/248: stat d9/de/f1b 0 2024-04-20T10:59:12.740 INFO:tasks.workunit.client.0.smithi027.stdout:8/249: dread - d9/de/d1e/d4a/f45 zero size 2024-04-20T10:59:12.740 INFO:tasks.workunit.client.0.smithi027.stdout:2/108: dwrite d5/f17 [0,4194304] 0 2024-04-20T10:59:12.740 INFO:tasks.workunit.client.0.smithi027.stdout:2/109: chown d5/c23 238168 1 2024-04-20T10:59:12.740 INFO:tasks.workunit.client.0.smithi027.stdout:6/182: dwrite da/d21/d28/f3e [0,4194304] 0 2024-04-20T10:59:12.741 INFO:tasks.workunit.client.0.smithi027.stdout:3/238: dread d2/f16 [0,4194304] 0 2024-04-20T10:59:12.741 INFO:tasks.workunit.client.0.smithi027.stdout:3/239: write d2/dd/f1a [1169540,22411] 0 2024-04-20T10:59:12.742 INFO:tasks.workunit.client.0.smithi027.stdout:8/250: mkdir d9/de/d20/d57 0 2024-04-20T10:59:12.742 INFO:tasks.workunit.client.0.smithi027.stdout:8/251: chown d9/de/f50 4109949 1 2024-04-20T10:59:12.742 INFO:tasks.workunit.client.0.smithi027.stdout:8/252: write d9/de/d1e/d49/f4c [27944,85408] 0 2024-04-20T10:59:12.744 INFO:tasks.workunit.client.0.smithi027.stdout:2/110: rmdir d5 39 2024-04-20T10:59:12.746 INFO:tasks.workunit.client.0.smithi027.stdout:0/150: getdents d0/d7/d12 0 2024-04-20T10:59:12.749 INFO:tasks.workunit.client.0.smithi027.stdout:3/240: dread d2/dd/f19 [0,4194304] 0 2024-04-20T10:59:12.751 INFO:tasks.workunit.client.0.smithi027.stdout:6/183: creat da/df/d19/d2e/d32/f44 x:0 0 0 2024-04-20T10:59:12.752 INFO:tasks.workunit.client.0.smithi027.stdout:8/253: symlink d9/de/d20/d26/l58 0 2024-04-20T10:59:12.754 INFO:tasks.workunit.client.0.smithi027.stdout:2/111: rename d5/l11 to d5/l25 0 2024-04-20T10:59:12.755 INFO:tasks.workunit.client.0.smithi027.stdout:3/241: dread d2/dd/f1a [0,4194304] 0 2024-04-20T10:59:12.755 INFO:tasks.workunit.client.0.smithi027.stdout:3/242: fdatasync d2/dd/d22/f2f 0 2024-04-20T10:59:12.755 INFO:tasks.workunit.client.0.smithi027.stdout:0/151: mkdir d0/d7/d12/d25 0 2024-04-20T10:59:12.759 INFO:tasks.workunit.client.0.smithi027.stdout:6/184: rename da/dd/fe to da/df/d19/d2e/f45 0 2024-04-20T10:59:12.759 INFO:tasks.workunit.client.0.smithi027.stdout:8/254: getdents d9/de 0 2024-04-20T10:59:12.760 INFO:tasks.workunit.client.0.smithi027.stdout:6/185: write da/f1b [126076,81861] 0 2024-04-20T10:59:12.761 INFO:tasks.workunit.client.0.smithi027.stdout:4/200: sync 2024-04-20T10:59:12.761 INFO:tasks.workunit.client.0.smithi027.stdout:7/161: sync 2024-04-20T10:59:12.761 INFO:tasks.workunit.client.0.smithi027.stdout:5/155: sync 2024-04-20T10:59:12.761 INFO:tasks.workunit.client.0.smithi027.stdout:9/138: sync 2024-04-20T10:59:12.761 INFO:tasks.workunit.client.0.smithi027.stdout:4/201: write d0/d14/d17/d1b/d2f/f42 [3634979,129605] 0 2024-04-20T10:59:12.761 INFO:tasks.workunit.client.0.smithi027.stdout:4/202: write d0/d14/d17/d18/f38 [492403,91040] 0 2024-04-20T10:59:12.762 INFO:tasks.workunit.client.0.smithi027.stdout:2/112: mkdir d5/d26 0 2024-04-20T10:59:12.762 INFO:tasks.workunit.client.0.smithi027.stdout:3/243: creat d2/d18/d2c/f47 x:0 0 0 2024-04-20T10:59:12.762 INFO:tasks.workunit.client.0.smithi027.stdout:2/113: write d5/d1e/f22 [809172,75191] 0 2024-04-20T10:59:12.768 INFO:tasks.workunit.client.0.smithi027.stdout:0/152: creat d0/d7/f26 x:0 0 0 2024-04-20T10:59:12.768 INFO:tasks.workunit.client.0.smithi027.stdout:0/153: chown d0/d7/c18 27 1 2024-04-20T10:59:12.768 INFO:tasks.workunit.client.0.smithi027.stdout:0/154: chown d0/d7/l10 118662 1 2024-04-20T10:59:12.768 INFO:tasks.workunit.client.0.smithi027.stdout:8/255: mkdir d9/de/d20/d48/d59 0 2024-04-20T10:59:12.768 INFO:tasks.workunit.client.0.smithi027.stdout:7/162: truncate d4/d1b/ff 194595 0 2024-04-20T10:59:12.768 INFO:tasks.workunit.client.0.smithi027.stdout:5/156: creat d19/d1b/d1c/d1e/d22/f38 x:0 0 0 2024-04-20T10:59:12.768 INFO:tasks.workunit.client.0.smithi027.stdout:5/157: write d19/d1b/d1c/d1e/d22/f38 [508969,81535] 0 2024-04-20T10:59:12.768 INFO:tasks.workunit.client.0.smithi027.stdout:5/158: chown cf 50815362 1 2024-04-20T10:59:12.768 INFO:tasks.workunit.client.0.smithi027.stdout:5/159: write d19/d1b/d1c/d1e/d25/f34 [42580,123946] 0 2024-04-20T10:59:12.769 INFO:tasks.workunit.client.0.smithi027.stdout:5/160: chown d19 166303 1 2024-04-20T10:59:12.771 INFO:tasks.workunit.client.0.smithi027.stdout:4/203: mknod d0/d14/d17/d18/c55 0 2024-04-20T10:59:12.771 INFO:tasks.workunit.client.0.smithi027.stdout:3/244: rmdir d2/d18/d2e 39 2024-04-20T10:59:12.772 INFO:tasks.workunit.client.0.smithi027.stdout:6/186: write da/df/f17 [307919,72820] 0 2024-04-20T10:59:12.772 INFO:tasks.workunit.client.0.smithi027.stdout:0/155: creat d0/d7/f27 x:0 0 0 2024-04-20T10:59:12.774 INFO:tasks.workunit.client.0.smithi027.stdout:5/161: rename f8 to d19/d1b/d20/f39 0 2024-04-20T10:59:12.775 INFO:tasks.workunit.client.0.smithi027.stdout:7/163: dread d4/d5/f6 [0,4194304] 0 2024-04-20T10:59:12.777 INFO:tasks.workunit.client.0.smithi027.stdout:1/265: read d3/d7/f8 [701955,91478] 0 2024-04-20T10:59:12.778 INFO:tasks.workunit.client.0.smithi027.stdout:3/245: symlink d2/dd/df/d23/l48 0 2024-04-20T10:59:12.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/162: symlink d19/d1b/d1c/d1e/d22/l3a 0 2024-04-20T10:59:12.781 INFO:tasks.workunit.client.0.smithi027.stdout:1/266: creat d3/d41/f61 x:0 0 0 2024-04-20T10:59:12.781 INFO:tasks.workunit.client.0.smithi027.stdout:5/163: creat d19/d1b/d1c/f3b x:0 0 0 2024-04-20T10:59:12.781 INFO:tasks.workunit.client.0.smithi027.stdout:9/139: dwrite d4/f1a [0,4194304] 0 2024-04-20T10:59:12.781 INFO:tasks.workunit.client.0.smithi027.stdout:5/164: truncate d19/d1b/d1c/d1e/d25/f2f 331092 0 2024-04-20T10:59:12.783 INFO:tasks.workunit.client.0.smithi027.stdout:1/267: mkdir d3/d7/d38/d62 0 2024-04-20T10:59:12.784 INFO:tasks.workunit.client.0.smithi027.stdout:1/268: fsync d3/f1b 0 2024-04-20T10:59:12.784 INFO:tasks.workunit.client.0.smithi027.stdout:4/204: write d0/fd [1839768,24457] 0 2024-04-20T10:59:12.785 INFO:tasks.workunit.client.0.smithi027.stdout:2/114: dwrite d5/d1e/f22 [0,4194304] 0 2024-04-20T10:59:12.786 INFO:tasks.workunit.client.0.smithi027.stdout:4/205: mknod d0/d14/d17/d1a/c56 0 2024-04-20T10:59:12.788 INFO:tasks.workunit.client.0.smithi027.stdout:9/140: rename d4/f1a to d4/d14/f28 0 2024-04-20T10:59:12.788 INFO:tasks.workunit.client.0.smithi027.stdout:6/187: write da/fb [1328620,103380] 0 2024-04-20T10:59:12.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/246: dread d2/dd/d22/f2f [0,4194304] 0 2024-04-20T10:59:12.791 INFO:tasks.workunit.client.0.smithi027.stdout:4/206: creat d0/d14/d17/d1b/d33/f57 x:0 0 0 2024-04-20T10:59:12.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/188: fsync da/fb 0 2024-04-20T10:59:12.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/189: fdatasync da/df/f10 0 2024-04-20T10:59:12.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/190: truncate da/f3b 1254026 0 2024-04-20T10:59:12.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/191: readlink l0 0 2024-04-20T10:59:12.794 INFO:tasks.workunit.client.0.smithi027.stdout:2/115: rename f3 to d5/d1e/d21/f27 0 2024-04-20T10:59:12.795 INFO:tasks.workunit.client.0.smithi027.stdout:9/141: mkdir d4/d1b/d1f/d20/d23/d29 0 2024-04-20T10:59:12.797 INFO:tasks.workunit.client.0.smithi027.stdout:4/207: mknod d0/d14/d17/d53/c58 0 2024-04-20T10:59:12.797 INFO:tasks.workunit.client.0.smithi027.stdout:4/208: dread - d0/f37 zero size 2024-04-20T10:59:12.798 INFO:tasks.workunit.client.0.smithi027.stdout:0/156: dwrite d0/d7/fa [4194304,4194304] 0 2024-04-20T10:59:12.799 INFO:tasks.workunit.client.0.smithi027.stdout:6/192: unlink da/d21/d28/l31 0 2024-04-20T10:59:12.799 INFO:tasks.workunit.client.0.smithi027.stdout:6/193: dread - da/d21/f36 zero size 2024-04-20T10:59:12.799 INFO:tasks.workunit.client.0.smithi027.stdout:8/256: dwrite d9/de/d1e/d4a/f4e [4194304,4194304] 0 2024-04-20T10:59:12.800 INFO:tasks.workunit.client.0.smithi027.stdout:8/257: dread - d9/f34 zero size 2024-04-20T10:59:12.800 INFO:tasks.workunit.client.0.smithi027.stdout:8/258: chown d9/f24 62753101 1 2024-04-20T10:59:12.800 INFO:tasks.workunit.client.0.smithi027.stdout:8/259: chown d9/f39 0 1 2024-04-20T10:59:12.800 INFO:tasks.workunit.client.0.smithi027.stdout:8/260: write d9/de/f1d [143968,71460] 0 2024-04-20T10:59:12.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/116: creat d5/f28 x:0 0 0 2024-04-20T10:59:12.801 INFO:tasks.workunit.client.0.smithi027.stdout:9/142: mkdir d4/d2a 0 2024-04-20T10:59:12.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/209: getdents d0/d14/d17 0 2024-04-20T10:59:12.807 INFO:tasks.workunit.client.0.smithi027.stdout:1/269: getdents d3/d7/d38 0 2024-04-20T10:59:12.807 INFO:tasks.workunit.client.0.smithi027.stdout:7/164: dread d4/d5/f19 [0,4194304] 0 2024-04-20T10:59:12.807 INFO:tasks.workunit.client.0.smithi027.stdout:7/165: write d4/d5/f24 [1044017,118949] 0 2024-04-20T10:59:12.807 INFO:tasks.workunit.client.0.smithi027.stdout:5/165: dwrite f3 [0,4194304] 0 2024-04-20T10:59:12.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/166: readlink d19/d1b/d1c/d1e/l36 0 2024-04-20T10:59:12.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/167: chown d19/f27 275079058 1 2024-04-20T10:59:12.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/168: truncate d19/d1b/d1c/d1e/d22/f38 811510 0 2024-04-20T10:59:12.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/169: write d19/d1b/d1c/f3b [292897,127932] 0 2024-04-20T10:59:12.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/170: dread - d19/f27 zero size 2024-04-20T10:59:12.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/171: write fc [1397876,113240] 0 2024-04-20T10:59:12.808 INFO:tasks.workunit.client.0.smithi027.stdout:0/157: unlink d0/d7/f16 0 2024-04-20T10:59:12.808 INFO:tasks.workunit.client.0.smithi027.stdout:6/194: rename l0 to da/df/d19/d2e/l46 0 2024-04-20T10:59:12.810 INFO:tasks.workunit.client.0.smithi027.stdout:9/143: link d4/l10 d4/d2a/l2b 0 2024-04-20T10:59:12.813 INFO:tasks.workunit.client.0.smithi027.stdout:2/117: dread d5/f17 [0,4194304] 0 2024-04-20T10:59:12.813 INFO:tasks.workunit.client.0.smithi027.stdout:2/118: write d5/f28 [436720,1361] 0 2024-04-20T10:59:12.813 INFO:tasks.workunit.client.0.smithi027.stdout:2/119: write d5/f28 [973271,2139] 0 2024-04-20T10:59:12.813 INFO:tasks.workunit.client.0.smithi027.stdout:1/270: link d3/d20/c27 d3/d41/c63 0 2024-04-20T10:59:12.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/210: rename d0/l2 to d0/d48/l59 0 2024-04-20T10:59:12.829 INFO:tasks.workunit.client.0.smithi027.stdout:8/261: dwrite d9/de/d18/f2d [0,4194304] 0 2024-04-20T10:59:12.831 INFO:tasks.workunit.client.0.smithi027.stdout:1/271: unlink d3/d7/c40 0 2024-04-20T10:59:12.832 INFO:tasks.workunit.client.0.smithi027.stdout:0/158: dwrite d0/f15 [0,4194304] 0 2024-04-20T10:59:12.838 INFO:tasks.workunit.client.0.smithi027.stdout:9/144: dwrite d4/f1c [4194304,4194304] 0 2024-04-20T10:59:12.838 INFO:tasks.workunit.client.0.smithi027.stdout:9/145: fdatasync d4/f7 0 2024-04-20T10:59:12.838 INFO:tasks.workunit.client.0.smithi027.stdout:9/146: chown d4/d14/c25 8314218 1 2024-04-20T10:59:12.838 INFO:tasks.workunit.client.0.smithi027.stdout:9/147: truncate d4/d14/f27 600652 0 2024-04-20T10:59:12.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/195: dwrite da/dd/f25 [0,4194304] 0 2024-04-20T10:59:12.847 INFO:tasks.workunit.client.0.smithi027.stdout:7/166: rename d4/d5/f6 to d4/f26 0 2024-04-20T10:59:12.847 INFO:tasks.workunit.client.0.smithi027.stdout:7/167: readlink d4/le 0 2024-04-20T10:59:12.848 INFO:tasks.workunit.client.0.smithi027.stdout:4/211: mknod d0/d14/d17/d53/c5a 0 2024-04-20T10:59:12.848 INFO:tasks.workunit.client.0.smithi027.stdout:7/168: write d4/d5/f24 [745884,26381] 0 2024-04-20T10:59:12.849 INFO:tasks.workunit.client.0.smithi027.stdout:8/262: symlink d9/de/d1e/l5a 0 2024-04-20T10:59:12.854 INFO:tasks.workunit.client.0.smithi027.stdout:6/196: truncate da/fb 267846 0 2024-04-20T10:59:12.854 INFO:tasks.workunit.client.0.smithi027.stdout:6/197: fdatasync da/df/f1a 0 2024-04-20T10:59:12.854 INFO:tasks.workunit.client.0.smithi027.stdout:6/198: chown da/df/d19/d2e/f41 7637367 1 2024-04-20T10:59:12.854 INFO:tasks.workunit.client.0.smithi027.stdout:6/199: write da/f3b [261810,57659] 0 2024-04-20T10:59:12.855 INFO:tasks.workunit.client.0.smithi027.stdout:6/200: fdatasync da/df/f17 0 2024-04-20T10:59:12.855 INFO:tasks.workunit.client.0.smithi027.stdout:5/172: rename d19/d1b/d20 to d19/d1b/d1c/d1e/d3c 0 2024-04-20T10:59:12.856 INFO:tasks.workunit.client.0.smithi027.stdout:4/212: symlink d0/d14/d17/d53/l5b 0 2024-04-20T10:59:12.856 INFO:tasks.workunit.client.0.smithi027.stdout:4/213: read d0/d14/d17/d1b/d2f/f42 [291526,107578] 0 2024-04-20T10:59:12.856 INFO:tasks.workunit.client.0.smithi027.stdout:8/263: write d9/de/d1e/d4a/f4e [8162454,122741] 0 2024-04-20T10:59:12.857 INFO:tasks.workunit.client.0.smithi027.stdout:8/264: fsync d9/de/f1d 0 2024-04-20T10:59:12.857 INFO:tasks.workunit.client.0.smithi027.stdout:7/169: creat d4/d1b/f27 x:0 0 0 2024-04-20T10:59:12.858 INFO:tasks.workunit.client.0.smithi027.stdout:6/201: mknod da/dd/d3d/c47 0 2024-04-20T10:59:12.858 INFO:tasks.workunit.client.0.smithi027.stdout:6/202: fdatasync da/d21/d28/d2d/f43 0 2024-04-20T10:59:12.859 INFO:tasks.workunit.client.0.smithi027.stdout:1/272: rename d3/d7/d28/c4c to d3/c64 0 2024-04-20T10:59:12.861 INFO:tasks.workunit.client.0.smithi027.stdout:4/214: mkdir d0/d14/d17/d1b/d33/d5c 0 2024-04-20T10:59:12.862 INFO:tasks.workunit.client.0.smithi027.stdout:8/265: mkdir d9/de/d20/d4f/d5b 0 2024-04-20T10:59:12.863 INFO:tasks.workunit.client.0.smithi027.stdout:6/203: mknod da/df/d19/d2e/d42/c48 0 2024-04-20T10:59:12.863 INFO:tasks.workunit.client.0.smithi027.stdout:6/204: readlink da/df/d19/l27 0 2024-04-20T10:59:12.865 INFO:tasks.workunit.client.0.smithi027.stdout:4/215: mknod d0/d14/c5d 0 2024-04-20T10:59:12.865 INFO:tasks.workunit.client.0.smithi027.stdout:4/216: fdatasync d0/d14/f4a 0 2024-04-20T10:59:12.866 INFO:tasks.workunit.client.0.smithi027.stdout:8/266: link d9/f24 d9/de/d1e/d49/f5c 0 2024-04-20T10:59:12.868 INFO:tasks.workunit.client.0.smithi027.stdout:7/170: dread d4/f26 [0,4194304] 0 2024-04-20T10:59:12.869 INFO:tasks.workunit.client.0.smithi027.stdout:6/205: unlink f4 0 2024-04-20T10:59:12.869 INFO:tasks.workunit.client.0.smithi027.stdout:6/206: fdatasync da/df/d1f/f3f 0 2024-04-20T10:59:12.869 INFO:tasks.workunit.client.0.smithi027.stdout:6/207: write da/df/d1f/f3f [54175,125290] 0 2024-04-20T10:59:12.873 INFO:tasks.workunit.client.0.smithi027.stdout:4/217: truncate d0/f13 3348942 0 2024-04-20T10:59:12.873 INFO:tasks.workunit.client.0.smithi027.stdout:4/218: write d0/fb [2811630,94762] 0 2024-04-20T10:59:12.873 INFO:tasks.workunit.client.0.smithi027.stdout:4/219: chown d0/d14/d17/d53 58 1 2024-04-20T10:59:12.874 INFO:tasks.workunit.client.0.smithi027.stdout:8/267: creat d9/de/d20/d4f/d5b/f5d x:0 0 0 2024-04-20T10:59:12.875 INFO:tasks.workunit.client.0.smithi027.stdout:6/208: creat da/df/f49 x:0 0 0 2024-04-20T10:59:12.875 INFO:tasks.workunit.client.0.smithi027.stdout:1/273: dread d3/d20/f3c [0,4194304] 0 2024-04-20T10:59:12.877 INFO:tasks.workunit.client.0.smithi027.stdout:5/173: dwrite d19/d1b/d1c/d1e/d22/f30 [0,4194304] 0 2024-04-20T10:59:12.877 INFO:tasks.workunit.client.0.smithi027.stdout:5/174: fdatasync d19/d1b/d1c/f3b 0 2024-04-20T10:59:12.877 INFO:tasks.workunit.client.0.smithi027.stdout:6/209: mknod da/d21/d28/d2d/c4a 0 2024-04-20T10:59:12.878 INFO:tasks.workunit.client.0.smithi027.stdout:4/220: mkdir d0/d5e 0 2024-04-20T10:59:12.879 INFO:tasks.workunit.client.0.smithi027.stdout:4/221: write d0/d14/d17/f16 [1478087,127547] 0 2024-04-20T10:59:12.881 INFO:tasks.workunit.client.0.smithi027.stdout:5/175: creat d19/d1b/d1c/d1e/d3c/f3d x:0 0 0 2024-04-20T10:59:12.884 INFO:tasks.workunit.client.0.smithi027.stdout:8/268: unlink d9/de/d20/d26/f3f 0 2024-04-20T10:59:12.884 INFO:tasks.workunit.client.0.smithi027.stdout:4/222: truncate d0/d14/d17/f2e 949766 0 2024-04-20T10:59:12.885 INFO:tasks.workunit.client.0.smithi027.stdout:7/171: dwrite d4/f26 [0,4194304] 0 2024-04-20T10:59:12.886 INFO:tasks.workunit.client.0.smithi027.stdout:7/172: creat d4/d1b/f28 x:0 0 0 2024-04-20T10:59:12.886 INFO:tasks.workunit.client.0.smithi027.stdout:7/173: write d4/d5/f1f [4890760,60850] 0 2024-04-20T10:59:12.888 INFO:tasks.workunit.client.0.smithi027.stdout:7/174: creat d4/d5/d8/d21/f29 x:0 0 0 2024-04-20T10:59:12.888 INFO:tasks.workunit.client.0.smithi027.stdout:6/210: dread da/f2a [0,4194304] 0 2024-04-20T10:59:12.888 INFO:tasks.workunit.client.0.smithi027.stdout:6/211: readlink da/df/d19/l24 0 2024-04-20T10:59:12.888 INFO:tasks.workunit.client.0.smithi027.stdout:0/159: rename d0/d7/c14 to d0/d11/c28 0 2024-04-20T10:59:12.890 INFO:tasks.workunit.client.0.smithi027.stdout:6/212: rename da/l18 to da/dd/d3c/l4b 0 2024-04-20T10:59:12.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/160: truncate d0/d19/f1d 68850 0 2024-04-20T10:59:12.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/161: chown d0/d7/d12/d25 294 1 2024-04-20T10:59:12.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/162: chown d0/d11 84280 1 2024-04-20T10:59:12.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/163: stat d0/d19/f1d 0 2024-04-20T10:59:12.892 INFO:tasks.workunit.client.0.smithi027.stdout:1/274: dwrite d3/d7/d2d/f53 [0,4194304] 0 2024-04-20T10:59:12.892 INFO:tasks.workunit.client.0.smithi027.stdout:1/275: fdatasync d3/d20/f3c 0 2024-04-20T10:59:12.893 INFO:tasks.workunit.client.0.smithi027.stdout:2/120: dread d5/f28 [0,4194304] 0 2024-04-20T10:59:12.897 INFO:tasks.workunit.client.0.smithi027.stdout:1/276: creat d3/d7/d38/f65 x:0 0 0 2024-04-20T10:59:12.898 INFO:tasks.workunit.client.0.smithi027.stdout:1/277: fdatasync d3/d7/d38/f65 0 2024-04-20T10:59:12.899 INFO:tasks.workunit.client.0.smithi027.stdout:2/121: link d5/dc/d16/c14 d5/d26/c29 0 2024-04-20T10:59:12.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/164: symlink d0/d7/d12/l29 0 2024-04-20T10:59:12.901 INFO:tasks.workunit.client.0.smithi027.stdout:1/278: mknod d3/d41/c66 0 2024-04-20T10:59:12.901 INFO:tasks.workunit.client.0.smithi027.stdout:1/279: write d3/d7/d28/d35/f52 [19176,17421] 0 2024-04-20T10:59:12.901 INFO:tasks.workunit.client.0.smithi027.stdout:1/280: dread - d3/f1b zero size 2024-04-20T10:59:12.901 INFO:tasks.workunit.client.0.smithi027.stdout:6/213: dread da/d21/d28/f34 [0,4194304] 0 2024-04-20T10:59:12.902 INFO:tasks.workunit.client.0.smithi027.stdout:1/281: dread d3/d7/f34 [0,4194304] 0 2024-04-20T10:59:12.902 INFO:tasks.workunit.client.0.smithi027.stdout:1/282: dread - d3/d7/f57 zero size 2024-04-20T10:59:12.902 INFO:tasks.workunit.client.0.smithi027.stdout:2/122: mkdir d5/dc/d16/d2a 0 2024-04-20T10:59:12.904 INFO:tasks.workunit.client.0.smithi027.stdout:1/283: link d3/d20/c5b d3/d45/d5d/c67 0 2024-04-20T10:59:12.904 INFO:tasks.workunit.client.0.smithi027.stdout:1/284: dread - d3/d7/d38/f65 zero size 2024-04-20T10:59:12.905 INFO:tasks.workunit.client.0.smithi027.stdout:7/175: write d4/d5/f19 [353649,83282] 0 2024-04-20T10:59:12.907 INFO:tasks.workunit.client.0.smithi027.stdout:7/176: write d4/d1b/ff [132072,41610] 0 2024-04-20T10:59:12.908 INFO:tasks.workunit.client.0.smithi027.stdout:3/247: sync 2024-04-20T10:59:12.910 INFO:tasks.workunit.client.0.smithi027.stdout:4/223: truncate d0/f13 4105850 0 2024-04-20T10:59:12.910 INFO:tasks.workunit.client.0.smithi027.stdout:4/224: rename d0 to d0/d14/d17/d1a/d3b/d5f 22 2024-04-20T10:59:12.910 INFO:tasks.workunit.client.0.smithi027.stdout:3/248: truncate d2/f1f 3348296 0 2024-04-20T10:59:12.912 INFO:tasks.workunit.client.0.smithi027.stdout:3/249: mknod d2/d18/d2e/d41/c49 0 2024-04-20T10:59:12.913 INFO:tasks.workunit.client.0.smithi027.stdout:0/165: dread d0/ff [0,4194304] 0 2024-04-20T10:59:12.913 INFO:tasks.workunit.client.0.smithi027.stdout:3/250: getdents d2/d18/d2c 0 2024-04-20T10:59:12.914 INFO:tasks.workunit.client.0.smithi027.stdout:0/166: creat d0/d19/f2a x:0 0 0 2024-04-20T10:59:12.914 INFO:tasks.workunit.client.0.smithi027.stdout:0/167: dread - d0/d7/f26 zero size 2024-04-20T10:59:12.915 INFO:tasks.workunit.client.0.smithi027.stdout:3/251: mknod d2/dd/c4a 0 2024-04-20T10:59:12.915 INFO:tasks.workunit.client.0.smithi027.stdout:3/252: truncate d2/d18/f26 142866 0 2024-04-20T10:59:12.915 INFO:tasks.workunit.client.0.smithi027.stdout:8/269: dwrite d9/de/f47 [0,4194304] 0 2024-04-20T10:59:12.917 INFO:tasks.workunit.client.0.smithi027.stdout:7/177: write d4/fc [3094683,63268] 0 2024-04-20T10:59:12.917 INFO:tasks.workunit.client.0.smithi027.stdout:0/168: readlink d0/d7/d12/l22 0 2024-04-20T10:59:12.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/270: rename d9/fb to d9/de/f5e 0 2024-04-20T10:59:12.919 INFO:tasks.workunit.client.0.smithi027.stdout:8/271: fsync d9/fd 0 2024-04-20T10:59:12.921 INFO:tasks.workunit.client.0.smithi027.stdout:0/169: rename d0/d24 to d0/d19/d2b 0 2024-04-20T10:59:12.921 INFO:tasks.workunit.client.0.smithi027.stdout:4/225: dread d0/d14/d17/d1b/d33/f35 [0,4194304] 0 2024-04-20T10:59:12.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/226: symlink d0/d14/d17/l60 0 2024-04-20T10:59:12.926 INFO:tasks.workunit.client.0.smithi027.stdout:0/170: mkdir d0/d11/d2c 0 2024-04-20T10:59:12.926 INFO:tasks.workunit.client.0.smithi027.stdout:0/171: dread - d0/d7/f26 zero size 2024-04-20T10:59:12.926 INFO:tasks.workunit.client.0.smithi027.stdout:0/172: readlink d0/d7/d12/l22 0 2024-04-20T10:59:12.926 INFO:tasks.workunit.client.0.smithi027.stdout:0/173: dread - d0/f1f zero size 2024-04-20T10:59:12.926 INFO:tasks.workunit.client.0.smithi027.stdout:3/253: read d2/d18/d2c/f30 [2745021,52225] 0 2024-04-20T10:59:12.926 INFO:tasks.workunit.client.0.smithi027.stdout:3/254: truncate d2/f17 8529441 0 2024-04-20T10:59:12.930 INFO:tasks.workunit.client.0.smithi027.stdout:4/227: creat d0/d14/d17/d1b/f61 x:0 0 0 2024-04-20T10:59:12.930 INFO:tasks.workunit.client.0.smithi027.stdout:4/228: write d0/f3a [413331,101999] 0 2024-04-20T10:59:12.932 INFO:tasks.workunit.client.0.smithi027.stdout:4/229: mknod d0/d14/d17/d1b/d33/d5c/c62 0 2024-04-20T10:59:12.932 INFO:tasks.workunit.client.0.smithi027.stdout:4/230: chown d0/d5e 31 1 2024-04-20T10:59:12.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/231: symlink d0/d14/d17/d1b/d2f/l63 0 2024-04-20T10:59:12.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/123: write d5/f28 [489132,129207] 0 2024-04-20T10:59:12.935 INFO:tasks.workunit.client.0.smithi027.stdout:1/285: dwrite d3/d20/f33 [0,4194304] 0 2024-04-20T10:59:12.935 INFO:tasks.workunit.client.0.smithi027.stdout:6/214: dwrite da/f2a [0,4194304] 0 2024-04-20T10:59:12.935 INFO:tasks.workunit.client.0.smithi027.stdout:6/215: chown c5 254 1 2024-04-20T10:59:12.939 INFO:tasks.workunit.client.0.smithi027.stdout:2/124: rename d5/d26/c29 to d5/d26/c2b 0 2024-04-20T10:59:12.940 INFO:tasks.workunit.client.0.smithi027.stdout:3/255: getdents d2/dd 0 2024-04-20T10:59:12.944 INFO:tasks.workunit.client.0.smithi027.stdout:0/174: dwrite d0/d7/f26 [0,4194304] 0 2024-04-20T10:59:12.944 INFO:tasks.workunit.client.0.smithi027.stdout:8/272: dwrite d9/de/f50 [0,4194304] 0 2024-04-20T10:59:12.945 INFO:tasks.workunit.client.0.smithi027.stdout:1/286: rename d3/d20/c27 to d3/d41/c68 0 2024-04-20T10:59:12.946 INFO:tasks.workunit.client.0.smithi027.stdout:1/287: chown d3/d7/c2a 22047672 1 2024-04-20T10:59:12.952 INFO:tasks.workunit.client.0.smithi027.stdout:3/256: rename d2/dd/df/d23/f2a to d2/dd/f4b 0 2024-04-20T10:59:12.952 INFO:tasks.workunit.client.0.smithi027.stdout:7/178: dwrite d4/f26 [0,4194304] 0 2024-04-20T10:59:12.953 INFO:tasks.workunit.client.0.smithi027.stdout:6/216: mknod da/df/d19/d2e/d42/c4c 0 2024-04-20T10:59:12.954 INFO:tasks.workunit.client.0.smithi027.stdout:1/288: symlink d3/d7/d2d/l69 0 2024-04-20T10:59:12.954 INFO:tasks.workunit.client.0.smithi027.stdout:1/289: truncate d3/d7/d28/d35/f52 566343 0 2024-04-20T10:59:12.956 INFO:tasks.workunit.client.0.smithi027.stdout:8/273: rename d9/c46 to d9/de/d1e/d49/c5f 0 2024-04-20T10:59:12.957 INFO:tasks.workunit.client.0.smithi027.stdout:2/125: dwrite d5/fd [0,4194304] 0 2024-04-20T10:59:12.957 INFO:tasks.workunit.client.0.smithi027.stdout:2/126: write d5/d1e/d21/f27 [3487399,5342] 0 2024-04-20T10:59:12.957 INFO:tasks.workunit.client.0.smithi027.stdout:3/257: mkdir d2/d18/d2c/d4c 0 2024-04-20T10:59:12.957 INFO:tasks.workunit.client.0.smithi027.stdout:3/258: chown d2/dd/c1c 410 1 2024-04-20T10:59:12.958 INFO:tasks.workunit.client.0.smithi027.stdout:7/179: symlink d4/d5/l2a 0 2024-04-20T10:59:12.958 INFO:tasks.workunit.client.0.smithi027.stdout:7/180: fsync d4/d5/d8/d21/f29 0 2024-04-20T10:59:12.959 INFO:tasks.workunit.client.0.smithi027.stdout:6/217: creat da/df/f4d x:0 0 0 2024-04-20T10:59:12.959 INFO:tasks.workunit.client.0.smithi027.stdout:6/218: stat da/d21/d28 0 2024-04-20T10:59:12.959 INFO:tasks.workunit.client.0.smithi027.stdout:6/219: readlink da/df/l37 0 2024-04-20T10:59:12.960 INFO:tasks.workunit.client.0.smithi027.stdout:1/290: rmdir d3/d7/d28 39 2024-04-20T10:59:12.960 INFO:tasks.workunit.client.0.smithi027.stdout:1/291: readlink d3/d7/d28/l31 0 2024-04-20T10:59:12.960 INFO:tasks.workunit.client.0.smithi027.stdout:1/292: write d3/d41/f61 [13245,74948] 0 2024-04-20T10:59:12.961 INFO:tasks.workunit.client.0.smithi027.stdout:8/274: rename d9/de/d1e/d49/f4c to d9/de/d20/d48/d59/f60 0 2024-04-20T10:59:12.963 INFO:tasks.workunit.client.0.smithi027.stdout:2/127: creat d5/d26/f2c x:0 0 0 2024-04-20T10:59:12.963 INFO:tasks.workunit.client.0.smithi027.stdout:0/175: dwrite d0/f20 [0,4194304] 0 2024-04-20T10:59:12.963 INFO:tasks.workunit.client.0.smithi027.stdout:2/128: chown d5/d1e 13778764 1 2024-04-20T10:59:12.963 INFO:tasks.workunit.client.0.smithi027.stdout:3/259: symlink d2/d18/d2c/d4c/l4d 0 2024-04-20T10:59:12.964 INFO:tasks.workunit.client.0.smithi027.stdout:7/181: symlink d4/d1b/l2b 0 2024-04-20T10:59:12.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/182: chown d4 0 1 2024-04-20T10:59:12.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/183: chown d4/d5/c9 56851 1 2024-04-20T10:59:12.965 INFO:tasks.workunit.client.0.smithi027.stdout:6/220: link da/df/f11 da/df/d19/d2e/d42/f4e 0 2024-04-20T10:59:12.966 INFO:tasks.workunit.client.0.smithi027.stdout:1/293: truncate d3/f1c 67013 0 2024-04-20T10:59:12.968 INFO:tasks.workunit.client.0.smithi027.stdout:8/275: unlink d9/de/c4d 0 2024-04-20T10:59:12.968 INFO:tasks.workunit.client.0.smithi027.stdout:4/232: read d0/d14/d17/d18/f38 [366085,90526] 0 2024-04-20T10:59:12.975 INFO:tasks.workunit.client.0.smithi027.stdout:6/221: dread da/dd/f25 [0,4194304] 0 2024-04-20T10:59:12.975 INFO:tasks.workunit.client.0.smithi027.stdout:7/184: symlink d4/d5/d8/d21/l2c 0 2024-04-20T10:59:12.975 INFO:tasks.workunit.client.0.smithi027.stdout:1/294: rename d3/d20/c29 to d3/d20/c6a 0 2024-04-20T10:59:12.976 INFO:tasks.workunit.client.0.smithi027.stdout:8/276: rename d9/de/d1e/f1f to d9/de/d20/d26/d3c/f61 0 2024-04-20T10:59:12.976 INFO:tasks.workunit.client.0.smithi027.stdout:7/185: write d4/f26 [4466352,81314] 0 2024-04-20T10:59:12.977 INFO:tasks.workunit.client.0.smithi027.stdout:6/222: creat da/df/f4f x:0 0 0 2024-04-20T10:59:12.978 INFO:tasks.workunit.client.0.smithi027.stdout:1/295: symlink d3/d7/d28/l6b 0 2024-04-20T10:59:12.979 INFO:tasks.workunit.client.0.smithi027.stdout:3/260: dread d2/f5 [0,4194304] 0 2024-04-20T10:59:12.979 INFO:tasks.workunit.client.0.smithi027.stdout:3/261: write d2/dd/df/f3a [820657,6615] 0 2024-04-20T10:59:12.979 INFO:tasks.workunit.client.0.smithi027.stdout:3/262: write d2/dd/f4b [762173,37416] 0 2024-04-20T10:59:12.979 INFO:tasks.workunit.client.0.smithi027.stdout:3/263: readlink d2/d18/d2e/l37 0 2024-04-20T10:59:12.979 INFO:tasks.workunit.client.0.smithi027.stdout:1/296: dread d3/d7/fc [0,4194304] 0 2024-04-20T10:59:12.979 INFO:tasks.workunit.client.0.smithi027.stdout:8/277: mkdir d9/de/d20/d48/d62 0 2024-04-20T10:59:12.980 INFO:tasks.workunit.client.0.smithi027.stdout:7/186: symlink d4/d5/d8/l2d 0 2024-04-20T10:59:12.981 INFO:tasks.workunit.client.0.smithi027.stdout:3/264: mknod d2/dd/df/c4e 0 2024-04-20T10:59:12.982 INFO:tasks.workunit.client.0.smithi027.stdout:8/278: link d9/de/d20/d4f/c56 d9/de/d20/d48/c63 0 2024-04-20T10:59:12.982 INFO:tasks.workunit.client.0.smithi027.stdout:8/279: truncate d9/de/d20/d48/d59/f60 953915 0 2024-04-20T10:59:12.983 INFO:tasks.workunit.client.0.smithi027.stdout:7/187: creat d4/f2e x:0 0 0 2024-04-20T10:59:12.983 INFO:tasks.workunit.client.0.smithi027.stdout:7/188: chown d4/d5/d8/d21/c22 15 1 2024-04-20T10:59:12.985 INFO:tasks.workunit.client.0.smithi027.stdout:8/280: mkdir d9/de/d64 0 2024-04-20T10:59:12.986 INFO:tasks.workunit.client.0.smithi027.stdout:2/129: dwrite d5/d1e/f22 [0,4194304] 0 2024-04-20T10:59:12.986 INFO:tasks.workunit.client.0.smithi027.stdout:0/176: dwrite d0/f15 [0,4194304] 0 2024-04-20T10:59:12.987 INFO:tasks.workunit.client.0.smithi027.stdout:8/281: truncate d9/de/f3a 2198711 0 2024-04-20T10:59:12.987 INFO:tasks.workunit.client.0.smithi027.stdout:2/130: symlink d5/d26/l2d 0 2024-04-20T10:59:12.988 INFO:tasks.workunit.client.0.smithi027.stdout:3/265: dread d2/d18/f29 [0,4194304] 0 2024-04-20T10:59:12.990 INFO:tasks.workunit.client.0.smithi027.stdout:1/297: dread d3/d7/d2d/f3f [0,4194304] 0 2024-04-20T10:59:12.992 INFO:tasks.workunit.client.0.smithi027.stdout:2/131: rename d5/d26/c2b to d5/dc/c2e 0 2024-04-20T10:59:12.992 INFO:tasks.workunit.client.0.smithi027.stdout:9/148: sync 2024-04-20T10:59:12.992 INFO:tasks.workunit.client.0.smithi027.stdout:9/149: dread - d4/d1b/f24 zero size 2024-04-20T10:59:12.992 INFO:tasks.workunit.client.0.smithi027.stdout:9/150: chown d4/d1b/d1f/d20/d23 146582 1 2024-04-20T10:59:12.992 INFO:tasks.workunit.client.0.smithi027.stdout:0/177: creat d0/d7/d12/d1a/f2d x:0 0 0 2024-04-20T10:59:12.993 INFO:tasks.workunit.client.0.smithi027.stdout:1/298: mkdir d3/d45/d5d/d6c 0 2024-04-20T10:59:12.997 INFO:tasks.workunit.client.0.smithi027.stdout:0/178: dread d0/d7/fa [0,4194304] 0 2024-04-20T10:59:12.997 INFO:tasks.workunit.client.0.smithi027.stdout:0/179: dread - d0/d7/f27 zero size 2024-04-20T10:59:12.997 INFO:tasks.workunit.client.0.smithi027.stdout:0/180: dread - d0/f1f zero size 2024-04-20T10:59:12.998 INFO:tasks.workunit.client.0.smithi027.stdout:2/132: stat d5/dc/l1d 0 2024-04-20T10:59:12.998 INFO:tasks.workunit.client.0.smithi027.stdout:2/133: write d5/f28 [1777215,50816] 0 2024-04-20T10:59:12.998 INFO:tasks.workunit.client.0.smithi027.stdout:9/151: mknod d4/c2c 0 2024-04-20T10:59:12.999 INFO:tasks.workunit.client.0.smithi027.stdout:7/189: dwrite d4/d1b/f28 [0,4194304] 0 2024-04-20T10:59:12.999 INFO:tasks.workunit.client.0.smithi027.stdout:1/299: symlink d3/d7/d2d/l6d 0 2024-04-20T10:59:13.000 INFO:tasks.workunit.client.0.smithi027.stdout:0/181: truncate d0/f20 164303 0 2024-04-20T10:59:13.001 INFO:tasks.workunit.client.0.smithi027.stdout:3/266: dread d2/d18/f26 [0,4194304] 0 2024-04-20T10:59:13.001 INFO:tasks.workunit.client.0.smithi027.stdout:9/152: mknod d4/d2a/c2d 0 2024-04-20T10:59:13.002 INFO:tasks.workunit.client.0.smithi027.stdout:8/282: getdents d9/de 0 2024-04-20T10:59:13.005 INFO:tasks.workunit.client.0.smithi027.stdout:7/190: rename d4/d5/d8/d21/f29 to d4/d5/d8/f2f 0 2024-04-20T10:59:13.005 INFO:tasks.workunit.client.0.smithi027.stdout:3/267: write d2/dd/f19 [970545,66897] 0 2024-04-20T10:59:13.006 INFO:tasks.workunit.client.0.smithi027.stdout:2/134: write d5/d1e/f22 [4022227,66930] 0 2024-04-20T10:59:13.011 INFO:tasks.workunit.client.0.smithi027.stdout:1/300: creat d3/d7/d38/d62/f6e x:0 0 0 2024-04-20T10:59:13.011 INFO:tasks.workunit.client.0.smithi027.stdout:1/301: write d3/d7/f23 [797532,13760] 0 2024-04-20T10:59:13.011 INFO:tasks.workunit.client.0.smithi027.stdout:2/135: write d5/dc/fe [775,102889] 0 2024-04-20T10:59:13.011 INFO:tasks.workunit.client.0.smithi027.stdout:7/191: getdents d4/d5 0 2024-04-20T10:59:13.011 INFO:tasks.workunit.client.0.smithi027.stdout:2/136: unlink d5/l25 0 2024-04-20T10:59:13.012 INFO:tasks.workunit.client.0.smithi027.stdout:2/137: chown d5/fd 554 1 2024-04-20T10:59:13.012 INFO:tasks.workunit.client.0.smithi027.stdout:2/138: stat d5/dc/f15 0 2024-04-20T10:59:13.013 INFO:tasks.workunit.client.0.smithi027.stdout:9/153: rename d4/d2a/c2d to d4/d14/c2e 0 2024-04-20T10:59:13.014 INFO:tasks.workunit.client.0.smithi027.stdout:7/192: truncate d4/d5/f1f 3735990 0 2024-04-20T10:59:13.014 INFO:tasks.workunit.client.0.smithi027.stdout:7/193: chown d4/d5/f19 206 1 2024-04-20T10:59:13.014 INFO:tasks.workunit.client.0.smithi027.stdout:7/194: chown d4/c1d 5454 1 2024-04-20T10:59:13.015 INFO:tasks.workunit.client.0.smithi027.stdout:9/154: mknod d4/d2a/c2f 0 2024-04-20T10:59:13.015 INFO:tasks.workunit.client.0.smithi027.stdout:9/155: write d4/d1b/f22 [458831,85806] 0 2024-04-20T10:59:13.016 INFO:tasks.workunit.client.0.smithi027.stdout:9/156: write d4/d14/f27 [424713,121528] 0 2024-04-20T10:59:13.016 INFO:tasks.workunit.client.0.smithi027.stdout:9/157: dread - d4/d14/f21 zero size 2024-04-20T10:59:13.016 INFO:tasks.workunit.client.0.smithi027.stdout:9/158: stat d4/d1b 0 2024-04-20T10:59:13.017 INFO:tasks.workunit.client.0.smithi027.stdout:8/283: rename d9/de/d1e/d4a/f55 to d9/de/d1e/f65 0 2024-04-20T10:59:13.021 INFO:tasks.workunit.client.0.smithi027.stdout:9/159: rename d4/ce to d4/d14/c30 0 2024-04-20T10:59:13.022 INFO:tasks.workunit.client.0.smithi027.stdout:0/182: dread d0/ff [0,4194304] 0 2024-04-20T10:59:13.024 INFO:tasks.workunit.client.0.smithi027.stdout:7/195: dread d4/d5/f14 [0,4194304] 0 2024-04-20T10:59:13.025 INFO:tasks.workunit.client.0.smithi027.stdout:0/183: rename d0/d19 to d0/d11/d2c/d2e 0 2024-04-20T10:59:13.025 INFO:tasks.workunit.client.0.smithi027.stdout:2/139: fdatasync d5/f28 0 2024-04-20T10:59:13.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/184: mknod d0/d7/d12/d1a/c2f 0 2024-04-20T10:59:13.030 INFO:tasks.workunit.client.0.smithi027.stdout:5/176: sync 2024-04-20T10:59:13.031 INFO:tasks.workunit.client.0.smithi027.stdout:0/185: mkdir d0/d11/d2c/d30 0 2024-04-20T10:59:13.031 INFO:tasks.workunit.client.0.smithi027.stdout:0/186: stat d0/d7/f27 0 2024-04-20T10:59:13.037 INFO:tasks.workunit.client.0.smithi027.stdout:1/302: dwrite d3/f17 [0,4194304] 0 2024-04-20T10:59:13.037 INFO:tasks.workunit.client.0.smithi027.stdout:1/303: stat d3/d7/d28/l31 0 2024-04-20T10:59:13.040 INFO:tasks.workunit.client.0.smithi027.stdout:3/268: dwrite d2/dd/df/f46 [0,4194304] 0 2024-04-20T10:59:13.044 INFO:tasks.workunit.client.0.smithi027.stdout:2/140: dread d5/d1e/f22 [0,4194304] 0 2024-04-20T10:59:13.044 INFO:tasks.workunit.client.0.smithi027.stdout:9/160: dwrite d4/d1b/f24 [0,4194304] 0 2024-04-20T10:59:13.044 INFO:tasks.workunit.client.0.smithi027.stdout:9/161: chown d4/l10 29 1 2024-04-20T10:59:13.052 INFO:tasks.workunit.client.0.smithi027.stdout:7/196: dwrite d4/d1b/ff [0,4194304] 0 2024-04-20T10:59:13.054 INFO:tasks.workunit.client.0.smithi027.stdout:9/162: dread d4/f7 [4194304,4194304] 0 2024-04-20T10:59:13.063 INFO:tasks.workunit.client.0.smithi027.stdout:8/284: dwrite d9/de/f2f [0,4194304] 0 2024-04-20T10:59:13.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/285: dwrite d9/f24 [0,4194304] 0 2024-04-20T10:59:13.183 INFO:tasks.workunit.client.0.smithi027.stdout:9/163: dread d4/fc [0,4194304] 0 2024-04-20T10:59:13.289 INFO:tasks.workunit.client.0.smithi027.stdout:5/177: mknod d19/d1b/d1c/d1e/d22/c3e 0 2024-04-20T10:59:13.289 INFO:tasks.workunit.client.0.smithi027.stdout:0/187: creat d0/d7/d12/d25/f31 x:0 0 0 2024-04-20T10:59:13.289 INFO:tasks.workunit.client.0.smithi027.stdout:0/188: stat d0/d7/d12/d25/f31 0 2024-04-20T10:59:13.290 INFO:tasks.workunit.client.0.smithi027.stdout:1/304: mknod d3/d45/c6f 0 2024-04-20T10:59:13.290 INFO:tasks.workunit.client.0.smithi027.stdout:1/305: fdatasync d3/d7/d28/f5c 0 2024-04-20T10:59:13.290 INFO:tasks.workunit.client.0.smithi027.stdout:3/269: getdents d2/d18/d2c/d4c 0 2024-04-20T10:59:13.291 INFO:tasks.workunit.client.0.smithi027.stdout:2/141: symlink d5/dc/d16/l2f 0 2024-04-20T10:59:13.292 INFO:tasks.workunit.client.0.smithi027.stdout:7/197: mknod d4/c30 0 2024-04-20T10:59:13.297 INFO:tasks.workunit.client.0.smithi027.stdout:8/286: creat d9/de/d1e/f66 x:0 0 0 2024-04-20T10:59:13.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/164: rename d4/d1b/d1f/d20 to d4/d1b/d31 0 2024-04-20T10:59:13.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/165: stat d4/d14/l1d 0 2024-04-20T10:59:13.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/166: chown d4/d14/f19 918399 1 2024-04-20T10:59:13.299 INFO:tasks.workunit.client.0.smithi027.stdout:1/306: creat d3/d7/d28/f70 x:0 0 0 2024-04-20T10:59:13.300 INFO:tasks.workunit.client.0.smithi027.stdout:3/270: creat d2/d18/d2c/f4f x:0 0 0 2024-04-20T10:59:13.302 INFO:tasks.workunit.client.0.smithi027.stdout:4/233: sync 2024-04-20T10:59:13.303 INFO:tasks.workunit.client.0.smithi027.stdout:2/142: getdents d5/dc 0 2024-04-20T10:59:13.303 INFO:tasks.workunit.client.0.smithi027.stdout:5/178: rename fd to d19/d1b/d1c/d1e/f3f 0 2024-04-20T10:59:13.305 INFO:tasks.workunit.client.0.smithi027.stdout:8/287: dread d9/de/d18/f22 [0,4194304] 0 2024-04-20T10:59:13.305 INFO:tasks.workunit.client.0.smithi027.stdout:1/307: link d3/d45/l60 d3/d7/l71 0 2024-04-20T10:59:13.306 INFO:tasks.workunit.client.0.smithi027.stdout:6/223: sync 2024-04-20T10:59:13.306 INFO:tasks.workunit.client.0.smithi027.stdout:4/234: rmdir d0/d14/d17 39 2024-04-20T10:59:13.306 INFO:tasks.workunit.client.0.smithi027.stdout:6/224: dread - da/d21/d28/d2d/f43 zero size 2024-04-20T10:59:13.308 INFO:tasks.workunit.client.0.smithi027.stdout:8/288: creat d9/de/d20/d57/f67 x:0 0 0 2024-04-20T10:59:13.308 INFO:tasks.workunit.client.0.smithi027.stdout:9/167: write d4/f1e [2066754,28891] 0 2024-04-20T10:59:13.308 INFO:tasks.workunit.client.0.smithi027.stdout:9/168: readlink d4/lf 0 2024-04-20T10:59:13.308 INFO:tasks.workunit.client.0.smithi027.stdout:9/169: readlink d4/d2a/l2b 0 2024-04-20T10:59:13.309 INFO:tasks.workunit.client.0.smithi027.stdout:0/189: rename d0/f15 to d0/d11/f32 0 2024-04-20T10:59:13.311 INFO:tasks.workunit.client.0.smithi027.stdout:8/289: rmdir d9/de/d20/d26 39 2024-04-20T10:59:13.313 INFO:tasks.workunit.client.0.smithi027.stdout:9/170: mknod d4/d2a/c32 0 2024-04-20T10:59:13.313 INFO:tasks.workunit.client.0.smithi027.stdout:1/308: dread d3/d7/f42 [0,4194304] 0 2024-04-20T10:59:13.313 INFO:tasks.workunit.client.0.smithi027.stdout:1/309: stat d3/d7/d38/f56 0 2024-04-20T10:59:13.314 INFO:tasks.workunit.client.0.smithi027.stdout:3/271: rename d2/f5 to d2/dd/df/d23/f50 0 2024-04-20T10:59:13.316 INFO:tasks.workunit.client.0.smithi027.stdout:8/290: creat d9/de/d1e/f68 x:0 0 0 2024-04-20T10:59:13.317 INFO:tasks.workunit.client.0.smithi027.stdout:1/310: creat d3/d45/d5d/d6c/f72 x:0 0 0 2024-04-20T10:59:13.319 INFO:tasks.workunit.client.0.smithi027.stdout:9/171: write d4/f1c [1067048,10058] 0 2024-04-20T10:59:13.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/235: rename d0/d14/d17/c47 to d0/d48/c64 0 2024-04-20T10:59:13.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/236: fsync d0/d14/d17/f2e 0 2024-04-20T10:59:13.320 INFO:tasks.workunit.client.0.smithi027.stdout:8/291: link d9/de/d1e/f65 d9/de/d20/d26/d3c/d4b/f69 0 2024-04-20T10:59:13.320 INFO:tasks.workunit.client.0.smithi027.stdout:8/292: readlink d9/de/d1e/d4a/l40 0 2024-04-20T10:59:13.320 INFO:tasks.workunit.client.0.smithi027.stdout:1/311: getdents d3/d45/d5d 0 2024-04-20T10:59:13.322 INFO:tasks.workunit.client.0.smithi027.stdout:9/172: symlink d4/d1b/d1f/l33 0 2024-04-20T10:59:13.322 INFO:tasks.workunit.client.0.smithi027.stdout:9/173: fsync d4/d14/f27 0 2024-04-20T10:59:13.322 INFO:tasks.workunit.client.0.smithi027.stdout:3/272: write d2/f12 [2758267,48264] 0 2024-04-20T10:59:13.322 INFO:tasks.workunit.client.0.smithi027.stdout:3/273: fdatasync d2/dd/f10 0 2024-04-20T10:59:13.323 INFO:tasks.workunit.client.0.smithi027.stdout:4/237: symlink d0/d14/d17/d53/l65 0 2024-04-20T10:59:13.324 INFO:tasks.workunit.client.0.smithi027.stdout:1/312: mkdir d3/d7/d2d/d73 0 2024-04-20T10:59:13.326 INFO:tasks.workunit.client.0.smithi027.stdout:1/313: symlink d3/d45/d5d/d6c/l74 0 2024-04-20T10:59:13.326 INFO:tasks.workunit.client.0.smithi027.stdout:3/274: write d2/d18/f3f [3268684,60996] 0 2024-04-20T10:59:13.326 INFO:tasks.workunit.client.0.smithi027.stdout:3/275: chown d2/dd/f4b 1371 1 2024-04-20T10:59:13.327 INFO:tasks.workunit.client.0.smithi027.stdout:3/276: truncate d2/d18/d2e/f35 826184 0 2024-04-20T10:59:13.335 INFO:tasks.workunit.client.0.smithi027.stdout:2/143: dwrite d5/f28 [0,4194304] 0 2024-04-20T10:59:13.335 INFO:tasks.workunit.client.0.smithi027.stdout:5/179: dwrite f17 [0,4194304] 0 2024-04-20T10:59:13.337 INFO:tasks.workunit.client.0.smithi027.stdout:0/190: dwrite d0/d7/d12/d1a/f2d [0,4194304] 0 2024-04-20T10:59:13.337 INFO:tasks.workunit.client.0.smithi027.stdout:2/144: mkdir d5/d26/d30 0 2024-04-20T10:59:13.338 INFO:tasks.workunit.client.0.smithi027.stdout:5/180: mkdir d19/d1b/d1c/d1e/d25/d40 0 2024-04-20T10:59:13.342 INFO:tasks.workunit.client.0.smithi027.stdout:8/293: rename d9/de/d20/d57 to d9/de/d64/d6a 0 2024-04-20T10:59:13.343 INFO:tasks.workunit.client.0.smithi027.stdout:0/191: rmdir d0/d11/d2c/d30 0 2024-04-20T10:59:13.343 INFO:tasks.workunit.client.0.smithi027.stdout:9/174: getdents d4/d2a 0 2024-04-20T10:59:13.346 INFO:tasks.workunit.client.0.smithi027.stdout:4/238: rename d0/d14/d17/d53/l65 to d0/d14/d17/d18/l66 0 2024-04-20T10:59:13.346 INFO:tasks.workunit.client.0.smithi027.stdout:3/277: fsync d2/dd/df/d23/f50 0 2024-04-20T10:59:13.347 INFO:tasks.workunit.client.0.smithi027.stdout:3/278: fsync d2/dd/df/d33/f3c 0 2024-04-20T10:59:13.347 INFO:tasks.workunit.client.0.smithi027.stdout:0/192: link d0/d7/d12/d1a/c23 d0/d7/d12/d25/c33 0 2024-04-20T10:59:13.349 INFO:tasks.workunit.client.0.smithi027.stdout:9/175: symlink d4/d1b/l34 0 2024-04-20T10:59:13.354 INFO:tasks.workunit.client.0.smithi027.stdout:3/279: write d2/f17 [770027,105014] 0 2024-04-20T10:59:13.355 INFO:tasks.workunit.client.0.smithi027.stdout:1/314: rename d3/d20/c58 to d3/d41/c75 0 2024-04-20T10:59:13.355 INFO:tasks.workunit.client.0.smithi027.stdout:4/239: creat d0/d14/d17/d1b/d2f/f67 x:0 0 0 2024-04-20T10:59:13.355 INFO:tasks.workunit.client.0.smithi027.stdout:4/240: readlink d0/l10 0 2024-04-20T10:59:13.355 INFO:tasks.workunit.client.0.smithi027.stdout:0/193: getdents d0/d7 0 2024-04-20T10:59:13.355 INFO:tasks.workunit.client.0.smithi027.stdout:3/280: mknod d2/d18/d2c/d4c/c51 0 2024-04-20T10:59:13.356 INFO:tasks.workunit.client.0.smithi027.stdout:8/294: rename d9/c52 to d9/de/d20/d53/c6b 0 2024-04-20T10:59:13.357 INFO:tasks.workunit.client.0.smithi027.stdout:1/315: symlink d3/d7/d38/d62/l76 0 2024-04-20T10:59:13.358 INFO:tasks.workunit.client.0.smithi027.stdout:4/241: mkdir d0/d48/d68 0 2024-04-20T10:59:13.359 INFO:tasks.workunit.client.0.smithi027.stdout:0/194: mkdir d0/d7/d34 0 2024-04-20T10:59:13.359 INFO:tasks.workunit.client.0.smithi027.stdout:4/242: mknod d0/d14/d17/d1b/c69 0 2024-04-20T10:59:13.360 INFO:tasks.workunit.client.0.smithi027.stdout:0/195: mkdir d0/d11/d2c/d2e/d35 0 2024-04-20T10:59:13.361 INFO:tasks.workunit.client.0.smithi027.stdout:4/243: rmdir d0/d14/d17/d18 39 2024-04-20T10:59:13.361 INFO:tasks.workunit.client.0.smithi027.stdout:4/244: write d0/d14/d17/f16 [1833878,96898] 0 2024-04-20T10:59:13.361 INFO:tasks.workunit.client.0.smithi027.stdout:4/245: fdatasync d0/d14/d17/d1b/f61 0 2024-04-20T10:59:13.361 INFO:tasks.workunit.client.0.smithi027.stdout:4/246: chown d0/d14/d17/d1b/f30 9346 1 2024-04-20T10:59:13.361 INFO:tasks.workunit.client.0.smithi027.stdout:4/247: fdatasync d0/d14/d17/d1b/d2f/f42 0 2024-04-20T10:59:13.362 INFO:tasks.workunit.client.0.smithi027.stdout:4/248: mkdir d0/d14/d17/d53/d6a 0 2024-04-20T10:59:13.365 INFO:tasks.workunit.client.0.smithi027.stdout:5/181: dwrite d19/d1b/d1c/d1e/f3f [0,4194304] 0 2024-04-20T10:59:13.365 INFO:tasks.workunit.client.0.smithi027.stdout:2/145: dwrite d5/dc/f1c [0,4194304] 0 2024-04-20T10:59:13.367 INFO:tasks.workunit.client.0.smithi027.stdout:5/182: creat d19/d1b/d1c/d1e/d25/f41 x:0 0 0 2024-04-20T10:59:13.370 INFO:tasks.workunit.client.0.smithi027.stdout:0/196: dread d0/d7/fa [0,4194304] 0 2024-04-20T10:59:13.372 INFO:tasks.workunit.client.0.smithi027.stdout:3/281: write d2/f9 [1426906,68459] 0 2024-04-20T10:59:13.372 INFO:tasks.workunit.client.0.smithi027.stdout:3/282: write d2/dd/df/d33/f3c [615284,1365] 0 2024-04-20T10:59:13.372 INFO:tasks.workunit.client.0.smithi027.stdout:3/283: write d2/dd/df/d23/f50 [2240184,87827] 0 2024-04-20T10:59:13.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/284: mknod d2/d18/c52 0 2024-04-20T10:59:13.375 INFO:tasks.workunit.client.0.smithi027.stdout:3/285: creat d2/d18/f53 x:0 0 0 2024-04-20T10:59:13.377 INFO:tasks.workunit.client.0.smithi027.stdout:2/146: read d5/fd [1074036,31659] 0 2024-04-20T10:59:13.379 INFO:tasks.workunit.client.0.smithi027.stdout:3/286: dread d2/d18/d2e/f35 [0,4194304] 0 2024-04-20T10:59:13.381 INFO:tasks.workunit.client.0.smithi027.stdout:5/183: write f17 [1665989,24897] 0 2024-04-20T10:59:13.381 INFO:tasks.workunit.client.0.smithi027.stdout:5/184: truncate d19/d1b/d1c/d1e/d22/f30 4449724 0 2024-04-20T10:59:13.381 INFO:tasks.workunit.client.0.smithi027.stdout:5/185: stat f0 0 2024-04-20T10:59:13.383 INFO:tasks.workunit.client.0.smithi027.stdout:5/186: creat d19/d1b/d1c/d1e/d22/f42 x:0 0 0 2024-04-20T10:59:13.383 INFO:tasks.workunit.client.0.smithi027.stdout:5/187: truncate d19/f27 553722 0 2024-04-20T10:59:13.384 INFO:tasks.workunit.client.0.smithi027.stdout:5/188: creat d19/d1b/f43 x:0 0 0 2024-04-20T10:59:13.385 INFO:tasks.workunit.client.0.smithi027.stdout:5/189: dread d19/d1b/d1c/d1e/d3c/f31 [0,4194304] 0 2024-04-20T10:59:13.386 INFO:tasks.workunit.client.0.smithi027.stdout:5/190: mknod d19/c44 0 2024-04-20T10:59:13.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/147: dread f1 [0,4194304] 0 2024-04-20T10:59:13.387 INFO:tasks.workunit.client.0.smithi027.stdout:5/191: dread f18 [0,4194304] 0 2024-04-20T10:59:13.389 INFO:tasks.workunit.client.0.smithi027.stdout:5/192: creat d19/d1b/d1c/f45 x:0 0 0 2024-04-20T10:59:13.391 INFO:tasks.workunit.client.0.smithi027.stdout:9/176: dwrite d4/d1b/f24 [0,4194304] 0 2024-04-20T10:59:13.391 INFO:tasks.workunit.client.0.smithi027.stdout:1/316: dwrite d3/d7/f55 [0,4194304] 0 2024-04-20T10:59:13.398 INFO:tasks.workunit.client.0.smithi027.stdout:0/197: dwrite d0/d7/f1c [0,4194304] 0 2024-04-20T10:59:13.398 INFO:tasks.workunit.client.0.smithi027.stdout:2/148: dread d5/dc/f1c [0,4194304] 0 2024-04-20T10:59:13.399 INFO:tasks.workunit.client.0.smithi027.stdout:8/295: dwrite d9/f33 [0,4194304] 0 2024-04-20T10:59:13.399 INFO:tasks.workunit.client.0.smithi027.stdout:0/198: readlink d0/d7/d12/l29 0 2024-04-20T10:59:13.399 INFO:tasks.workunit.client.0.smithi027.stdout:8/296: dread - d9/de/d64/d6a/f67 zero size 2024-04-20T10:59:13.399 INFO:tasks.workunit.client.0.smithi027.stdout:0/199: truncate d0/f1f 289910 0 2024-04-20T10:59:13.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/149: write d5/dc/f19 [782041,81932] 0 2024-04-20T10:59:13.400 INFO:tasks.workunit.client.0.smithi027.stdout:3/287: dwrite d2/d18/d2e/d41/f45 [0,4194304] 0 2024-04-20T10:59:13.400 INFO:tasks.workunit.client.0.smithi027.stdout:1/317: write d3/d7/f55 [1696304,64361] 0 2024-04-20T10:59:13.400 INFO:tasks.workunit.client.0.smithi027.stdout:9/177: creat d4/d1b/d31/d23/d29/f35 x:0 0 0 2024-04-20T10:59:13.400 INFO:tasks.workunit.client.0.smithi027.stdout:9/178: dread - d4/d1b/d31/d23/d29/f35 zero size 2024-04-20T10:59:13.404 INFO:tasks.workunit.client.0.smithi027.stdout:0/200: link d0/d7/d12/d1a/c23 d0/d11/d2c/c36 0 2024-04-20T10:59:13.405 INFO:tasks.workunit.client.0.smithi027.stdout:8/297: mkdir d9/de/d64/d6c 0 2024-04-20T10:59:13.405 INFO:tasks.workunit.client.0.smithi027.stdout:9/179: write d4/f1c [7559992,125287] 0 2024-04-20T10:59:13.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/150: rmdir d5/d26 39 2024-04-20T10:59:13.407 INFO:tasks.workunit.client.0.smithi027.stdout:3/288: creat d2/d18/d2c/f54 x:0 0 0 2024-04-20T10:59:13.408 INFO:tasks.workunit.client.0.smithi027.stdout:1/318: symlink d3/d45/d5d/l77 0 2024-04-20T10:59:13.409 INFO:tasks.workunit.client.0.smithi027.stdout:0/201: symlink d0/d11/d2c/d2e/l37 0 2024-04-20T10:59:13.411 INFO:tasks.workunit.client.0.smithi027.stdout:8/298: unlink d9/de/f50 0 2024-04-20T10:59:13.411 INFO:tasks.workunit.client.0.smithi027.stdout:8/299: read d9/de/d18/f21 [936233,10076] 0 2024-04-20T10:59:13.411 INFO:tasks.workunit.client.0.smithi027.stdout:8/300: chown d9/de/d20/d26/d3c 42229690 1 2024-04-20T10:59:13.411 INFO:tasks.workunit.client.0.smithi027.stdout:1/319: write d3/d7/d2d/f3f [2739435,22465] 0 2024-04-20T10:59:13.412 INFO:tasks.workunit.client.0.smithi027.stdout:9/180: rename d4/fc to d4/d1b/d31/d23/f36 0 2024-04-20T10:59:13.412 INFO:tasks.workunit.client.0.smithi027.stdout:9/181: fdatasync d4/d1b/d31/d23/d29/f35 0 2024-04-20T10:59:13.414 INFO:tasks.workunit.client.0.smithi027.stdout:0/202: write d0/ff [4189785,69341] 0 2024-04-20T10:59:13.414 INFO:tasks.workunit.client.0.smithi027.stdout:0/203: chown d0 1699 1 2024-04-20T10:59:13.415 INFO:tasks.workunit.client.0.smithi027.stdout:8/301: mkdir d9/de/d1e/d4a/d6d 0 2024-04-20T10:59:13.418 INFO:tasks.workunit.client.0.smithi027.stdout:1/320: dread d3/d7/d28/f3d [0,4194304] 0 2024-04-20T10:59:13.419 INFO:tasks.workunit.client.0.smithi027.stdout:3/289: mknod d2/d18/d2e/d41/c55 0 2024-04-20T10:59:13.419 INFO:tasks.workunit.client.0.smithi027.stdout:3/290: read d2/f9 [1079093,58013] 0 2024-04-20T10:59:13.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/302: mknod d9/de/d64/c6e 0 2024-04-20T10:59:13.428 INFO:tasks.workunit.client.0.smithi027.stdout:3/291: read d2/f14 [2077863,11361] 0 2024-04-20T10:59:13.428 INFO:tasks.workunit.client.0.smithi027.stdout:9/182: link d4/d14/l16 d4/d14/l37 0 2024-04-20T10:59:13.428 INFO:tasks.workunit.client.0.smithi027.stdout:9/183: write d4/f1e [1162535,6158] 0 2024-04-20T10:59:13.428 INFO:tasks.workunit.client.0.smithi027.stdout:9/184: truncate d4/d14/f28 4620686 0 2024-04-20T10:59:13.428 INFO:tasks.workunit.client.0.smithi027.stdout:9/185: fdatasync d4/d1b/d31/d23/d29/f35 0 2024-04-20T10:59:13.431 INFO:tasks.workunit.client.0.smithi027.stdout:5/193: dwrite f17 [0,4194304] 0 2024-04-20T10:59:13.432 INFO:tasks.workunit.client.0.smithi027.stdout:5/194: write d19/d1b/d1c/d1e/d3c/f39 [1304688,88718] 0 2024-04-20T10:59:13.437 INFO:tasks.workunit.client.0.smithi027.stdout:5/195: link d19/d1b/d1c/d1e/d25/f34 d19/d1b/d1c/d1e/d3c/f46 0 2024-04-20T10:59:13.438 INFO:tasks.workunit.client.0.smithi027.stdout:5/196: mkdir d19/d1b/d1c/d1e/d22/d47 0 2024-04-20T10:59:13.438 INFO:tasks.workunit.client.0.smithi027.stdout:1/321: dwrite d3/d7/d28/f59 [0,4194304] 0 2024-04-20T10:59:13.441 INFO:tasks.workunit.client.0.smithi027.stdout:8/303: dwrite d9/f34 [0,4194304] 0 2024-04-20T10:59:13.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/186: dread d4/d14/f19 [0,4194304] 0 2024-04-20T10:59:13.447 INFO:tasks.workunit.client.0.smithi027.stdout:5/197: dread f2 [0,4194304] 0 2024-04-20T10:59:13.447 INFO:tasks.workunit.client.0.smithi027.stdout:5/198: readlink l10 0 2024-04-20T10:59:13.697 INFO:tasks.workunit.client.0.smithi027.stdout:1/322: creat d3/d7/d38/f78 x:0 0 0 2024-04-20T10:59:13.698 INFO:tasks.workunit.client.0.smithi027.stdout:7/198: sync 2024-04-20T10:59:13.699 INFO:tasks.workunit.client.0.smithi027.stdout:8/304: link d9/de/f1b d9/de/d20/d26/d3c/f6f 0 2024-04-20T10:59:13.700 INFO:tasks.workunit.client.0.smithi027.stdout:1/323: symlink d3/d45/l79 0 2024-04-20T10:59:13.701 INFO:tasks.workunit.client.0.smithi027.stdout:9/187: rename d4/d14/f28 to d4/d14/f38 0 2024-04-20T10:59:13.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/324: unlink d3/c22 0 2024-04-20T10:59:13.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/325: chown d3/d7/d28/f59 59544 1 2024-04-20T10:59:13.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/326: creat d3/d7/f7a x:0 0 0 2024-04-20T10:59:13.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/327: write d3/d20/f4d [754260,21495] 0 2024-04-20T10:59:13.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/328: truncate d3/d7/f1f 1543031 0 2024-04-20T10:59:13.705 INFO:tasks.workunit.client.0.smithi027.stdout:2/151: sync 2024-04-20T10:59:13.705 INFO:tasks.workunit.client.0.smithi027.stdout:3/292: sync 2024-04-20T10:59:13.705 INFO:tasks.workunit.client.0.smithi027.stdout:4/249: sync 2024-04-20T10:59:13.706 INFO:tasks.workunit.client.0.smithi027.stdout:6/225: sync 2024-04-20T10:59:13.706 INFO:tasks.workunit.client.0.smithi027.stdout:5/199: sync 2024-04-20T10:59:13.706 INFO:tasks.workunit.client.0.smithi027.stdout:4/250: readlink d0/d14/d17/d1a/l4d 0 2024-04-20T10:59:13.706 INFO:tasks.workunit.client.0.smithi027.stdout:8/305: write d9/de/d1e/d49/f5c [2516646,8278] 0 2024-04-20T10:59:13.706 INFO:tasks.workunit.client.0.smithi027.stdout:1/329: truncate d3/d20/f3c 2973474 0 2024-04-20T10:59:13.708 INFO:tasks.workunit.client.0.smithi027.stdout:3/293: link d2/c2d d2/c56 0 2024-04-20T10:59:13.710 INFO:tasks.workunit.client.0.smithi027.stdout:2/152: truncate d5/d1e/d21/f27 664908 0 2024-04-20T10:59:13.710 INFO:tasks.workunit.client.0.smithi027.stdout:2/153: fsync d5/f17 0 2024-04-20T10:59:13.710 INFO:tasks.workunit.client.0.smithi027.stdout:6/226: symlink da/dd/l50 0 2024-04-20T10:59:13.712 INFO:tasks.workunit.client.0.smithi027.stdout:5/200: mknod d19/c48 0 2024-04-20T10:59:13.712 INFO:tasks.workunit.client.0.smithi027.stdout:5/201: chown cf 0 1 2024-04-20T10:59:13.712 INFO:tasks.workunit.client.0.smithi027.stdout:5/202: chown fb 396815 1 2024-04-20T10:59:13.712 INFO:tasks.workunit.client.0.smithi027.stdout:5/203: stat d19/d1b/d1c/f3b 0 2024-04-20T10:59:13.713 INFO:tasks.workunit.client.0.smithi027.stdout:0/204: write d0/d7/f1c [4088319,89283] 0 2024-04-20T10:59:13.714 INFO:tasks.workunit.client.0.smithi027.stdout:4/251: mkdir d0/d14/d6b 0 2024-04-20T10:59:13.715 INFO:tasks.workunit.client.0.smithi027.stdout:8/306: symlink d9/de/d20/d26/d3c/d4b/l70 0 2024-04-20T10:59:13.716 INFO:tasks.workunit.client.0.smithi027.stdout:2/154: mknod d5/d26/d30/c31 0 2024-04-20T10:59:13.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/199: dwrite d4/d5/f19 [0,4194304] 0 2024-04-20T10:59:13.722 INFO:tasks.workunit.client.0.smithi027.stdout:6/227: mkdir da/d21/d28/d2d/d51 0 2024-04-20T10:59:13.722 INFO:tasks.workunit.client.0.smithi027.stdout:5/204: symlink d19/d1b/d1c/l49 0 2024-04-20T10:59:13.723 INFO:tasks.workunit.client.0.smithi027.stdout:0/205: mknod d0/d11/d2c/d2e/d35/c38 0 2024-04-20T10:59:13.723 INFO:tasks.workunit.client.0.smithi027.stdout:5/205: read - d19/d1b/d1c/d1e/d25/f41 zero size 2024-04-20T10:59:13.723 INFO:tasks.workunit.client.0.smithi027.stdout:8/307: rename d9/de/f3e to d9/de/f71 0 2024-04-20T10:59:13.724 INFO:tasks.workunit.client.0.smithi027.stdout:2/155: mkdir d5/dc/d16/d32 0 2024-04-20T10:59:13.726 INFO:tasks.workunit.client.0.smithi027.stdout:7/200: creat d4/d5/d8/d21/f31 x:0 0 0 2024-04-20T10:59:13.728 INFO:tasks.workunit.client.0.smithi027.stdout:5/206: symlink d19/l4a 0 2024-04-20T10:59:13.728 INFO:tasks.workunit.client.0.smithi027.stdout:5/207: write d19/d1b/d1c/d1e/d3c/f3d [741879,103446] 0 2024-04-20T10:59:13.728 INFO:tasks.workunit.client.0.smithi027.stdout:5/208: dread - d19/d1b/d1c/f45 zero size 2024-04-20T10:59:13.728 INFO:tasks.workunit.client.0.smithi027.stdout:5/209: chown d19/d1b/d1c/d1e/d25 52 1 2024-04-20T10:59:13.729 INFO:tasks.workunit.client.0.smithi027.stdout:8/308: mknod d9/de/d20/d26/d3c/d4b/c72 0 2024-04-20T10:59:13.730 INFO:tasks.workunit.client.0.smithi027.stdout:9/188: write d4/d14/f38 [3017932,73547] 0 2024-04-20T10:59:13.730 INFO:tasks.workunit.client.0.smithi027.stdout:9/189: read d4/f7 [7362731,5702] 0 2024-04-20T10:59:13.730 INFO:tasks.workunit.client.0.smithi027.stdout:2/156: creat d5/d1e/d21/f33 x:0 0 0 2024-04-20T10:59:13.732 INFO:tasks.workunit.client.0.smithi027.stdout:0/206: mknod d0/d7/d12/c39 0 2024-04-20T10:59:13.733 INFO:tasks.workunit.client.0.smithi027.stdout:3/294: dwrite d2/d18/d2e/d41/f45 [0,4194304] 0 2024-04-20T10:59:13.733 INFO:tasks.workunit.client.0.smithi027.stdout:5/210: unlink d19/d1b/d1c/d1e/l36 0 2024-04-20T10:59:13.736 INFO:tasks.workunit.client.0.smithi027.stdout:2/157: rename d5/l24 to d5/dc/d16/d32/l34 0 2024-04-20T10:59:13.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/207: rename d0/d7/d12 to d0/d7/d12/d1a/d3a 22 2024-04-20T10:59:13.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/208: dread - d0/d7/f27 zero size 2024-04-20T10:59:13.738 INFO:tasks.workunit.client.0.smithi027.stdout:3/295: unlink d2/dd/df/f42 0 2024-04-20T10:59:13.738 INFO:tasks.workunit.client.0.smithi027.stdout:3/296: chown d2/d18/d2e/d41/c55 6633 1 2024-04-20T10:59:13.742 INFO:tasks.workunit.client.0.smithi027.stdout:5/211: rename cf to d19/d1b/d1c/d1e/d22/c4b 0 2024-04-20T10:59:13.743 INFO:tasks.workunit.client.0.smithi027.stdout:5/212: read f18 [540614,107066] 0 2024-04-20T10:59:13.744 INFO:tasks.workunit.client.0.smithi027.stdout:0/209: link d0/d7/f27 d0/d7/d34/f3b 0 2024-04-20T10:59:13.747 INFO:tasks.workunit.client.0.smithi027.stdout:3/297: mkdir d2/dd/d57 0 2024-04-20T10:59:13.747 INFO:tasks.workunit.client.0.smithi027.stdout:3/298: read d2/d18/f29 [258837,91684] 0 2024-04-20T10:59:13.747 INFO:tasks.workunit.client.0.smithi027.stdout:3/299: write d2/dd/d22/f2f [4800936,21028] 0 2024-04-20T10:59:13.747 INFO:tasks.workunit.client.0.smithi027.stdout:3/300: dread - d2/d18/d2c/f4f zero size 2024-04-20T10:59:13.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/213: rmdir d19/d1b/d1c/d1e 39 2024-04-20T10:59:13.748 INFO:tasks.workunit.client.0.smithi027.stdout:0/210: stat d0/d7/d12/d1a/c23 0 2024-04-20T10:59:13.752 INFO:tasks.workunit.client.0.smithi027.stdout:4/252: dwrite d0/d14/d17/d18/f2d [0,4194304] 0 2024-04-20T10:59:13.755 INFO:tasks.workunit.client.0.smithi027.stdout:3/301: creat d2/dd/f58 x:0 0 0 2024-04-20T10:59:13.755 INFO:tasks.workunit.client.0.smithi027.stdout:5/214: mknod d19/c4c 0 2024-04-20T10:59:13.755 INFO:tasks.workunit.client.0.smithi027.stdout:0/211: rmdir d0/d7/d12/d1a 39 2024-04-20T10:59:13.757 INFO:tasks.workunit.client.0.smithi027.stdout:3/302: creat d2/dd/d57/f59 x:0 0 0 2024-04-20T10:59:13.757 INFO:tasks.workunit.client.0.smithi027.stdout:5/215: truncate d19/d1b/d1c/d1e/d3c/f39 292280 0 2024-04-20T10:59:13.757 INFO:tasks.workunit.client.0.smithi027.stdout:5/216: fdatasync d19/d1b/d1c/d1e/d3c/f24 0 2024-04-20T10:59:13.757 INFO:tasks.workunit.client.0.smithi027.stdout:0/212: mknod d0/d7/d34/c3c 0 2024-04-20T10:59:13.758 INFO:tasks.workunit.client.0.smithi027.stdout:3/303: creat d2/d18/d2e/d41/f5a x:0 0 0 2024-04-20T10:59:13.758 INFO:tasks.workunit.client.0.smithi027.stdout:2/158: dread d5/d1e/d21/f27 [0,4194304] 0 2024-04-20T10:59:13.759 INFO:tasks.workunit.client.0.smithi027.stdout:2/159: fsync f1 0 2024-04-20T10:59:13.760 INFO:tasks.workunit.client.0.smithi027.stdout:3/304: rename d2/f12 to d2/d18/f5b 0 2024-04-20T10:59:13.760 INFO:tasks.workunit.client.0.smithi027.stdout:3/305: chown d2/f1d 26 1 2024-04-20T10:59:13.761 INFO:tasks.workunit.client.0.smithi027.stdout:3/306: chown d2/dd/d57 448846 1 2024-04-20T10:59:13.761 INFO:tasks.workunit.client.0.smithi027.stdout:1/330: dwrite d3/d7/f57 [0,4194304] 0 2024-04-20T10:59:13.761 INFO:tasks.workunit.client.0.smithi027.stdout:2/160: unlink d5/fd 0 2024-04-20T10:59:13.763 INFO:tasks.workunit.client.0.smithi027.stdout:6/228: dwrite da/df/f33 [0,4194304] 0 2024-04-20T10:59:13.763 INFO:tasks.workunit.client.0.smithi027.stdout:9/190: dwrite d4/d1b/f24 [0,4194304] 0 2024-04-20T10:59:13.763 INFO:tasks.workunit.client.0.smithi027.stdout:9/191: truncate d4/d1b/f22 697467 0 2024-04-20T10:59:13.763 INFO:tasks.workunit.client.0.smithi027.stdout:7/201: dwrite d4/d5/f24 [0,4194304] 0 2024-04-20T10:59:13.764 INFO:tasks.workunit.client.0.smithi027.stdout:1/331: unlink d3/d7/d2d/c3b 0 2024-04-20T10:59:13.764 INFO:tasks.workunit.client.0.smithi027.stdout:1/332: write d3/d7/fc [923864,40551] 0 2024-04-20T10:59:13.764 INFO:tasks.workunit.client.0.smithi027.stdout:1/333: chown d3/d45/d5d 100094 1 2024-04-20T10:59:13.764 INFO:tasks.workunit.client.0.smithi027.stdout:3/307: symlink d2/d18/d2e/d41/l5c 0 2024-04-20T10:59:13.765 INFO:tasks.workunit.client.0.smithi027.stdout:8/309: dwrite d9/de/f2f [0,4194304] 0 2024-04-20T10:59:13.766 INFO:tasks.workunit.client.0.smithi027.stdout:9/192: rename d4/l10 to d4/l39 0 2024-04-20T10:59:13.769 INFO:tasks.workunit.client.0.smithi027.stdout:1/334: dread d3/d7/f34 [0,4194304] 0 2024-04-20T10:59:13.769 INFO:tasks.workunit.client.0.smithi027.stdout:2/161: write d5/f17 [3252578,117738] 0 2024-04-20T10:59:13.769 INFO:tasks.workunit.client.0.smithi027.stdout:2/162: truncate d5/d26/f2c 523301 0 2024-04-20T10:59:13.769 INFO:tasks.workunit.client.0.smithi027.stdout:7/202: getdents d4 0 2024-04-20T10:59:13.769 INFO:tasks.workunit.client.0.smithi027.stdout:0/213: fdatasync d0/f20 0 2024-04-20T10:59:13.769 INFO:tasks.workunit.client.0.smithi027.stdout:3/308: mknod d2/dd/df/d23/c5d 0 2024-04-20T10:59:13.772 INFO:tasks.workunit.client.0.smithi027.stdout:9/193: creat d4/d1b/d1f/f3a x:0 0 0 2024-04-20T10:59:13.772 INFO:tasks.workunit.client.0.smithi027.stdout:9/194: write d4/d1b/d31/d23/f36 [1170113,111879] 0 2024-04-20T10:59:13.774 INFO:tasks.workunit.client.0.smithi027.stdout:2/163: link d5/d26/d30/c31 d5/dc/d16/c35 0 2024-04-20T10:59:13.774 INFO:tasks.workunit.client.0.smithi027.stdout:2/164: chown d5/dc/l1d 906936 1 2024-04-20T10:59:13.774 INFO:tasks.workunit.client.0.smithi027.stdout:2/165: rename d5/dc/d16 to d5/dc/d16/d36 22 2024-04-20T10:59:13.774 INFO:tasks.workunit.client.0.smithi027.stdout:2/166: chown d5/d26/l2d 717164991 1 2024-04-20T10:59:13.774 INFO:tasks.workunit.client.0.smithi027.stdout:2/167: fdatasync f1 0 2024-04-20T10:59:13.774 INFO:tasks.workunit.client.0.smithi027.stdout:8/310: dread d9/f24 [0,4194304] 0 2024-04-20T10:59:13.774 INFO:tasks.workunit.client.0.smithi027.stdout:8/311: write d9/de/d20/d26/d3c/f44 [844531,27700] 0 2024-04-20T10:59:13.774 INFO:tasks.workunit.client.0.smithi027.stdout:8/312: stat d9/f54 0 2024-04-20T10:59:13.777 INFO:tasks.workunit.client.0.smithi027.stdout:7/203: rename d4/f17 to d4/f32 0 2024-04-20T10:59:13.777 INFO:tasks.workunit.client.0.smithi027.stdout:0/214: truncate d0/f1f 227634 0 2024-04-20T10:59:13.778 INFO:tasks.workunit.client.0.smithi027.stdout:0/215: stat d0/d7/d12/l22 0 2024-04-20T10:59:13.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/217: dwrite d19/d1b/d1c/f45 [0,4194304] 0 2024-04-20T10:59:13.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/218: chown d19/d1b/d1c/d1e/d22/d47 84011 1 2024-04-20T10:59:13.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/219: fdatasync d19/d1b/d1c/d1e/d22/f38 0 2024-04-20T10:59:13.779 INFO:tasks.workunit.client.0.smithi027.stdout:5/220: dread d19/d1b/f33 [0,4194304] 0 2024-04-20T10:59:13.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/229: dread da/df/d1f/f3f [0,4194304] 0 2024-04-20T10:59:13.780 INFO:tasks.workunit.client.0.smithi027.stdout:9/195: mkdir d4/d1b/d31/d23/d3b 0 2024-04-20T10:59:13.780 INFO:tasks.workunit.client.0.smithi027.stdout:9/196: write d4/d14/f27 [987587,115972] 0 2024-04-20T10:59:13.780 INFO:tasks.workunit.client.0.smithi027.stdout:9/197: write d4/d14/f21 [674664,109329] 0 2024-04-20T10:59:13.780 INFO:tasks.workunit.client.0.smithi027.stdout:9/198: write d4/f7 [5159707,74314] 0 2024-04-20T10:59:13.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/253: dwrite d0/d14/d17/d1b/f30 [0,4194304] 0 2024-04-20T10:59:13.781 INFO:tasks.workunit.client.0.smithi027.stdout:1/335: dread d3/d7/d28/f59 [0,4194304] 0 2024-04-20T10:59:13.783 INFO:tasks.workunit.client.0.smithi027.stdout:7/204: write d4/f2e [968004,112655] 0 2024-04-20T10:59:13.786 INFO:tasks.workunit.client.0.smithi027.stdout:6/230: mknod da/d21/d28/d2d/d51/c52 0 2024-04-20T10:59:13.786 INFO:tasks.workunit.client.0.smithi027.stdout:5/221: getdents d19/d1b/d1c/d1e/d22 0 2024-04-20T10:59:13.788 INFO:tasks.workunit.client.0.smithi027.stdout:9/199: creat d4/d1b/d31/f3c x:0 0 0 2024-04-20T10:59:13.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/309: dwrite d2/f1f [0,4194304] 0 2024-04-20T10:59:13.794 INFO:tasks.workunit.client.0.smithi027.stdout:4/254: mkdir d0/d5e/d6c 0 2024-04-20T10:59:13.794 INFO:tasks.workunit.client.0.smithi027.stdout:4/255: fdatasync d0/d14/d17/d1b/d33/f35 0 2024-04-20T10:59:13.794 INFO:tasks.workunit.client.0.smithi027.stdout:5/222: dread d19/d1b/d1c/f45 [0,4194304] 0 2024-04-20T10:59:13.794 INFO:tasks.workunit.client.0.smithi027.stdout:5/223: chown d19/d1b/d1c/d1e/d25/f41 3042 1 2024-04-20T10:59:13.794 INFO:tasks.workunit.client.0.smithi027.stdout:5/224: fdatasync d19/d1b/d1c/d1e/d25/f2f 0 2024-04-20T10:59:13.794 INFO:tasks.workunit.client.0.smithi027.stdout:1/336: mknod d3/d45/c7b 0 2024-04-20T10:59:13.795 INFO:tasks.workunit.client.0.smithi027.stdout:0/216: dwrite d0/d7/f26 [0,4194304] 0 2024-04-20T10:59:13.795 INFO:tasks.workunit.client.0.smithi027.stdout:5/225: dread d19/d1b/d1c/d1e/d3c/f31 [0,4194304] 0 2024-04-20T10:59:13.818 INFO:tasks.workunit.client.0.smithi027.stdout:5/226: dwrite d19/d1b/d1c/d1e/d3c/f46 [0,4194304] 0 2024-04-20T10:59:14.050 INFO:tasks.workunit.client.0.smithi027.stdout:6/231: mkdir da/d21/d28/d2d/d53 0 2024-04-20T10:59:14.050 INFO:tasks.workunit.client.0.smithi027.stdout:3/310: mknod d2/d18/d2e/c5e 0 2024-04-20T10:59:14.050 INFO:tasks.workunit.client.0.smithi027.stdout:4/256: getdents d0/d14/d17/d1a 0 2024-04-20T10:59:14.050 INFO:tasks.workunit.client.0.smithi027.stdout:9/200: truncate d4/d14/f19 2899725 0 2024-04-20T10:59:14.050 INFO:tasks.workunit.client.0.smithi027.stdout:1/337: getdents d3/d7/d28 0 2024-04-20T10:59:14.050 INFO:tasks.workunit.client.0.smithi027.stdout:8/313: truncate d9/de/d20/d26/d3c/f44 670411 0 2024-04-20T10:59:14.050 INFO:tasks.workunit.client.0.smithi027.stdout:0/217: mkdir d0/d11/d2c/d2e/d2b/d3d 0 2024-04-20T10:59:14.051 INFO:tasks.workunit.client.0.smithi027.stdout:3/311: dread d2/d18/d2e/f35 [0,4194304] 0 2024-04-20T10:59:14.051 INFO:tasks.workunit.client.0.smithi027.stdout:3/312: write d2/f17 [9068170,63122] 0 2024-04-20T10:59:14.051 INFO:tasks.workunit.client.0.smithi027.stdout:5/227: creat d19/f4d x:0 0 0 2024-04-20T10:59:14.052 INFO:tasks.workunit.client.0.smithi027.stdout:5/228: fdatasync f18 0 2024-04-20T10:59:14.062 INFO:tasks.workunit.client.0.smithi027.stdout:6/232: truncate da/d21/d28/d2d/f2f 943530 0 2024-04-20T10:59:14.063 INFO:tasks.workunit.client.0.smithi027.stdout:9/201: creat d4/d1b/d31/f3d x:0 0 0 2024-04-20T10:59:14.063 INFO:tasks.workunit.client.0.smithi027.stdout:9/202: fsync d4/d14/f19 0 2024-04-20T10:59:14.063 INFO:tasks.workunit.client.0.smithi027.stdout:9/203: stat d4/f1e 0 2024-04-20T10:59:14.064 INFO:tasks.workunit.client.0.smithi027.stdout:1/338: link d3/d7/c19 d3/d7/d38/c7c 0 2024-04-20T10:59:14.065 INFO:tasks.workunit.client.0.smithi027.stdout:8/314: truncate f2 4063228 0 2024-04-20T10:59:14.065 INFO:tasks.workunit.client.0.smithi027.stdout:8/315: chown d9/de/f71 31550 1 2024-04-20T10:59:14.066 INFO:tasks.workunit.client.0.smithi027.stdout:5/229: mknod d19/c4e 0 2024-04-20T10:59:14.068 INFO:tasks.workunit.client.0.smithi027.stdout:7/205: rename d4/d1b/f27 to d4/d5/f33 0 2024-04-20T10:59:14.069 INFO:tasks.workunit.client.0.smithi027.stdout:6/233: mkdir da/d21/d28/d54 0 2024-04-20T10:59:14.069 INFO:tasks.workunit.client.0.smithi027.stdout:9/204: creat d4/d1b/d31/d23/d29/f3e x:0 0 0 2024-04-20T10:59:14.071 INFO:tasks.workunit.client.0.smithi027.stdout:1/339: link d3/f1c d3/d7/d2d/d73/f7d 0 2024-04-20T10:59:14.072 INFO:tasks.workunit.client.0.smithi027.stdout:8/316: creat d9/de/d20/f73 x:0 0 0 2024-04-20T10:59:14.072 INFO:tasks.workunit.client.0.smithi027.stdout:3/313: dwrite d2/dd/df/f46 [4194304,4194304] 0 2024-04-20T10:59:14.075 INFO:tasks.workunit.client.0.smithi027.stdout:5/230: rename d19/d1b/d1c/d1e/d25/f41 to d19/f4f 0 2024-04-20T10:59:14.075 INFO:tasks.workunit.client.0.smithi027.stdout:5/231: chown d19/d1b/d1c/d1e 15203771 1 2024-04-20T10:59:14.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/206: mknod d4/c34 0 2024-04-20T10:59:14.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/207: chown d4/d5/d8/c15 68 1 2024-04-20T10:59:14.080 INFO:tasks.workunit.client.0.smithi027.stdout:4/257: dwrite d0/d14/d17/d1b/f61 [0,4194304] 0 2024-04-20T10:59:14.082 INFO:tasks.workunit.client.0.smithi027.stdout:9/205: write d4/d1b/f24 [2169052,41490] 0 2024-04-20T10:59:14.082 INFO:tasks.workunit.client.0.smithi027.stdout:5/232: rename d19/d1b/d1c/d1e/f3f to d19/d1b/d1c/d1e/d22/d47/f50 0 2024-04-20T10:59:14.084 INFO:tasks.workunit.client.0.smithi027.stdout:0/218: dread d0/d11/f21 [0,4194304] 0 2024-04-20T10:59:14.086 INFO:tasks.workunit.client.0.smithi027.stdout:7/208: unlink d4/c13 0 2024-04-20T10:59:14.086 INFO:tasks.workunit.client.0.smithi027.stdout:2/168: sync 2024-04-20T10:59:14.086 INFO:tasks.workunit.client.0.smithi027.stdout:2/169: fdatasync d5/d1e/d21/f33 0 2024-04-20T10:59:14.088 INFO:tasks.workunit.client.0.smithi027.stdout:4/258: rename d0/d14/l34 to d0/d14/d17/d1a/d3b/l6d 0 2024-04-20T10:59:14.092 INFO:tasks.workunit.client.0.smithi027.stdout:0/219: unlink d0/d7/d12/l22 0 2024-04-20T10:59:14.093 INFO:tasks.workunit.client.0.smithi027.stdout:0/220: dread - d0/d7/f27 zero size 2024-04-20T10:59:14.093 INFO:tasks.workunit.client.0.smithi027.stdout:7/209: unlink d4/d5/d8/d21/c22 0 2024-04-20T10:59:14.095 INFO:tasks.workunit.client.0.smithi027.stdout:9/206: rename d4/d1b/d1f/f3a to d4/d1b/d31/d23/d3b/f3f 0 2024-04-20T10:59:14.095 INFO:tasks.workunit.client.0.smithi027.stdout:9/207: chown d4/d1b/d1f/l26 54 1 2024-04-20T10:59:14.099 INFO:tasks.workunit.client.0.smithi027.stdout:2/170: write d5/dc/f1c [1989518,32588] 0 2024-04-20T10:59:14.100 INFO:tasks.workunit.client.0.smithi027.stdout:6/234: dwrite da/d21/d28/f3a [0,4194304] 0 2024-04-20T10:59:14.100 INFO:tasks.workunit.client.0.smithi027.stdout:0/221: creat d0/d7/d12/f3e x:0 0 0 2024-04-20T10:59:14.101 INFO:tasks.workunit.client.0.smithi027.stdout:7/210: rename d4/f32 to d4/d1b/f35 0 2024-04-20T10:59:14.101 INFO:tasks.workunit.client.0.smithi027.stdout:7/211: write d4/d1b/f20 [4564808,10322] 0 2024-04-20T10:59:14.102 INFO:tasks.workunit.client.0.smithi027.stdout:2/171: dread d5/d1e/d21/f27 [0,4194304] 0 2024-04-20T10:59:14.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/212: read d4/fc [3455341,6983] 0 2024-04-20T10:59:14.108 INFO:tasks.workunit.client.0.smithi027.stdout:3/314: dwrite d2/d18/d2e/f35 [0,4194304] 0 2024-04-20T10:59:14.109 INFO:tasks.workunit.client.0.smithi027.stdout:2/172: mkdir d5/dc/d16/d37 0 2024-04-20T10:59:14.110 INFO:tasks.workunit.client.0.smithi027.stdout:3/315: dread d2/dd/df/d33/f3c [0,4194304] 0 2024-04-20T10:59:14.111 INFO:tasks.workunit.client.0.smithi027.stdout:3/316: chown d2/d18/d2c/f4f 7 1 2024-04-20T10:59:14.111 INFO:tasks.workunit.client.0.smithi027.stdout:0/222: link d0/d7/d12/l17 d0/d7/d12/d1a/l3f 0 2024-04-20T10:59:14.113 INFO:tasks.workunit.client.0.smithi027.stdout:7/213: dread d4/f2e [0,4194304] 0 2024-04-20T10:59:14.113 INFO:tasks.workunit.client.0.smithi027.stdout:1/340: dwrite d3/d7/f57 [0,4194304] 0 2024-04-20T10:59:14.113 INFO:tasks.workunit.client.0.smithi027.stdout:5/233: dwrite d19/d1b/d1c/d1e/d22/f42 [0,4194304] 0 2024-04-20T10:59:14.114 INFO:tasks.workunit.client.0.smithi027.stdout:5/234: chown fb 793104 1 2024-04-20T10:59:14.114 INFO:tasks.workunit.client.0.smithi027.stdout:2/173: unlink d5/dc/d16/d32/l34 0 2024-04-20T10:59:14.114 INFO:tasks.workunit.client.0.smithi027.stdout:6/235: read da/df/d19/d2e/f45 [1668571,43724] 0 2024-04-20T10:59:14.115 INFO:tasks.workunit.client.0.smithi027.stdout:3/317: symlink d2/dd/df/d23/l5f 0 2024-04-20T10:59:14.115 INFO:tasks.workunit.client.0.smithi027.stdout:7/214: fdatasync d4/d1b/ff 0 2024-04-20T10:59:14.115 INFO:tasks.workunit.client.0.smithi027.stdout:7/215: readlink d4/d1b/l2b 0 2024-04-20T10:59:14.115 INFO:tasks.workunit.client.0.smithi027.stdout:7/216: stat d4/d5/d8 0 2024-04-20T10:59:14.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/223: symlink d0/d7/d12/d1a/l40 0 2024-04-20T10:59:14.117 INFO:tasks.workunit.client.0.smithi027.stdout:0/224: truncate d0/d7/d34/f3b 603732 0 2024-04-20T10:59:14.122 INFO:tasks.workunit.client.0.smithi027.stdout:1/341: mkdir d3/d7e 0 2024-04-20T10:59:14.122 INFO:tasks.workunit.client.0.smithi027.stdout:5/235: dread f2 [4194304,4194304] 0 2024-04-20T10:59:14.122 INFO:tasks.workunit.client.0.smithi027.stdout:1/342: truncate d3/d7/d28/d35/f5e 867126 0 2024-04-20T10:59:14.122 INFO:tasks.workunit.client.0.smithi027.stdout:5/236: write d19/d1b/f43 [690166,76289] 0 2024-04-20T10:59:14.122 INFO:tasks.workunit.client.0.smithi027.stdout:7/217: dread d4/d5/f1f [0,4194304] 0 2024-04-20T10:59:14.124 INFO:tasks.workunit.client.0.smithi027.stdout:3/318: mknod d2/dd/df/d23/c60 0 2024-04-20T10:59:14.124 INFO:tasks.workunit.client.0.smithi027.stdout:3/319: chown d2/dd/df/f3a 10113825 1 2024-04-20T10:59:14.125 INFO:tasks.workunit.client.0.smithi027.stdout:1/343: getdents d3/d7/d2d 0 2024-04-20T10:59:14.126 INFO:tasks.workunit.client.0.smithi027.stdout:5/237: mknod d19/c51 0 2024-04-20T10:59:14.127 INFO:tasks.workunit.client.0.smithi027.stdout:7/218: symlink d4/d5/d8/l36 0 2024-04-20T10:59:14.127 INFO:tasks.workunit.client.0.smithi027.stdout:7/219: truncate d4/d5/d8/f2f 214859 0 2024-04-20T10:59:14.128 INFO:tasks.workunit.client.0.smithi027.stdout:6/236: creat da/df/d19/d2e/d32/f55 x:0 0 0 2024-04-20T10:59:14.128 INFO:tasks.workunit.client.0.smithi027.stdout:6/237: chown da/df/d1f 53 1 2024-04-20T10:59:14.129 INFO:tasks.workunit.client.0.smithi027.stdout:3/320: rename d2/dd/df/d33/c38 to d2/dd/d22/c61 0 2024-04-20T10:59:14.130 INFO:tasks.workunit.client.0.smithi027.stdout:1/344: getdents d3/d7/d38/d62 0 2024-04-20T10:59:14.131 INFO:tasks.workunit.client.0.smithi027.stdout:5/238: creat d19/f52 x:0 0 0 2024-04-20T10:59:14.131 INFO:tasks.workunit.client.0.smithi027.stdout:5/239: write f17 [4836268,120243] 0 2024-04-20T10:59:14.132 INFO:tasks.workunit.client.0.smithi027.stdout:7/220: mkdir d4/d1b/d37 0 2024-04-20T10:59:14.133 INFO:tasks.workunit.client.0.smithi027.stdout:6/238: unlink da/dd/d3c/l4b 0 2024-04-20T10:59:14.135 INFO:tasks.workunit.client.0.smithi027.stdout:3/321: rmdir d2/d18/d3e 0 2024-04-20T10:59:14.135 INFO:tasks.workunit.client.0.smithi027.stdout:3/322: fdatasync d2/d18/f5b 0 2024-04-20T10:59:14.135 INFO:tasks.workunit.client.0.smithi027.stdout:3/323: stat d2/dd/d22/l40 0 2024-04-20T10:59:14.136 INFO:tasks.workunit.client.0.smithi027.stdout:5/240: getdents d19/d1b/d1c/d1e/d3c 0 2024-04-20T10:59:14.136 INFO:tasks.workunit.client.0.smithi027.stdout:5/241: stat d19/d1b/d1c/d1e/d22/d47 0 2024-04-20T10:59:14.136 INFO:tasks.workunit.client.0.smithi027.stdout:2/174: dwrite d5/d1e/f22 [4194304,4194304] 0 2024-04-20T10:59:14.137 INFO:tasks.workunit.client.0.smithi027.stdout:7/221: chown d4/c30 495767 1 2024-04-20T10:59:14.137 INFO:tasks.workunit.client.0.smithi027.stdout:6/239: mknod da/df/c56 0 2024-04-20T10:59:14.137 INFO:tasks.workunit.client.0.smithi027.stdout:6/240: chown da/d21/d28/d2d/d53 17330679 1 2024-04-20T10:59:14.138 INFO:tasks.workunit.client.0.smithi027.stdout:0/225: dwrite d0/f20 [4194304,4194304] 0 2024-04-20T10:59:14.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/324: rename d2/d18/l31 to d2/d18/d2c/l62 0 2024-04-20T10:59:14.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/325: fsync d2/d18/f53 0 2024-04-20T10:59:14.139 INFO:tasks.workunit.client.0.smithi027.stdout:5/242: mknod d19/d1b/c53 0 2024-04-20T10:59:14.139 INFO:tasks.workunit.client.0.smithi027.stdout:5/243: write d19/d1b/d1c/d1e/d3c/f24 [751671,76821] 0 2024-04-20T10:59:14.139 INFO:tasks.workunit.client.0.smithi027.stdout:5/244: truncate d19/d1b/d1c/f3b 957303 0 2024-04-20T10:59:14.141 INFO:tasks.workunit.client.0.smithi027.stdout:7/222: creat d4/d5/d8/f38 x:0 0 0 2024-04-20T10:59:14.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/326: write d2/d18/f3f [972860,60009] 0 2024-04-20T10:59:14.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/327: chown d2/dd/f19 13 1 2024-04-20T10:59:14.148 INFO:tasks.workunit.client.0.smithi027.stdout:1/345: dwrite d3/d7/f34 [0,4194304] 0 2024-04-20T10:59:14.394 INFO:tasks.workunit.client.0.smithi027.stdout:6/241: unlink da/dd/f13 0 2024-04-20T10:59:14.394 INFO:tasks.workunit.client.0.smithi027.stdout:0/226: rename d0/d7/d12/d1a/c2f to d0/d7/d12/d1a/c41 0 2024-04-20T10:59:14.394 INFO:tasks.workunit.client.0.smithi027.stdout:6/242: write da/df/f17 [3809271,30827] 0 2024-04-20T10:59:14.394 INFO:tasks.workunit.client.0.smithi027.stdout:6/243: truncate da/df/f4d 526286 0 2024-04-20T10:59:14.394 INFO:tasks.workunit.client.0.smithi027.stdout:2/175: rename d5/c23 to d5/d26/c38 0 2024-04-20T10:59:14.394 INFO:tasks.workunit.client.0.smithi027.stdout:6/244: read da/fb [230235,70323] 0 2024-04-20T10:59:14.394 INFO:tasks.workunit.client.0.smithi027.stdout:6/245: fsync da/df/f33 0 2024-04-20T10:59:14.394 INFO:tasks.workunit.client.0.smithi027.stdout:5/245: symlink d19/d1b/d1c/d1e/l54 0 2024-04-20T10:59:14.395 INFO:tasks.workunit.client.0.smithi027.stdout:7/223: mknod d4/d1b/d37/c39 0 2024-04-20T10:59:14.395 INFO:tasks.workunit.client.0.smithi027.stdout:3/328: creat d2/d18/f63 x:0 0 0 2024-04-20T10:59:14.396 INFO:tasks.workunit.client.0.smithi027.stdout:1/346: mknod d3/d7/d2d/c7f 0 2024-04-20T10:59:14.397 INFO:tasks.workunit.client.0.smithi027.stdout:1/347: read d3/f1c [2047,27349] 0 2024-04-20T10:59:14.401 INFO:tasks.workunit.client.0.smithi027.stdout:0/227: creat d0/f42 x:0 0 0 2024-04-20T10:59:14.402 INFO:tasks.workunit.client.0.smithi027.stdout:0/228: chown d0/d11 7 1 2024-04-20T10:59:14.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/176: symlink d5/d26/d30/l39 0 2024-04-20T10:59:14.405 INFO:tasks.workunit.client.0.smithi027.stdout:6/246: unlink da/d21/d28/d2d/f2f 0 2024-04-20T10:59:14.407 INFO:tasks.workunit.client.0.smithi027.stdout:7/224: truncate d4/d1b/f20 1991459 0 2024-04-20T10:59:14.407 INFO:tasks.workunit.client.0.smithi027.stdout:7/225: chown d4/d5/f19 200 1 2024-04-20T10:59:14.408 INFO:tasks.workunit.client.0.smithi027.stdout:1/348: rmdir d3/d20 39 2024-04-20T10:59:14.410 INFO:tasks.workunit.client.0.smithi027.stdout:0/229: creat d0/d7/d34/f43 x:0 0 0 2024-04-20T10:59:14.411 INFO:tasks.workunit.client.0.smithi027.stdout:2/177: creat d5/d26/f3a x:0 0 0 2024-04-20T10:59:14.411 INFO:tasks.workunit.client.0.smithi027.stdout:2/178: dread - d5/d1e/d21/f33 zero size 2024-04-20T10:59:14.413 INFO:tasks.workunit.client.0.smithi027.stdout:7/226: stat d4/c1d 0 2024-04-20T10:59:14.414 INFO:tasks.workunit.client.0.smithi027.stdout:0/230: creat d0/d7/d12/f44 x:0 0 0 2024-04-20T10:59:14.416 INFO:tasks.workunit.client.0.smithi027.stdout:2/179: creat d5/d26/f3b x:0 0 0 2024-04-20T10:59:14.416 INFO:tasks.workunit.client.0.smithi027.stdout:2/180: chown d5/dc/d16/c35 2 1 2024-04-20T10:59:14.418 INFO:tasks.workunit.client.0.smithi027.stdout:0/231: truncate d0/d7/d12/d1a/f2d 2544919 0 2024-04-20T10:59:14.419 INFO:tasks.workunit.client.0.smithi027.stdout:5/246: dwrite f0 [8388608,4194304] 0 2024-04-20T10:59:14.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/247: chown d19/d1b/d1c/f45 291074123 1 2024-04-20T10:59:14.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/248: readlink d19/l2d 0 2024-04-20T10:59:14.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/249: write d19/d1b/d1c/d1e/d3c/f24 [335096,7128] 0 2024-04-20T10:59:14.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/181: creat d5/d26/d30/f3c x:0 0 0 2024-04-20T10:59:14.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/182: write d5/d1e/d21/f27 [815399,101566] 0 2024-04-20T10:59:14.422 INFO:tasks.workunit.client.0.smithi027.stdout:7/227: mknod d4/c3a 0 2024-04-20T10:59:14.422 INFO:tasks.workunit.client.0.smithi027.stdout:6/247: dread da/d21/d28/f3e [0,4194304] 0 2024-04-20T10:59:14.424 INFO:tasks.workunit.client.0.smithi027.stdout:2/183: mkdir d5/d26/d3d 0 2024-04-20T10:59:14.425 INFO:tasks.workunit.client.0.smithi027.stdout:9/208: sync 2024-04-20T10:59:14.425 INFO:tasks.workunit.client.0.smithi027.stdout:4/259: sync 2024-04-20T10:59:14.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/317: sync 2024-04-20T10:59:14.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/318: truncate d9/de/d20/d26/d3c/f44 838653 0 2024-04-20T10:59:14.426 INFO:tasks.workunit.client.0.smithi027.stdout:8/319: write d9/de/d20/d48/d59/f60 [1910713,97451] 0 2024-04-20T10:59:14.426 INFO:tasks.workunit.client.0.smithi027.stdout:8/320: write d9/de/d20/f73 [323010,40462] 0 2024-04-20T10:59:14.426 INFO:tasks.workunit.client.0.smithi027.stdout:8/321: readlink d9/de/d1e/d4a/l3d 0 2024-04-20T10:59:14.427 INFO:tasks.workunit.client.0.smithi027.stdout:3/329: dwrite d2/dd/df/d33/f3c [0,4194304] 0 2024-04-20T10:59:14.428 INFO:tasks.workunit.client.0.smithi027.stdout:9/209: rename d4/d1b/l34 to d4/d1b/d31/d23/d3b/l40 0 2024-04-20T10:59:14.429 INFO:tasks.workunit.client.0.smithi027.stdout:4/260: mknod d0/d14/d17/d1a/c6e 0 2024-04-20T10:59:14.429 INFO:tasks.workunit.client.0.smithi027.stdout:6/248: dread da/df/f33 [0,4194304] 0 2024-04-20T10:59:14.429 INFO:tasks.workunit.client.0.smithi027.stdout:6/249: truncate da/df/d1f/f3f 1118753 0 2024-04-20T10:59:14.429 INFO:tasks.workunit.client.0.smithi027.stdout:8/322: mkdir d9/de/d20/d48/d59/d74 0 2024-04-20T10:59:14.429 INFO:tasks.workunit.client.0.smithi027.stdout:1/349: dwrite d3/d7/f8 [0,4194304] 0 2024-04-20T10:59:14.431 INFO:tasks.workunit.client.0.smithi027.stdout:4/261: mkdir d0/d14/d17/d1b/d33/d5c/d6f 0 2024-04-20T10:59:14.431 INFO:tasks.workunit.client.0.smithi027.stdout:1/350: dread d3/d41/f61 [0,4194304] 0 2024-04-20T10:59:14.434 INFO:tasks.workunit.client.0.smithi027.stdout:8/323: mkdir d9/de/d1e/d4a/d6d/d75 0 2024-04-20T10:59:14.437 INFO:tasks.workunit.client.0.smithi027.stdout:7/228: dread d4/d5/f19 [0,4194304] 0 2024-04-20T10:59:14.437 INFO:tasks.workunit.client.0.smithi027.stdout:5/250: dwrite d19/f4f [0,4194304] 0 2024-04-20T10:59:14.442 INFO:tasks.workunit.client.0.smithi027.stdout:6/250: getdents da/d21/d28/d2d 0 2024-04-20T10:59:14.443 INFO:tasks.workunit.client.0.smithi027.stdout:6/251: readlink da/df/d19/l27 0 2024-04-20T10:59:14.443 INFO:tasks.workunit.client.0.smithi027.stdout:3/330: mknod d2/d18/d2e/d41/c64 0 2024-04-20T10:59:14.443 INFO:tasks.workunit.client.0.smithi027.stdout:2/184: fsync d5/d26/f3a 0 2024-04-20T10:59:14.443 INFO:tasks.workunit.client.0.smithi027.stdout:4/262: creat d0/d14/d17/d1a/f70 x:0 0 0 2024-04-20T10:59:14.443 INFO:tasks.workunit.client.0.smithi027.stdout:4/263: chown d0/d14/d17/f1e 38556836 1 2024-04-20T10:59:14.443 INFO:tasks.workunit.client.0.smithi027.stdout:5/251: write f2 [5761861,116061] 0 2024-04-20T10:59:14.447 INFO:tasks.workunit.client.0.smithi027.stdout:7/229: write d4/d5/f24 [3648926,3140] 0 2024-04-20T10:59:14.448 INFO:tasks.workunit.client.0.smithi027.stdout:0/232: getdents d0/d7/d12 0 2024-04-20T10:59:14.449 INFO:tasks.workunit.client.0.smithi027.stdout:1/351: creat d3/d41/f80 x:0 0 0 2024-04-20T10:59:14.452 INFO:tasks.workunit.client.0.smithi027.stdout:5/252: dread f3 [0,4194304] 0 2024-04-20T10:59:14.453 INFO:tasks.workunit.client.0.smithi027.stdout:5/253: readlink d19/d1b/d1c/d1e/l54 0 2024-04-20T10:59:14.455 INFO:tasks.workunit.client.0.smithi027.stdout:8/324: dwrite d9/de/d18/f21 [0,4194304] 0 2024-04-20T10:59:14.455 INFO:tasks.workunit.client.0.smithi027.stdout:2/185: creat d5/d1e/d21/f3e x:0 0 0 2024-04-20T10:59:14.456 INFO:tasks.workunit.client.0.smithi027.stdout:4/264: mkdir d0/d14/d71 0 2024-04-20T10:59:14.456 INFO:tasks.workunit.client.0.smithi027.stdout:8/325: dread d9/f39 [0,4194304] 0 2024-04-20T10:59:14.456 INFO:tasks.workunit.client.0.smithi027.stdout:8/326: truncate d9/de/d64/d6a/f67 3885 0 2024-04-20T10:59:14.456 INFO:tasks.workunit.client.0.smithi027.stdout:8/327: stat d9/de/f47 0 2024-04-20T10:59:14.457 INFO:tasks.workunit.client.0.smithi027.stdout:7/230: mkdir d4/d5/d8/d3b 0 2024-04-20T10:59:14.459 INFO:tasks.workunit.client.0.smithi027.stdout:0/233: symlink d0/d11/d2c/d2e/l45 0 2024-04-20T10:59:14.459 INFO:tasks.workunit.client.0.smithi027.stdout:6/252: symlink da/df/d19/d2e/l57 0 2024-04-20T10:59:14.460 INFO:tasks.workunit.client.0.smithi027.stdout:5/254: creat d19/d1b/d1c/d1e/d22/d47/f55 x:0 0 0 2024-04-20T10:59:14.461 INFO:tasks.workunit.client.0.smithi027.stdout:5/255: fdatasync d19/d1b/f43 0 2024-04-20T10:59:14.462 INFO:tasks.workunit.client.0.smithi027.stdout:4/265: creat d0/d14/d17/d18/f72 x:0 0 0 2024-04-20T10:59:14.462 INFO:tasks.workunit.client.0.smithi027.stdout:4/266: stat d0/d14/d17/d1b/f21 0 2024-04-20T10:59:14.462 INFO:tasks.workunit.client.0.smithi027.stdout:4/267: dread - d0/f37 zero size 2024-04-20T10:59:14.462 INFO:tasks.workunit.client.0.smithi027.stdout:2/186: dread d5/f17 [0,4194304] 0 2024-04-20T10:59:14.462 INFO:tasks.workunit.client.0.smithi027.stdout:2/187: dread - d5/d26/f3b zero size 2024-04-20T10:59:14.463 INFO:tasks.workunit.client.0.smithi027.stdout:8/328: mknod d9/de/d64/c76 0 2024-04-20T10:59:14.463 INFO:tasks.workunit.client.0.smithi027.stdout:7/231: link d4/c3a d4/d1b/d37/c3c 0 2024-04-20T10:59:14.464 INFO:tasks.workunit.client.0.smithi027.stdout:6/253: mkdir da/dd/d3c/d58 0 2024-04-20T10:59:14.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/256: creat d19/d1b/d1c/d1e/f56 x:0 0 0 2024-04-20T10:59:14.468 INFO:tasks.workunit.client.0.smithi027.stdout:5/257: creat d19/d1b/d1c/d1e/f57 x:0 0 0 2024-04-20T10:59:14.469 INFO:tasks.workunit.client.0.smithi027.stdout:6/254: getdents da/d21/d28 0 2024-04-20T10:59:14.469 INFO:tasks.workunit.client.0.smithi027.stdout:6/255: write da/df/d19/d2e/f45 [665297,2151] 0 2024-04-20T10:59:14.470 INFO:tasks.workunit.client.0.smithi027.stdout:8/329: write d9/de/f3a [1629174,25343] 0 2024-04-20T10:59:14.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/258: rmdir d19/d1b/d1c 39 2024-04-20T10:59:14.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/259: write d19/f4d [737521,66759] 0 2024-04-20T10:59:14.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/260: readlink d19/d1b/d1c/d1e/d3c/l35 0 2024-04-20T10:59:14.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/261: write d19/d1b/d1c/d1e/d22/d47/f55 [122208,23364] 0 2024-04-20T10:59:14.471 INFO:tasks.workunit.client.0.smithi027.stdout:3/331: dwrite d2/fe [0,4194304] 0 2024-04-20T10:59:14.471 INFO:tasks.workunit.client.0.smithi027.stdout:3/332: stat d2/d18/d2c/d4c 0 2024-04-20T10:59:14.471 INFO:tasks.workunit.client.0.smithi027.stdout:6/256: truncate da/fb 682306 0 2024-04-20T10:59:14.472 INFO:tasks.workunit.client.0.smithi027.stdout:1/352: dwrite d3/d7/f34 [0,4194304] 0 2024-04-20T10:59:14.472 INFO:tasks.workunit.client.0.smithi027.stdout:1/353: fsync d3/d45/d5d/d6c/f72 0 2024-04-20T10:59:14.472 INFO:tasks.workunit.client.0.smithi027.stdout:2/188: dread d5/f17 [0,4194304] 0 2024-04-20T10:59:14.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/232: read d4/d5/fa [3519481,19687] 0 2024-04-20T10:59:14.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/233: write d4/d5/d8/f38 [730359,65880] 0 2024-04-20T10:59:14.478 INFO:tasks.workunit.client.0.smithi027.stdout:1/354: dread d3/d7/d2d/f3f [0,4194304] 0 2024-04-20T10:59:14.478 INFO:tasks.workunit.client.0.smithi027.stdout:2/189: dread d5/f17 [0,4194304] 0 2024-04-20T10:59:14.479 INFO:tasks.workunit.client.0.smithi027.stdout:0/234: dwrite d0/f20 [0,4194304] 0 2024-04-20T10:59:14.480 INFO:tasks.workunit.client.0.smithi027.stdout:4/268: dwrite d0/f37 [0,4194304] 0 2024-04-20T10:59:14.480 INFO:tasks.workunit.client.0.smithi027.stdout:4/269: read - d0/d14/d17/d1b/d33/f57 zero size 2024-04-20T10:59:14.482 INFO:tasks.workunit.client.0.smithi027.stdout:3/333: dread d2/dd/df/d23/f50 [0,4194304] 0 2024-04-20T10:59:14.485 INFO:tasks.workunit.client.0.smithi027.stdout:0/235: write d0/d7/d12/d1a/f2d [1369569,127516] 0 2024-04-20T10:59:14.485 INFO:tasks.workunit.client.0.smithi027.stdout:0/236: truncate d0/f1f 657951 0 2024-04-20T10:59:14.486 INFO:tasks.workunit.client.0.smithi027.stdout:0/237: dread - d0/d11/d2c/d2e/f2a zero size 2024-04-20T10:59:14.729 INFO:tasks.workunit.client.0.smithi027.stdout:5/262: mkdir d19/d1b/d1c/d1e/d58 0 2024-04-20T10:59:14.730 INFO:tasks.workunit.client.0.smithi027.stdout:5/263: fdatasync d19/d1b/d1c/d1e/d3c/f3d 0 2024-04-20T10:59:14.730 INFO:tasks.workunit.client.0.smithi027.stdout:6/257: creat da/dd/f59 x:0 0 0 2024-04-20T10:59:14.731 INFO:tasks.workunit.client.0.smithi027.stdout:1/355: rename d3/d20/f3c to d3/d7/d38/d62/f81 0 2024-04-20T10:59:14.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/190: creat d5/dc/d16/d37/f3f x:0 0 0 2024-04-20T10:59:14.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/191: dread - d5/dc/d16/d37/f3f zero size 2024-04-20T10:59:14.734 INFO:tasks.workunit.client.0.smithi027.stdout:8/330: dread d9/de/d1e/d4a/f4e [4194304,4194304] 0 2024-04-20T10:59:14.734 INFO:tasks.workunit.client.0.smithi027.stdout:4/270: creat d0/d14/d17/d1b/d2f/f73 x:0 0 0 2024-04-20T10:59:14.734 INFO:tasks.workunit.client.0.smithi027.stdout:8/331: fsync d9/de/d1e/d4a/f35 0 2024-04-20T10:59:14.735 INFO:tasks.workunit.client.0.smithi027.stdout:3/334: mknod d2/dd/df/c65 0 2024-04-20T10:59:14.735 INFO:tasks.workunit.client.0.smithi027.stdout:3/335: stat d2/d18/d2c/d4c/c51 0 2024-04-20T10:59:14.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/238: symlink d0/d11/d2c/d2e/l46 0 2024-04-20T10:59:14.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/239: write d0/d7/d12/d1a/f2d [3061662,103992] 0 2024-04-20T10:59:14.738 INFO:tasks.workunit.client.0.smithi027.stdout:3/336: read d2/fe [687813,108909] 0 2024-04-20T10:59:14.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/264: mknod d19/d1b/d1c/d1e/d22/c59 0 2024-04-20T10:59:14.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/265: write d19/d1b/d1c/d1e/d25/f2f [740149,46644] 0 2024-04-20T10:59:14.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/266: read d19/d1b/d1c/d1e/d25/f2f [762024,22344] 0 2024-04-20T10:59:14.747 INFO:tasks.workunit.client.0.smithi027.stdout:5/267: write d19/d1b/d1c/d1e/f57 [290473,89300] 0 2024-04-20T10:59:14.748 INFO:tasks.workunit.client.0.smithi027.stdout:3/337: write d2/dd/df/d33/f3c [4160084,49025] 0 2024-04-20T10:59:14.749 INFO:tasks.workunit.client.0.smithi027.stdout:1/356: link d3/d45/d5d/c67 d3/d45/d5d/c82 0 2024-04-20T10:59:14.750 INFO:tasks.workunit.client.0.smithi027.stdout:8/332: truncate d9/de/d18/f22 1628266 0 2024-04-20T10:59:14.751 INFO:tasks.workunit.client.0.smithi027.stdout:4/271: creat d0/d14/d17/d1b/d33/d5c/d6f/f74 x:0 0 0 2024-04-20T10:59:14.753 INFO:tasks.workunit.client.0.smithi027.stdout:0/240: rmdir d0/d7/d12/d1a 39 2024-04-20T10:59:14.755 INFO:tasks.workunit.client.0.smithi027.stdout:6/258: symlink da/l5a 0 2024-04-20T10:59:14.755 INFO:tasks.workunit.client.0.smithi027.stdout:6/259: truncate da/df/f4f 803550 0 2024-04-20T10:59:14.758 INFO:tasks.workunit.client.0.smithi027.stdout:3/338: getdents d2/d18/d2e/d41 0 2024-04-20T10:59:14.759 INFO:tasks.workunit.client.0.smithi027.stdout:1/357: creat d3/d7e/f83 x:0 0 0 2024-04-20T10:59:14.760 INFO:tasks.workunit.client.0.smithi027.stdout:5/268: dwrite d19/d1b/f43 [0,4194304] 0 2024-04-20T10:59:14.761 INFO:tasks.workunit.client.0.smithi027.stdout:8/333: creat d9/de/d20/d53/f77 x:0 0 0 2024-04-20T10:59:14.761 INFO:tasks.workunit.client.0.smithi027.stdout:0/241: dread d0/d11/f32 [0,4194304] 0 2024-04-20T10:59:14.762 INFO:tasks.workunit.client.0.smithi027.stdout:4/272: symlink d0/d48/d68/l75 0 2024-04-20T10:59:14.762 INFO:tasks.workunit.client.0.smithi027.stdout:4/273: write d0/d14/d17/d1b/d33/f57 [503393,2523] 0 2024-04-20T10:59:14.764 INFO:tasks.workunit.client.0.smithi027.stdout:6/260: dread da/d21/d28/f34 [0,4194304] 0 2024-04-20T10:59:14.764 INFO:tasks.workunit.client.0.smithi027.stdout:3/339: unlink d2/dd/df/f46 0 2024-04-20T10:59:14.764 INFO:tasks.workunit.client.0.smithi027.stdout:6/261: write da/df/d19/d2e/d42/f4e [622798,115727] 0 2024-04-20T10:59:14.764 INFO:tasks.workunit.client.0.smithi027.stdout:6/262: write da/df/d19/d2e/d42/f4e [2917563,401] 0 2024-04-20T10:59:14.766 INFO:tasks.workunit.client.0.smithi027.stdout:1/358: creat d3/d7/d2d/f84 x:0 0 0 2024-04-20T10:59:14.768 INFO:tasks.workunit.client.0.smithi027.stdout:7/234: truncate d4/d5/d8/f38 722875 0 2024-04-20T10:59:14.769 INFO:tasks.workunit.client.0.smithi027.stdout:0/242: mkdir d0/d7/d12/d1a/d47 0 2024-04-20T10:59:14.769 INFO:tasks.workunit.client.0.smithi027.stdout:0/243: truncate d0/d7/f26 4516060 0 2024-04-20T10:59:14.770 INFO:tasks.workunit.client.0.smithi027.stdout:5/269: read d19/d1b/d1c/d1e/d22/f30 [3226108,25469] 0 2024-04-20T10:59:14.770 INFO:tasks.workunit.client.0.smithi027.stdout:5/270: truncate d19/d1b/d1c/d1e/f56 787422 0 2024-04-20T10:59:14.770 INFO:tasks.workunit.client.0.smithi027.stdout:5/271: truncate d19/d1b/d1c/d1e/d3c/f31 756659 0 2024-04-20T10:59:14.770 INFO:tasks.workunit.client.0.smithi027.stdout:4/274: mkdir d0/d48/d68/d76 0 2024-04-20T10:59:14.770 INFO:tasks.workunit.client.0.smithi027.stdout:4/275: chown d0/d14/d17/d53/l5b 12 1 2024-04-20T10:59:14.771 INFO:tasks.workunit.client.0.smithi027.stdout:0/244: dread d0/d11/d2c/d2e/f1d [0,4194304] 0 2024-04-20T10:59:14.772 INFO:tasks.workunit.client.0.smithi027.stdout:3/340: creat d2/f66 x:0 0 0 2024-04-20T10:59:14.772 INFO:tasks.workunit.client.0.smithi027.stdout:7/235: dread d4/d5/f14 [0,4194304] 0 2024-04-20T10:59:14.773 INFO:tasks.workunit.client.0.smithi027.stdout:6/263: chown da/f2a 6 1 2024-04-20T10:59:14.773 INFO:tasks.workunit.client.0.smithi027.stdout:6/264: getdents da/d21/d28/d2d 0 2024-04-20T10:59:14.775 INFO:tasks.workunit.client.0.smithi027.stdout:5/272: creat d19/d1b/d1c/d1e/d22/f5a x:0 0 0 2024-04-20T10:59:14.785 INFO:tasks.workunit.client.0.smithi027.stdout:4/276: read d0/d14/d17/d1b/f30 [1739491,83200] 0 2024-04-20T10:59:14.785 INFO:tasks.workunit.client.0.smithi027.stdout:0/245: rename d0/d7/c18 to d0/d7/d12/c48 0 2024-04-20T10:59:14.785 INFO:tasks.workunit.client.0.smithi027.stdout:6/265: creat da/d21/f5b x:0 0 0 2024-04-20T10:59:14.786 INFO:tasks.workunit.client.0.smithi027.stdout:4/277: mknod d0/d14/d17/d1b/c77 0 2024-04-20T10:59:14.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/246: mknod d0/d7/d12/d25/c49 0 2024-04-20T10:59:14.791 INFO:tasks.workunit.client.0.smithi027.stdout:4/278: unlink d0/d14/d17/d1b/l4e 0 2024-04-20T10:59:14.791 INFO:tasks.workunit.client.0.smithi027.stdout:4/279: fsync d0/f28 0 2024-04-20T10:59:14.792 INFO:tasks.workunit.client.0.smithi027.stdout:9/210: sync 2024-04-20T10:59:14.792 INFO:tasks.workunit.client.0.smithi027.stdout:7/236: rename d4/d5/d8/f2f to d4/f3d 0 2024-04-20T10:59:14.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/266: symlink da/df/d19/d2e/l5c 0 2024-04-20T10:59:14.793 INFO:tasks.workunit.client.0.smithi027.stdout:9/211: mknod d4/d1b/d31/d23/d3b/c41 0 2024-04-20T10:59:14.794 INFO:tasks.workunit.client.0.smithi027.stdout:7/237: creat d4/d5/d8/d3b/f3e x:0 0 0 2024-04-20T10:59:14.795 INFO:tasks.workunit.client.0.smithi027.stdout:8/334: dwrite d9/de/f1b [0,4194304] 0 2024-04-20T10:59:14.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/267: creat da/df/d19/f5d x:0 0 0 2024-04-20T10:59:14.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/268: fsync da/df/d19/d2e/d32/f44 0 2024-04-20T10:59:14.796 INFO:tasks.workunit.client.0.smithi027.stdout:9/212: rmdir d4/d1b/d31/d23/d3b 39 2024-04-20T10:59:14.796 INFO:tasks.workunit.client.0.smithi027.stdout:9/213: fsync d4/d1b/d31/d23/d29/f3e 0 2024-04-20T10:59:14.796 INFO:tasks.workunit.client.0.smithi027.stdout:9/214: stat d4/d1b/d31/d23/f36 0 2024-04-20T10:59:14.796 INFO:tasks.workunit.client.0.smithi027.stdout:9/215: dread - d4/d1b/d31/d23/d29/f3e zero size 2024-04-20T10:59:14.797 INFO:tasks.workunit.client.0.smithi027.stdout:9/216: chown d4/d1b/d1f/l26 2559392 1 2024-04-20T10:59:14.797 INFO:tasks.workunit.client.0.smithi027.stdout:7/238: symlink d4/d5/d8/d21/l3f 0 2024-04-20T10:59:14.797 INFO:tasks.workunit.client.0.smithi027.stdout:1/359: dwrite d3/d7/d2d/f5a [0,4194304] 0 2024-04-20T10:59:14.798 INFO:tasks.workunit.client.0.smithi027.stdout:8/335: mkdir d9/de/d1e/d4a/d6d/d78 0 2024-04-20T10:59:14.800 INFO:tasks.workunit.client.0.smithi027.stdout:6/269: creat da/d21/d28/d2d/d53/f5e x:0 0 0 2024-04-20T10:59:14.801 INFO:tasks.workunit.client.0.smithi027.stdout:9/217: unlink d4/d1b/d31/d23/d29/f3e 0 2024-04-20T10:59:14.801 INFO:tasks.workunit.client.0.smithi027.stdout:9/218: fdatasync d4/d1b/d31/f3d 0 2024-04-20T10:59:14.804 INFO:tasks.workunit.client.0.smithi027.stdout:8/336: mkdir d9/de/d1e/d4a/d6d/d79 0 2024-04-20T10:59:14.804 INFO:tasks.workunit.client.0.smithi027.stdout:8/337: write d9/ff [9068294,101802] 0 2024-04-20T10:59:14.805 INFO:tasks.workunit.client.0.smithi027.stdout:6/270: mknod da/df/d1f/c5f 0 2024-04-20T10:59:14.806 INFO:tasks.workunit.client.0.smithi027.stdout:3/341: dwrite d2/d18/f26 [0,4194304] 0 2024-04-20T10:59:14.807 INFO:tasks.workunit.client.0.smithi027.stdout:0/247: dread d0/d7/f26 [0,4194304] 0 2024-04-20T10:59:14.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/273: dwrite f0 [0,4194304] 0 2024-04-20T10:59:14.808 INFO:tasks.workunit.client.0.smithi027.stdout:0/248: write d0/f1f [319033,98172] 0 2024-04-20T10:59:14.808 INFO:tasks.workunit.client.0.smithi027.stdout:0/249: dread - d0/d7/d34/f43 zero size 2024-04-20T10:59:14.809 INFO:tasks.workunit.client.0.smithi027.stdout:0/250: write d0/d7/d12/f44 [956768,109] 0 2024-04-20T10:59:14.809 INFO:tasks.workunit.client.0.smithi027.stdout:9/219: dread d4/f7 [0,4194304] 0 2024-04-20T10:59:14.809 INFO:tasks.workunit.client.0.smithi027.stdout:8/338: mknod d9/de/d20/d48/d59/c7a 0 2024-04-20T10:59:14.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/280: write d0/f37 [2056697,12782] 0 2024-04-20T10:59:14.812 INFO:tasks.workunit.client.0.smithi027.stdout:0/251: dread d0/d7/f26 [0,4194304] 0 2024-04-20T10:59:14.812 INFO:tasks.workunit.client.0.smithi027.stdout:0/252: dread - d0/d7/d34/f43 zero size 2024-04-20T10:59:14.813 INFO:tasks.workunit.client.0.smithi027.stdout:6/271: mknod da/dd/d3c/d58/c60 0 2024-04-20T10:59:14.813 INFO:tasks.workunit.client.0.smithi027.stdout:3/342: mkdir d2/d18/d2c/d4c/d67 0 2024-04-20T10:59:14.813 INFO:tasks.workunit.client.0.smithi027.stdout:3/343: write d2/dd/df/d33/f3c [691773,99069] 0 2024-04-20T10:59:14.815 INFO:tasks.workunit.client.0.smithi027.stdout:8/339: dread f3 [0,4194304] 0 2024-04-20T10:59:14.819 INFO:tasks.workunit.client.0.smithi027.stdout:3/344: dread d2/f17 [0,4194304] 0 2024-04-20T10:59:14.819 INFO:tasks.workunit.client.0.smithi027.stdout:3/345: fdatasync d2/d18/d2e/f35 0 2024-04-20T10:59:14.823 INFO:tasks.workunit.client.0.smithi027.stdout:4/281: mknod d0/d48/d68/c78 0 2024-04-20T10:59:14.823 INFO:tasks.workunit.client.0.smithi027.stdout:1/360: dwrite d3/f1e [0,4194304] 0 2024-04-20T10:59:14.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/253: mknod d0/d7/d12/d1a/c4a 0 2024-04-20T10:59:14.824 INFO:tasks.workunit.client.0.smithi027.stdout:6/272: symlink da/d21/l61 0 2024-04-20T10:59:14.825 INFO:tasks.workunit.client.0.smithi027.stdout:8/340: mknod d9/de/d20/d48/c7b 0 2024-04-20T10:59:14.825 INFO:tasks.workunit.client.0.smithi027.stdout:8/341: chown f3 17555 1 2024-04-20T10:59:14.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/282: mkdir d0/d14/d17/d53/d79 0 2024-04-20T10:59:14.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/283: chown d0/d14/d17/d1b/d2f/f42 1508872383 1 2024-04-20T10:59:14.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/284: dread - d0/d14/d17/d1b/d2f/f67 zero size 2024-04-20T10:59:14.829 INFO:tasks.workunit.client.0.smithi027.stdout:8/342: mkdir d9/de/d1e/d4a/d6d/d75/d7c 0 2024-04-20T10:59:14.829 INFO:tasks.workunit.client.0.smithi027.stdout:8/343: read d9/de/d1e/d49/f5c [2088848,27350] 0 2024-04-20T10:59:14.829 INFO:tasks.workunit.client.0.smithi027.stdout:4/285: mknod d0/d14/d17/d1a/d3b/c7a 0 2024-04-20T10:59:14.831 INFO:tasks.workunit.client.0.smithi027.stdout:7/239: dwrite d4/d5/fa [0,4194304] 0 2024-04-20T10:59:14.831 INFO:tasks.workunit.client.0.smithi027.stdout:7/240: fsync d4/d5/f33 0 2024-04-20T10:59:14.831 INFO:tasks.workunit.client.0.smithi027.stdout:4/286: mkdir d0/d14/d17/d1b/d33/d5c/d6f/d7b 0 2024-04-20T10:59:14.834 INFO:tasks.workunit.client.0.smithi027.stdout:8/344: truncate d9/de/f1b 868502 0 2024-04-20T10:59:14.834 INFO:tasks.workunit.client.0.smithi027.stdout:8/345: chown d9/de/d20/d4f/d5b/f5d 204 1 2024-04-20T10:59:14.834 INFO:tasks.workunit.client.0.smithi027.stdout:5/274: dwrite d19/d1b/d1c/f3b [0,4194304] 0 2024-04-20T10:59:14.836 INFO:tasks.workunit.client.0.smithi027.stdout:9/220: dwrite d4/d1b/d31/f3c [0,4194304] 0 2024-04-20T10:59:14.836 INFO:tasks.workunit.client.0.smithi027.stdout:7/241: creat d4/d5/f40 x:0 0 0 2024-04-20T10:59:14.842 INFO:tasks.workunit.client.0.smithi027.stdout:8/346: mkdir d9/de/d1e/d4a/d6d/d7d 0 2024-04-20T10:59:14.842 INFO:tasks.workunit.client.0.smithi027.stdout:5/275: symlink d19/d1b/d1c/d1e/d25/d40/l5b 0 2024-04-20T10:59:14.842 INFO:tasks.workunit.client.0.smithi027.stdout:5/276: fdatasync d19/d1b/d1c/f45 0 2024-04-20T10:59:14.842 INFO:tasks.workunit.client.0.smithi027.stdout:5/277: readlink d19/d1b/d1c/d1e/d22/l3a 0 2024-04-20T10:59:14.842 INFO:tasks.workunit.client.0.smithi027.stdout:6/273: dwrite da/f2a [0,4194304] 0 2024-04-20T10:59:14.844 INFO:tasks.workunit.client.0.smithi027.stdout:5/278: link d19/d1b/d1c/d1e/l2c d19/d1b/d1c/d1e/d58/l5c 0 2024-04-20T10:59:14.846 INFO:tasks.workunit.client.0.smithi027.stdout:5/279: mknod d19/d1b/d1c/d1e/d25/c5d 0 2024-04-20T10:59:14.851 INFO:tasks.workunit.client.0.smithi027.stdout:0/254: dwrite d0/d11/d2c/d2e/f1d [0,4194304] 0 2024-04-20T10:59:14.851 INFO:tasks.workunit.client.0.smithi027.stdout:2/192: sync 2024-04-20T10:59:14.851 INFO:tasks.workunit.client.0.smithi027.stdout:0/255: getdents d0/d11/d2c/d2e/d35 0 2024-04-20T10:59:14.851 INFO:tasks.workunit.client.0.smithi027.stdout:4/287: dwrite d0/d14/d17/d1b/d2f/f67 [0,4194304] 0 2024-04-20T10:59:14.851 INFO:tasks.workunit.client.0.smithi027.stdout:8/347: dread d9/de/d18/f21 [0,4194304] 0 2024-04-20T10:59:14.851 INFO:tasks.workunit.client.0.smithi027.stdout:0/256: truncate d0/d7/d34/f3b 214199 0 2024-04-20T10:59:14.856 INFO:tasks.workunit.client.0.smithi027.stdout:4/288: rename d0/d14/d17/d1b/d2f/f42 to d0/d14/d17/d53/d6a/f7c 0 2024-04-20T10:59:14.856 INFO:tasks.workunit.client.0.smithi027.stdout:4/289: dread - d0/d14/d17/f1c zero size 2024-04-20T10:59:14.857 INFO:tasks.workunit.client.0.smithi027.stdout:9/221: dwrite d4/f7 [0,4194304] 0 2024-04-20T10:59:14.857 INFO:tasks.workunit.client.0.smithi027.stdout:8/348: mknod d9/de/d64/d6a/c7e 0 2024-04-20T10:59:14.858 INFO:tasks.workunit.client.0.smithi027.stdout:2/193: dread d5/d1e/f22 [4194304,4194304] 0 2024-04-20T10:59:14.858 INFO:tasks.workunit.client.0.smithi027.stdout:2/194: write d5/d26/f3b [688760,2716] 0 2024-04-20T10:59:14.862 INFO:tasks.workunit.client.0.smithi027.stdout:7/242: dwrite d4/d5/f1f [0,4194304] 0 2024-04-20T10:59:14.864 INFO:tasks.workunit.client.0.smithi027.stdout:4/290: mknod d0/d14/d17/d1b/d33/d5c/d6f/c7d 0 2024-04-20T10:59:14.864 INFO:tasks.workunit.client.0.smithi027.stdout:8/349: creat d9/de/d20/d26/f7f x:0 0 0 2024-04-20T10:59:14.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/257: mkdir d0/d11/d4b 0 2024-04-20T10:59:14.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/258: write d0/d11/f32 [1894318,59713] 0 2024-04-20T10:59:14.864 INFO:tasks.workunit.client.0.smithi027.stdout:5/280: dwrite d19/f27 [0,4194304] 0 2024-04-20T10:59:14.867 INFO:tasks.workunit.client.0.smithi027.stdout:4/291: mknod d0/d14/d17/d53/d6a/c7e 0 2024-04-20T10:59:14.869 INFO:tasks.workunit.client.0.smithi027.stdout:8/350: dread d9/f39 [0,4194304] 0 2024-04-20T10:59:14.871 INFO:tasks.workunit.client.0.smithi027.stdout:7/243: read d4/d5/f1f [301339,43003] 0 2024-04-20T10:59:14.871 INFO:tasks.workunit.client.0.smithi027.stdout:7/244: write d4/d5/f24 [4455426,425] 0 2024-04-20T10:59:14.871 INFO:tasks.workunit.client.0.smithi027.stdout:5/281: unlink d19/d1b/d1c/d1e/d22/f42 0 2024-04-20T10:59:14.872 INFO:tasks.workunit.client.0.smithi027.stdout:0/259: creat d0/d11/d2c/d2e/d2b/d3d/f4c x:0 0 0 2024-04-20T10:59:14.873 INFO:tasks.workunit.client.0.smithi027.stdout:4/292: creat d0/d14/d17/d1b/f7f x:0 0 0 2024-04-20T10:59:14.874 INFO:tasks.workunit.client.0.smithi027.stdout:8/351: creat d9/de/d20/d48/d59/d74/f80 x:0 0 0 2024-04-20T10:59:14.876 INFO:tasks.workunit.client.0.smithi027.stdout:2/195: dwrite d5/d1e/f22 [0,4194304] 0 2024-04-20T10:59:14.877 INFO:tasks.workunit.client.0.smithi027.stdout:7/245: link d4/d5/d8/l36 d4/d5/d8/d21/l41 0 2024-04-20T10:59:14.879 INFO:tasks.workunit.client.0.smithi027.stdout:4/293: rename d0/d14/d17/d1a/l4d to d0/d5e/d6c/l80 0 2024-04-20T10:59:14.880 INFO:tasks.workunit.client.0.smithi027.stdout:2/196: creat d5/dc/d16/d2a/f40 x:0 0 0 2024-04-20T10:59:14.882 INFO:tasks.workunit.client.0.smithi027.stdout:4/294: unlink d0/d14/d17/c52 0 2024-04-20T10:59:14.882 INFO:tasks.workunit.client.0.smithi027.stdout:4/295: chown d0/d14/d17/d18/c55 6 1 2024-04-20T10:59:14.883 INFO:tasks.workunit.client.0.smithi027.stdout:2/197: creat d5/dc/f41 x:0 0 0 2024-04-20T10:59:14.884 INFO:tasks.workunit.client.0.smithi027.stdout:4/296: rename d0/d14/d17/d53/l5b to d0/d14/d17/d18/l81 0 2024-04-20T10:59:14.884 INFO:tasks.workunit.client.0.smithi027.stdout:4/297: write d0/d14/d17/d18/f20 [1813160,43388] 0 2024-04-20T10:59:14.885 INFO:tasks.workunit.client.0.smithi027.stdout:2/198: unlink d5/dc/f15 0 2024-04-20T10:59:14.886 INFO:tasks.workunit.client.0.smithi027.stdout:8/352: dread d9/de/d20/d48/d59/f60 [0,4194304] 0 2024-04-20T10:59:14.886 INFO:tasks.workunit.client.0.smithi027.stdout:4/298: creat d0/d14/d17/d1b/d2f/f82 x:0 0 0 2024-04-20T10:59:14.887 INFO:tasks.workunit.client.0.smithi027.stdout:8/353: getdents d9/de/d20/d53 0 2024-04-20T10:59:14.888 INFO:tasks.workunit.client.0.smithi027.stdout:7/246: dread d4/fc [0,4194304] 0 2024-04-20T10:59:14.888 INFO:tasks.workunit.client.0.smithi027.stdout:7/247: write d4/d1b/f35 [1142425,74220] 0 2024-04-20T10:59:14.888 INFO:tasks.workunit.client.0.smithi027.stdout:4/299: getdents d0/d14/d17/d1b/d33/d5c 0 2024-04-20T10:59:14.888 INFO:tasks.workunit.client.0.smithi027.stdout:4/300: write d0/d14/d17/d18/f4b [607960,115974] 0 2024-04-20T10:59:14.890 INFO:tasks.workunit.client.0.smithi027.stdout:8/354: creat d9/de/d1e/d4a/d6d/d75/f81 x:0 0 0 2024-04-20T10:59:14.891 INFO:tasks.workunit.client.0.smithi027.stdout:7/248: write d4/d5/f14 [2957419,11258] 0 2024-04-20T10:59:14.891 INFO:tasks.workunit.client.0.smithi027.stdout:7/249: chown d4/d5/d8/d21/l41 7697990 1 2024-04-20T10:59:14.891 INFO:tasks.workunit.client.0.smithi027.stdout:7/250: write d4/d5/d8/d3b/f3e [21544,44304] 0 2024-04-20T10:59:14.891 INFO:tasks.workunit.client.0.smithi027.stdout:4/301: dread d0/d14/d17/d18/f36 [0,4194304] 0 2024-04-20T10:59:14.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/260: dwrite d0/f1f [0,4194304] 0 2024-04-20T10:59:14.892 INFO:tasks.workunit.client.0.smithi027.stdout:8/355: symlink d9/de/d64/d6a/l82 0 2024-04-20T10:59:14.892 INFO:tasks.workunit.client.0.smithi027.stdout:8/356: readlink d9/de/d1e/l42 0 2024-04-20T10:59:14.894 INFO:tasks.workunit.client.0.smithi027.stdout:4/302: getdents d0/d48/d68 0 2024-04-20T10:59:14.894 INFO:tasks.workunit.client.0.smithi027.stdout:7/251: creat d4/d5/d8/d3b/f42 x:0 0 0 2024-04-20T10:59:14.894 INFO:tasks.workunit.client.0.smithi027.stdout:7/252: chown d4/d5/d8/l36 3361 1 2024-04-20T10:59:14.896 INFO:tasks.workunit.client.0.smithi027.stdout:2/199: dread d5/f28 [0,4194304] 0 2024-04-20T10:59:14.896 INFO:tasks.workunit.client.0.smithi027.stdout:5/282: dwrite d19/f4f [0,4194304] 0 2024-04-20T10:59:14.896 INFO:tasks.workunit.client.0.smithi027.stdout:5/283: fdatasync d19/d1b/d1c/d1e/f56 0 2024-04-20T10:59:14.896 INFO:tasks.workunit.client.0.smithi027.stdout:5/284: stat d19/d1b/d1c/d1e/d3c/f31 0 2024-04-20T10:59:14.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/261: creat d0/d11/d2c/d2e/f4d x:0 0 0 2024-04-20T10:59:14.900 INFO:tasks.workunit.client.0.smithi027.stdout:7/253: rename d4/d5/f24 to d4/d5/d8/f43 0 2024-04-20T10:59:14.901 INFO:tasks.workunit.client.0.smithi027.stdout:2/200: rename d5/d1e/f22 to d5/d1e/f42 0 2024-04-20T10:59:14.902 INFO:tasks.workunit.client.0.smithi027.stdout:5/285: creat d19/d1b/d1c/d1e/d58/f5e x:0 0 0 2024-04-20T10:59:14.905 INFO:tasks.workunit.client.0.smithi027.stdout:7/254: rename d4/d1b/f20 to d4/d1b/d37/f44 0 2024-04-20T10:59:14.907 INFO:tasks.workunit.client.0.smithi027.stdout:2/201: link d5/d26/d30/c31 d5/d1e/c43 0 2024-04-20T10:59:14.907 INFO:tasks.workunit.client.0.smithi027.stdout:7/255: dread d4/d5/d8/f43 [4194304,4194304] 0 2024-04-20T10:59:14.908 INFO:tasks.workunit.client.0.smithi027.stdout:5/286: mknod d19/c5f 0 2024-04-20T10:59:14.909 INFO:tasks.workunit.client.0.smithi027.stdout:5/287: truncate f17 5196622 0 2024-04-20T10:59:14.910 INFO:tasks.workunit.client.0.smithi027.stdout:2/202: truncate d5/f28 3671751 0 2024-04-20T10:59:14.911 INFO:tasks.workunit.client.0.smithi027.stdout:7/256: mkdir d4/d5/d8/d45 0 2024-04-20T10:59:14.912 INFO:tasks.workunit.client.0.smithi027.stdout:2/203: mkdir d5/d44 0 2024-04-20T10:59:14.913 INFO:tasks.workunit.client.0.smithi027.stdout:2/204: link d5/dc/d16/l2f d5/dc/l45 0 2024-04-20T10:59:14.914 INFO:tasks.workunit.client.0.smithi027.stdout:4/303: dwrite d0/d14/d17/d1b/d33/f57 [0,4194304] 0 2024-04-20T10:59:14.914 INFO:tasks.workunit.client.0.smithi027.stdout:4/304: chown d0/d14/d17/d1a/l24 17895380 1 2024-04-20T10:59:14.915 INFO:tasks.workunit.client.0.smithi027.stdout:8/357: dwrite d9/de/d1e/d4a/f45 [0,4194304] 0 2024-04-20T10:59:14.915 INFO:tasks.workunit.client.0.smithi027.stdout:8/358: fsync f2 0 2024-04-20T10:59:14.917 INFO:tasks.workunit.client.0.smithi027.stdout:4/305: unlink d0/d14/d17/d1b/l41 0 2024-04-20T10:59:14.919 INFO:tasks.workunit.client.0.smithi027.stdout:7/257: read d4/d1b/ff [1125195,64950] 0 2024-04-20T10:59:14.919 INFO:tasks.workunit.client.0.smithi027.stdout:5/288: dread d19/d1b/d1c/d1e/d3c/f31 [0,4194304] 0 2024-04-20T10:59:14.921 INFO:tasks.workunit.client.0.smithi027.stdout:8/359: creat d9/de/d1e/d4a/d6d/d75/f83 x:0 0 0 2024-04-20T10:59:14.921 INFO:tasks.workunit.client.0.smithi027.stdout:4/306: dread d0/d14/d17/f16 [0,4194304] 0 2024-04-20T10:59:14.922 INFO:tasks.workunit.client.0.smithi027.stdout:7/258: mkdir d4/d5/d8/d46 0 2024-04-20T10:59:14.923 INFO:tasks.workunit.client.0.smithi027.stdout:5/289: creat d19/d1b/d1c/d1e/d58/f60 x:0 0 0 2024-04-20T10:59:14.923 INFO:tasks.workunit.client.0.smithi027.stdout:5/290: dread - d19/d1b/d1c/d1e/d22/f5a zero size 2024-04-20T10:59:14.925 INFO:tasks.workunit.client.0.smithi027.stdout:8/360: link d9/de/d20/d26/d3c/d4b/f69 d9/de/d20/d4f/d5b/f84 0 2024-04-20T10:59:14.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/307: symlink d0/d14/d17/d1a/d3b/l83 0 2024-04-20T10:59:14.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/308: stat d0/c29 0 2024-04-20T10:59:14.927 INFO:tasks.workunit.client.0.smithi027.stdout:7/259: mknod d4/d1b/c47 0 2024-04-20T10:59:14.928 INFO:tasks.workunit.client.0.smithi027.stdout:8/361: creat d9/de/d20/d48/f85 x:0 0 0 2024-04-20T10:59:14.929 INFO:tasks.workunit.client.0.smithi027.stdout:7/260: stat d4/d1b/c23 0 2024-04-20T10:59:14.929 INFO:tasks.workunit.client.0.smithi027.stdout:7/261: fdatasync d4/fc 0 2024-04-20T10:59:14.930 INFO:tasks.workunit.client.0.smithi027.stdout:8/362: rmdir d9/de/d20/d4f 39 2024-04-20T10:59:14.932 INFO:tasks.workunit.client.0.smithi027.stdout:7/262: creat d4/d5/d8/d21/f48 x:0 0 0 2024-04-20T10:59:14.934 INFO:tasks.workunit.client.0.smithi027.stdout:8/363: symlink d9/de/d1e/d4a/d6d/d79/l86 0 2024-04-20T10:59:14.939 INFO:tasks.workunit.client.0.smithi027.stdout:7/263: dread d4/d1b/f28 [0,4194304] 0 2024-04-20T10:59:14.950 INFO:tasks.workunit.client.0.smithi027.stdout:1/361: sync 2024-04-20T10:59:14.950 INFO:tasks.workunit.client.0.smithi027.stdout:3/346: sync 2024-04-20T10:59:14.951 INFO:tasks.workunit.client.0.smithi027.stdout:5/291: dwrite d19/d1b/d1c/d1e/d58/f5e [0,4194304] 0 2024-04-20T10:59:14.952 INFO:tasks.workunit.client.0.smithi027.stdout:4/309: dwrite d0/fb [0,4194304] 0 2024-04-20T10:59:14.967 INFO:tasks.workunit.client.0.smithi027.stdout:8/364: rename d9/de/d20/d26 to d9/de/d64/d6c/d87 0 2024-04-20T10:59:14.968 INFO:tasks.workunit.client.0.smithi027.stdout:7/264: rename d4/c30 to d4/d5/d8/d45/c49 0 2024-04-20T10:59:14.968 INFO:tasks.workunit.client.0.smithi027.stdout:4/310: dread d0/fd [0,4194304] 0 2024-04-20T10:59:14.969 INFO:tasks.workunit.client.0.smithi027.stdout:8/365: creat d9/de/d20/d4f/d5b/f88 x:0 0 0 2024-04-20T10:59:14.971 INFO:tasks.workunit.client.0.smithi027.stdout:4/311: dread d0/d14/f4a [0,4194304] 0 2024-04-20T10:59:14.971 INFO:tasks.workunit.client.0.smithi027.stdout:8/366: dread d9/f39 [0,4194304] 0 2024-04-20T10:59:14.971 INFO:tasks.workunit.client.0.smithi027.stdout:4/312: truncate d0/d14/d17/d18/f38 700005 0 2024-04-20T10:59:14.971 INFO:tasks.workunit.client.0.smithi027.stdout:4/313: dread - d0/d14/d17/d18/f39 zero size 2024-04-20T10:59:14.972 INFO:tasks.workunit.client.0.smithi027.stdout:1/362: dwrite d3/d7/d28/d35/f5e [0,4194304] 0 2024-04-20T10:59:14.972 INFO:tasks.workunit.client.0.smithi027.stdout:4/314: symlink d0/d14/d17/d53/d79/l84 0 2024-04-20T10:59:14.973 INFO:tasks.workunit.client.0.smithi027.stdout:1/363: mkdir d3/d20/d85 0 2024-04-20T10:59:14.973 INFO:tasks.workunit.client.0.smithi027.stdout:7/265: write d4/d5/f19 [1119669,112499] 0 2024-04-20T10:59:14.974 INFO:tasks.workunit.client.0.smithi027.stdout:4/315: rename d0/d14/d17/d1a/c6e to d0/d48/d68/c85 0 2024-04-20T10:59:14.974 INFO:tasks.workunit.client.0.smithi027.stdout:7/266: getdents d4/d1b/d37 0 2024-04-20T10:59:14.975 INFO:tasks.workunit.client.0.smithi027.stdout:3/347: dwrite d2/fe [4194304,4194304] 0 2024-04-20T10:59:14.975 INFO:tasks.workunit.client.0.smithi027.stdout:3/348: write d2/dd/d57/f59 [668635,115370] 0 2024-04-20T10:59:14.978 INFO:tasks.workunit.client.0.smithi027.stdout:3/349: mkdir d2/dd/df/d23/d68 0 2024-04-20T10:59:14.980 INFO:tasks.workunit.client.0.smithi027.stdout:1/364: dread d3/f17 [0,4194304] 0 2024-04-20T10:59:14.984 INFO:tasks.workunit.client.0.smithi027.stdout:1/365: write d3/d7/f1f [1310095,91033] 0 2024-04-20T10:59:14.987 INFO:tasks.workunit.client.0.smithi027.stdout:7/267: rename d4/d1b/d37 to d4/d1b/d4a 0 2024-04-20T10:59:14.991 INFO:tasks.workunit.client.0.smithi027.stdout:3/350: getdents d2/d18/d2c/d4c 0 2024-04-20T10:59:14.991 INFO:tasks.workunit.client.0.smithi027.stdout:4/316: dwrite d0/d14/d17/d18/f38 [0,4194304] 0 2024-04-20T10:59:14.995 INFO:tasks.workunit.client.0.smithi027.stdout:1/366: creat d3/d20/d85/f86 x:0 0 0 2024-04-20T10:59:14.997 INFO:tasks.workunit.client.0.smithi027.stdout:7/268: creat d4/d5/d8/d3b/f4b x:0 0 0 2024-04-20T10:59:14.997 INFO:tasks.workunit.client.0.smithi027.stdout:7/269: chown d4/d5 228045 1 2024-04-20T10:59:14.997 INFO:tasks.workunit.client.0.smithi027.stdout:7/270: write d4/d1b/f35 [1404469,58656] 0 2024-04-20T10:59:14.997 INFO:tasks.workunit.client.0.smithi027.stdout:7/271: getdents d4/d1b/d4a 0 2024-04-20T10:59:14.998 INFO:tasks.workunit.client.0.smithi027.stdout:1/367: getdents d3/d45/d5d/d6c 0 2024-04-20T10:59:15.001 INFO:tasks.workunit.client.0.smithi027.stdout:7/272: creat d4/d5/d8/f4c x:0 0 0 2024-04-20T10:59:15.001 INFO:tasks.workunit.client.0.smithi027.stdout:7/273: write d4/f2e [1666612,9133] 0 2024-04-20T10:59:15.001 INFO:tasks.workunit.client.0.smithi027.stdout:7/274: write d4/d5/d8/d21/f31 [1003961,52902] 0 2024-04-20T10:59:15.001 INFO:tasks.workunit.client.0.smithi027.stdout:7/275: write d4/d1b/ff [1192605,65357] 0 2024-04-20T10:59:15.002 INFO:tasks.workunit.client.0.smithi027.stdout:3/351: write d2/d18/d2e/d41/f45 [3314183,18119] 0 2024-04-20T10:59:15.002 INFO:tasks.workunit.client.0.smithi027.stdout:3/352: getdents d2/d18/d2e/d41 0 2024-04-20T10:59:15.002 INFO:tasks.workunit.client.0.smithi027.stdout:3/353: readlink d2/d18/d2c/d4c/l4d 0 2024-04-20T10:59:15.002 INFO:tasks.workunit.client.0.smithi027.stdout:1/368: link d3/d7/c15 d3/d41/c87 0 2024-04-20T10:59:15.006 INFO:tasks.workunit.client.0.smithi027.stdout:6/274: sync 2024-04-20T10:59:15.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/354: creat d2/d18/d2c/f69 x:0 0 0 2024-04-20T10:59:15.009 INFO:tasks.workunit.client.0.smithi027.stdout:3/355: dread d2/dd/f10 [0,4194304] 0 2024-04-20T10:59:15.009 INFO:tasks.workunit.client.0.smithi027.stdout:3/356: chown d2/d18/d2e/d41/f5a 213381 1 2024-04-20T10:59:15.010 INFO:tasks.workunit.client.0.smithi027.stdout:1/369: mknod d3/d7e/c88 0 2024-04-20T10:59:15.010 INFO:tasks.workunit.client.0.smithi027.stdout:1/370: dread - d3/d7/d38/f65 zero size 2024-04-20T10:59:15.010 INFO:tasks.workunit.client.0.smithi027.stdout:7/276: mkdir d4/d4d 0 2024-04-20T10:59:15.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/275: creat da/df/d19/d2e/d42/f62 x:0 0 0 2024-04-20T10:59:15.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/276: chown da/df/d1f/c2c 0 1 2024-04-20T10:59:15.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/277: truncate da/f2a 5192460 0 2024-04-20T10:59:15.012 INFO:tasks.workunit.client.0.smithi027.stdout:3/357: creat d2/f6a x:0 0 0 2024-04-20T10:59:15.013 INFO:tasks.workunit.client.0.smithi027.stdout:1/371: rmdir d3/d7/d2d/d73 39 2024-04-20T10:59:15.013 INFO:tasks.workunit.client.0.smithi027.stdout:1/372: fdatasync d3/d7/f23 0 2024-04-20T10:59:15.013 INFO:tasks.workunit.client.0.smithi027.stdout:1/373: readlink d3/d7/d38/d62/l76 0 2024-04-20T10:59:15.014 INFO:tasks.workunit.client.0.smithi027.stdout:6/278: symlink da/df/d19/d2e/d32/l63 0 2024-04-20T10:59:15.015 INFO:tasks.workunit.client.0.smithi027.stdout:6/279: dread - da/df/d19/d2e/d42/f62 zero size 2024-04-20T10:59:15.015 INFO:tasks.workunit.client.0.smithi027.stdout:6/280: fdatasync da/df/f29 0 2024-04-20T10:59:15.015 INFO:tasks.workunit.client.0.smithi027.stdout:3/358: symlink d2/d18/d2c/d4c/l6b 0 2024-04-20T10:59:15.016 INFO:tasks.workunit.client.0.smithi027.stdout:1/374: symlink d3/d7/d2d/d73/l89 0 2024-04-20T10:59:15.016 INFO:tasks.workunit.client.0.smithi027.stdout:1/375: truncate d3/d7/d28/f48 605415 0 2024-04-20T10:59:15.016 INFO:tasks.workunit.client.0.smithi027.stdout:7/277: mknod d4/c4e 0 2024-04-20T10:59:15.016 INFO:tasks.workunit.client.0.smithi027.stdout:7/278: readlink l3 0 2024-04-20T10:59:15.018 INFO:tasks.workunit.client.0.smithi027.stdout:6/281: link da/d21/d28/d2d/d53/f5e da/dd/d3d/f64 0 2024-04-20T10:59:15.021 INFO:tasks.workunit.client.0.smithi027.stdout:3/359: mkdir d2/d18/d2c/d4c/d67/d6c 0 2024-04-20T10:59:15.021 INFO:tasks.workunit.client.0.smithi027.stdout:3/360: stat d2/dd/df/l2b 0 2024-04-20T10:59:15.021 INFO:tasks.workunit.client.0.smithi027.stdout:3/361: write d2/d18/d2c/f4f [1014584,44270] 0 2024-04-20T10:59:15.021 INFO:tasks.workunit.client.0.smithi027.stdout:3/362: write d2/dd/df/d33/f3c [4787818,101314] 0 2024-04-20T10:59:15.022 INFO:tasks.workunit.client.0.smithi027.stdout:1/376: mknod d3/d20/c8a 0 2024-04-20T10:59:15.022 INFO:tasks.workunit.client.0.smithi027.stdout:7/279: truncate d4/fc 4773771 0 2024-04-20T10:59:15.022 INFO:tasks.workunit.client.0.smithi027.stdout:7/280: fsync d4/d5/d8/d3b/f3e 0 2024-04-20T10:59:15.025 INFO:tasks.workunit.client.0.smithi027.stdout:6/282: mkdir da/df/d19/d2e/d42/d65 0 2024-04-20T10:59:15.026 INFO:tasks.workunit.client.0.smithi027.stdout:3/363: creat d2/d18/d2c/d4c/d67/f6d x:0 0 0 2024-04-20T10:59:15.026 INFO:tasks.workunit.client.0.smithi027.stdout:3/364: chown d2/c56 311050219 1 2024-04-20T10:59:15.027 INFO:tasks.workunit.client.0.smithi027.stdout:1/377: creat d3/d7/d28/f8b x:0 0 0 2024-04-20T10:59:15.027 INFO:tasks.workunit.client.0.smithi027.stdout:7/281: chown d4/le 14438822 1 2024-04-20T10:59:15.030 INFO:tasks.workunit.client.0.smithi027.stdout:6/283: creat da/d21/f66 x:0 0 0 2024-04-20T10:59:15.030 INFO:tasks.workunit.client.0.smithi027.stdout:6/284: write da/df/f10 [4450449,78420] 0 2024-04-20T10:59:15.037 INFO:tasks.workunit.client.0.smithi027.stdout:6/285: dread da/d21/d28/f34 [0,4194304] 0 2024-04-20T10:59:15.038 INFO:tasks.workunit.client.0.smithi027.stdout:3/365: truncate d2/dd/df/d23/f50 6783405 0 2024-04-20T10:59:15.038 INFO:tasks.workunit.client.0.smithi027.stdout:1/378: write d3/f1e [1709652,85181] 0 2024-04-20T10:59:15.039 INFO:tasks.workunit.client.0.smithi027.stdout:6/286: getdents da/df 0 2024-04-20T10:59:15.043 INFO:tasks.workunit.client.0.smithi027.stdout:3/366: fdatasync d2/dd/df/d33/f3c 0 2024-04-20T10:59:15.043 INFO:tasks.workunit.client.0.smithi027.stdout:3/367: stat d2/d18/d2c/f69 0 2024-04-20T10:59:15.044 INFO:tasks.workunit.client.0.smithi027.stdout:9/222: sync 2024-04-20T10:59:15.044 INFO:tasks.workunit.client.0.smithi027.stdout:7/282: dwrite d4/d1b/f35 [0,4194304] 0 2024-04-20T10:59:15.054 INFO:tasks.workunit.client.0.smithi027.stdout:9/223: dread d4/d14/f38 [0,4194304] 0 2024-04-20T10:59:15.054 INFO:tasks.workunit.client.0.smithi027.stdout:9/224: truncate d4/d1b/d31/d23/d3b/f3f 259893 0 2024-04-20T10:59:15.059 INFO:tasks.workunit.client.0.smithi027.stdout:3/368: dwrite d2/f14 [0,4194304] 0 2024-04-20T10:59:15.060 INFO:tasks.workunit.client.0.smithi027.stdout:3/369: chown d2/dd/c4a 8578535 1 2024-04-20T10:59:15.060 INFO:tasks.workunit.client.0.smithi027.stdout:3/370: readlink d2/d18/d2c/d4c/l6b 0 2024-04-20T10:59:15.297 INFO:tasks.workunit.client.0.smithi027.stdout:1/379: creat d3/d41/f8c x:0 0 0 2024-04-20T10:59:15.297 INFO:tasks.workunit.client.0.smithi027.stdout:6/287: rename da/d21/l61 to da/dd/l67 0 2024-04-20T10:59:15.298 INFO:tasks.workunit.client.0.smithi027.stdout:6/288: read da/df/f4d [448655,108011] 0 2024-04-20T10:59:15.299 INFO:tasks.workunit.client.0.smithi027.stdout:7/283: creat d4/d4d/f4f x:0 0 0 2024-04-20T10:59:15.300 INFO:tasks.workunit.client.0.smithi027.stdout:9/225: creat d4/d1b/f42 x:0 0 0 2024-04-20T10:59:15.302 INFO:tasks.workunit.client.0.smithi027.stdout:3/371: creat d2/d18/d2c/d4c/f6e x:0 0 0 2024-04-20T10:59:15.304 INFO:tasks.workunit.client.0.smithi027.stdout:1/380: getdents d3 0 2024-04-20T10:59:15.305 INFO:tasks.workunit.client.0.smithi027.stdout:9/226: truncate d4/f7 7283212 0 2024-04-20T10:59:15.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/227: fdatasync d4/d1b/f42 0 2024-04-20T10:59:15.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/372: creat d2/d18/d2e/d41/f6f x:0 0 0 2024-04-20T10:59:15.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/373: stat d2 0 2024-04-20T10:59:15.307 INFO:tasks.workunit.client.0.smithi027.stdout:1/381: mknod d3/d7/d38/c8d 0 2024-04-20T10:59:15.309 INFO:tasks.workunit.client.0.smithi027.stdout:7/284: link d4/d5/d8/d21/l41 d4/l50 0 2024-04-20T10:59:15.309 INFO:tasks.workunit.client.0.smithi027.stdout:9/228: rmdir d4/d1b 39 2024-04-20T10:59:15.311 INFO:tasks.workunit.client.0.smithi027.stdout:3/374: mknod d2/d18/d2e/c70 0 2024-04-20T10:59:15.312 INFO:tasks.workunit.client.0.smithi027.stdout:1/382: creat d3/d20/f8e x:0 0 0 2024-04-20T10:59:15.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/285: getdents d4/d5/d8/d45 0 2024-04-20T10:59:15.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/286: write d4/f2e [2520723,119638] 0 2024-04-20T10:59:15.314 INFO:tasks.workunit.client.0.smithi027.stdout:8/367: sync 2024-04-20T10:59:15.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/317: sync 2024-04-20T10:59:15.314 INFO:tasks.workunit.client.0.smithi027.stdout:0/262: sync 2024-04-20T10:59:15.315 INFO:tasks.workunit.client.0.smithi027.stdout:2/205: sync 2024-04-20T10:59:15.315 INFO:tasks.workunit.client.0.smithi027.stdout:5/292: sync 2024-04-20T10:59:15.315 INFO:tasks.workunit.client.0.smithi027.stdout:4/318: readlink d0/d14/d17/d18/l66 0 2024-04-20T10:59:15.315 INFO:tasks.workunit.client.0.smithi027.stdout:4/319: chown d0/d14/d17/d1a/c56 15997387 1 2024-04-20T10:59:15.316 INFO:tasks.workunit.client.0.smithi027.stdout:3/375: rename d2/dd/df/c65 to d2/d18/d2e/c71 0 2024-04-20T10:59:15.318 INFO:tasks.workunit.client.0.smithi027.stdout:7/287: symlink d4/d5/d8/d3b/l51 0 2024-04-20T10:59:15.320 INFO:tasks.workunit.client.0.smithi027.stdout:9/229: mkdir d4/d14/d43 0 2024-04-20T10:59:15.321 INFO:tasks.workunit.client.0.smithi027.stdout:0/263: mkdir d0/d7/d4e 0 2024-04-20T10:59:15.322 INFO:tasks.workunit.client.0.smithi027.stdout:0/264: dread - d0/d11/d2c/d2e/d2b/d3d/f4c zero size 2024-04-20T10:59:15.323 INFO:tasks.workunit.client.0.smithi027.stdout:2/206: creat d5/dc/d16/d37/f46 x:0 0 0 2024-04-20T10:59:15.323 INFO:tasks.workunit.client.0.smithi027.stdout:2/207: chown d5/d1e/d21/f33 126 1 2024-04-20T10:59:15.324 INFO:tasks.workunit.client.0.smithi027.stdout:3/376: rmdir d2/d18/d2c/d4c/d67 39 2024-04-20T10:59:15.328 INFO:tasks.workunit.client.0.smithi027.stdout:9/230: unlink d4/d2a/c32 0 2024-04-20T10:59:15.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/208: dread d5/f17 [0,4194304] 0 2024-04-20T10:59:15.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/209: dread - d5/d1e/d21/f33 zero size 2024-04-20T10:59:15.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/377: mkdir d2/d18/d2c/d72 0 2024-04-20T10:59:15.330 INFO:tasks.workunit.client.0.smithi027.stdout:2/210: write d5/d26/f2c [354751,108263] 0 2024-04-20T10:59:15.331 INFO:tasks.workunit.client.0.smithi027.stdout:7/288: mkdir d4/d52 0 2024-04-20T10:59:15.332 INFO:tasks.workunit.client.0.smithi027.stdout:1/383: dwrite d3/d7/d28/f48 [0,4194304] 0 2024-04-20T10:59:15.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/231: creat d4/d14/f44 x:0 0 0 2024-04-20T10:59:15.334 INFO:tasks.workunit.client.0.smithi027.stdout:1/384: dread d3/d20/f46 [0,4194304] 0 2024-04-20T10:59:15.335 INFO:tasks.workunit.client.0.smithi027.stdout:3/378: symlink d2/dd/d57/l73 0 2024-04-20T10:59:15.336 INFO:tasks.workunit.client.0.smithi027.stdout:7/289: creat d4/d1b/f53 x:0 0 0 2024-04-20T10:59:15.337 INFO:tasks.workunit.client.0.smithi027.stdout:7/290: write d4/d5/d8/d21/f48 [725643,56655] 0 2024-04-20T10:59:15.337 INFO:tasks.workunit.client.0.smithi027.stdout:7/291: write d4/d5/d8/d21/f31 [1875584,32493] 0 2024-04-20T10:59:15.338 INFO:tasks.workunit.client.0.smithi027.stdout:9/232: stat d4/d14/c17 0 2024-04-20T10:59:15.339 INFO:tasks.workunit.client.0.smithi027.stdout:0/265: dread d0/d7/f27 [0,4194304] 0 2024-04-20T10:59:15.340 INFO:tasks.workunit.client.0.smithi027.stdout:0/266: dread - d0/d11/d2c/d2e/f2a zero size 2024-04-20T10:59:15.340 INFO:tasks.workunit.client.0.smithi027.stdout:1/385: rename d3/d7/d38/l50 to d3/d7/d2d/l8f 0 2024-04-20T10:59:15.340 INFO:tasks.workunit.client.0.smithi027.stdout:1/386: write d3/d20/f46 [627237,129322] 0 2024-04-20T10:59:15.341 INFO:tasks.workunit.client.0.smithi027.stdout:0/267: mknod d0/d7/d12/c4f 0 2024-04-20T10:59:15.342 INFO:tasks.workunit.client.0.smithi027.stdout:1/387: creat d3/d45/d5d/d6c/f90 x:0 0 0 2024-04-20T10:59:15.343 INFO:tasks.workunit.client.0.smithi027.stdout:0/268: creat d0/d7/d12/d1a/f50 x:0 0 0 2024-04-20T10:59:15.344 INFO:tasks.workunit.client.0.smithi027.stdout:0/269: getdents d0/d11 0 2024-04-20T10:59:15.344 INFO:tasks.workunit.client.0.smithi027.stdout:0/270: read d0/d7/d12/d1a/f2d [555651,27866] 0 2024-04-20T10:59:15.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/271: mknod d0/d7/d12/d1a/d47/c51 0 2024-04-20T10:59:15.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/292: dread d4/d5/d8/f38 [0,4194304] 0 2024-04-20T10:59:15.346 INFO:tasks.workunit.client.0.smithi027.stdout:0/272: creat d0/d11/d2c/f52 x:0 0 0 2024-04-20T10:59:15.348 INFO:tasks.workunit.client.0.smithi027.stdout:7/293: unlink d4/d5/d8/d45/c49 0 2024-04-20T10:59:15.348 INFO:tasks.workunit.client.0.smithi027.stdout:0/273: link d0/d7/d12/d1a/l40 d0/d11/d2c/d2e/d35/l53 0 2024-04-20T10:59:15.348 INFO:tasks.workunit.client.0.smithi027.stdout:0/274: dread - d0/d11/d2c/d2e/f4d zero size 2024-04-20T10:59:15.349 INFO:tasks.workunit.client.0.smithi027.stdout:7/294: creat d4/d5/f54 x:0 0 0 2024-04-20T10:59:15.350 INFO:tasks.workunit.client.0.smithi027.stdout:0/275: creat d0/d11/f54 x:0 0 0 2024-04-20T10:59:15.350 INFO:tasks.workunit.client.0.smithi027.stdout:0/276: chown d0/d7/d12/d1a/c4a 0 1 2024-04-20T10:59:15.350 INFO:tasks.workunit.client.0.smithi027.stdout:0/277: fsync d0/d11/d2c/d2e/d2b/d3d/f4c 0 2024-04-20T10:59:15.350 INFO:tasks.workunit.client.0.smithi027.stdout:0/278: write d0/d7/d12/d1a/f50 [895254,62594] 0 2024-04-20T10:59:15.351 INFO:tasks.workunit.client.0.smithi027.stdout:1/388: dread d3/d7/f34 [0,4194304] 0 2024-04-20T10:59:15.351 INFO:tasks.workunit.client.0.smithi027.stdout:7/295: rename d4/d1b/l2b to d4/d5/l55 0 2024-04-20T10:59:15.351 INFO:tasks.workunit.client.0.smithi027.stdout:7/296: truncate d4/d5/d8/f43 5000530 0 2024-04-20T10:59:15.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/279: mkdir d0/d11/d2c/d2e/d55 0 2024-04-20T10:59:15.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/280: dread - d0/d11/f54 zero size 2024-04-20T10:59:15.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/281: dread - d0/d11/f54 zero size 2024-04-20T10:59:15.352 INFO:tasks.workunit.client.0.smithi027.stdout:9/233: dread d4/f1c [4194304,4194304] 0 2024-04-20T10:59:15.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/320: dwrite d0/d14/d17/d1b/d2f/f73 [0,4194304] 0 2024-04-20T10:59:15.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/321: fdatasync d0/d14/d17/d1b/d33/f35 0 2024-04-20T10:59:15.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/322: readlink d0/l10 0 2024-04-20T10:59:15.354 INFO:tasks.workunit.client.0.smithi027.stdout:1/389: unlink d3/d7/f42 0 2024-04-20T10:59:15.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/323: write d0/d14/d17/d1b/d2f/f82 [43644,10858] 0 2024-04-20T10:59:15.355 INFO:tasks.workunit.client.0.smithi027.stdout:7/297: rename d4/d5/d8/d3b/f4b to d4/d52/f56 0 2024-04-20T10:59:15.356 INFO:tasks.workunit.client.0.smithi027.stdout:3/379: dread d2/d18/f26 [0,4194304] 0 2024-04-20T10:59:15.356 INFO:tasks.workunit.client.0.smithi027.stdout:9/234: rename d4/d1b/d31/d23/d3b to d4/d1b/d1f/d45 0 2024-04-20T10:59:15.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/235: rename d4/d1b/f42 to d4/d2a/f46 0 2024-04-20T10:59:15.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/236: fdatasync d4/d1b/d1f/d45/f3f 0 2024-04-20T10:59:15.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/237: getdents d4 0 2024-04-20T10:59:15.359 INFO:tasks.workunit.client.0.smithi027.stdout:5/293: dwrite d19/d1b/d1c/d1e/d3c/f46 [4194304,4194304] 0 2024-04-20T10:59:15.360 INFO:tasks.workunit.client.0.smithi027.stdout:8/368: dwrite d9/de/d1e/f66 [0,4194304] 0 2024-04-20T10:59:15.360 INFO:tasks.workunit.client.0.smithi027.stdout:8/369: write d9/de/d1e/f68 [837308,77047] 0 2024-04-20T10:59:15.360 INFO:tasks.workunit.client.0.smithi027.stdout:9/238: creat d4/d1b/d31/d23/f47 x:0 0 0 2024-04-20T10:59:15.362 INFO:tasks.workunit.client.0.smithi027.stdout:5/294: rename d19/d1b/d1c/d1e/d22/f38 to d19/d1b/d1c/d1e/d58/f61 0 2024-04-20T10:59:15.362 INFO:tasks.workunit.client.0.smithi027.stdout:5/295: fdatasync d19/d1b/d1c/d1e/d3c/f39 0 2024-04-20T10:59:15.363 INFO:tasks.workunit.client.0.smithi027.stdout:2/211: dwrite d5/d1e/d21/f27 [0,4194304] 0 2024-04-20T10:59:15.366 INFO:tasks.workunit.client.0.smithi027.stdout:4/324: read d0/d14/d17/d53/d6a/f7c [2289231,59234] 0 2024-04-20T10:59:15.367 INFO:tasks.workunit.client.0.smithi027.stdout:8/370: link d9/de/d20/d53/c6b d9/de/c89 0 2024-04-20T10:59:15.368 INFO:tasks.workunit.client.0.smithi027.stdout:8/371: fdatasync d9/de/d20/d48/d59/d74/f80 0 2024-04-20T10:59:15.369 INFO:tasks.workunit.client.0.smithi027.stdout:9/239: mknod d4/d1b/c48 0 2024-04-20T10:59:15.370 INFO:tasks.workunit.client.0.smithi027.stdout:5/296: mknod d19/d1b/d1c/c62 0 2024-04-20T10:59:15.373 INFO:tasks.workunit.client.0.smithi027.stdout:2/212: dread d5/dc/fe [0,4194304] 0 2024-04-20T10:59:15.374 INFO:tasks.workunit.client.0.smithi027.stdout:8/372: rmdir d9/de/d1e/d49 39 2024-04-20T10:59:15.374 INFO:tasks.workunit.client.0.smithi027.stdout:1/390: dwrite d3/d45/d5d/d6c/f90 [0,4194304] 0 2024-04-20T10:59:15.374 INFO:tasks.workunit.client.0.smithi027.stdout:8/373: read - d9/de/d1e/d4a/d6d/d75/f81 zero size 2024-04-20T10:59:15.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/240: creat d4/d14/f49 x:0 0 0 2024-04-20T10:59:15.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/241: dread - d4/d2a/f46 zero size 2024-04-20T10:59:15.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/242: readlink d4/l39 0 2024-04-20T10:59:15.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/243: write d4/d14/f44 [563733,40993] 0 2024-04-20T10:59:15.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/213: mkdir d5/d26/d47 0 2024-04-20T10:59:15.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/214: fsync d5/d26/f3b 0 2024-04-20T10:59:15.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/215: chown d5/d44 76 1 2024-04-20T10:59:15.380 INFO:tasks.workunit.client.0.smithi027.stdout:8/374: truncate d9/de/d64/d6c/d87/d3c/f44 170045 0 2024-04-20T10:59:15.382 INFO:tasks.workunit.client.0.smithi027.stdout:5/297: read f0 [9666933,52396] 0 2024-04-20T10:59:15.382 INFO:tasks.workunit.client.0.smithi027.stdout:3/380: dwrite d2/d18/f53 [0,4194304] 0 2024-04-20T10:59:15.382 INFO:tasks.workunit.client.0.smithi027.stdout:3/381: chown d2/dd/df/d23/l48 690093 1 2024-04-20T10:59:15.383 INFO:tasks.workunit.client.0.smithi027.stdout:1/391: link d3/d7/l14 d3/d7e/l91 0 2024-04-20T10:59:15.384 INFO:tasks.workunit.client.0.smithi027.stdout:9/244: unlink d4/l18 0 2024-04-20T10:59:15.390 INFO:tasks.workunit.client.0.smithi027.stdout:8/375: symlink d9/de/d20/d48/l8a 0 2024-04-20T10:59:15.390 INFO:tasks.workunit.client.0.smithi027.stdout:8/376: write d9/de/d20/d48/d59/f60 [2920790,80147] 0 2024-04-20T10:59:15.390 INFO:tasks.workunit.client.0.smithi027.stdout:8/377: write d9/f15 [477738,128401] 0 2024-04-20T10:59:15.392 INFO:tasks.workunit.client.0.smithi027.stdout:1/392: mkdir d3/d7/d28/d92 0 2024-04-20T10:59:15.394 INFO:tasks.workunit.client.0.smithi027.stdout:9/245: truncate d4/d1b/d31/d23/f36 2594142 0 2024-04-20T10:59:15.394 INFO:tasks.workunit.client.0.smithi027.stdout:9/246: readlink d4/d1b/d1f/l26 0 2024-04-20T10:59:15.397 INFO:tasks.workunit.client.0.smithi027.stdout:8/378: truncate d9/de/d1e/d4a/f35 2815988 0 2024-04-20T10:59:15.398 INFO:tasks.workunit.client.0.smithi027.stdout:9/247: link d4/f7 d4/f4a 0 2024-04-20T10:59:15.400 INFO:tasks.workunit.client.0.smithi027.stdout:8/379: symlink d9/de/d20/d53/l8b 0 2024-04-20T10:59:15.400 INFO:tasks.workunit.client.0.smithi027.stdout:5/298: dread f17 [0,4194304] 0 2024-04-20T10:59:15.402 INFO:tasks.workunit.client.0.smithi027.stdout:9/248: mkdir d4/d1b/d31/d23/d29/d4b 0 2024-04-20T10:59:15.403 INFO:tasks.workunit.client.0.smithi027.stdout:3/382: write d2/f16 [5292028,22956] 0 2024-04-20T10:59:15.403 INFO:tasks.workunit.client.0.smithi027.stdout:3/383: read d2/dd/f10 [7585506,43061] 0 2024-04-20T10:59:15.403 INFO:tasks.workunit.client.0.smithi027.stdout:8/380: link d9/de/d64/d6c/d87/f7f d9/de/d64/d6c/f8c 0 2024-04-20T10:59:15.404 INFO:tasks.workunit.client.0.smithi027.stdout:9/249: unlink d4/l39 0 2024-04-20T10:59:15.405 INFO:tasks.workunit.client.0.smithi027.stdout:3/384: symlink d2/dd/d57/l74 0 2024-04-20T10:59:15.405 INFO:tasks.workunit.client.0.smithi027.stdout:8/381: unlink d9/de/d1e/d4a/l40 0 2024-04-20T10:59:15.406 INFO:tasks.workunit.client.0.smithi027.stdout:9/250: creat d4/d1b/d31/d23/d29/f4c x:0 0 0 2024-04-20T10:59:15.409 INFO:tasks.workunit.client.0.smithi027.stdout:3/385: dread d2/f17 [0,4194304] 0 2024-04-20T10:59:15.409 INFO:tasks.workunit.client.0.smithi027.stdout:4/325: dwrite d0/d14/d17/d1a/f70 [0,4194304] 0 2024-04-20T10:59:15.410 INFO:tasks.workunit.client.0.smithi027.stdout:4/326: write d0/d14/d17/d18/f4b [318014,1878] 0 2024-04-20T10:59:15.410 INFO:tasks.workunit.client.0.smithi027.stdout:8/382: getdents d9/de/d20/d48/d59 0 2024-04-20T10:59:15.410 INFO:tasks.workunit.client.0.smithi027.stdout:8/383: symlink d9/de/d20/d48/d59/d74/l8d 2 2024-04-20T10:59:15.410 INFO:tasks.workunit.client.0.smithi027.stdout:8/384: fsync d9/de/d18/f2d 0 2024-04-20T10:59:15.412 INFO:tasks.workunit.client.0.smithi027.stdout:2/216: dwrite d5/f28 [0,4194304] 0 2024-04-20T10:59:15.412 INFO:tasks.workunit.client.0.smithi027.stdout:5/299: dread d19/d1b/d1c/f3b [0,4194304] 0 2024-04-20T10:59:15.426 INFO:tasks.workunit.client.0.smithi027.stdout:8/385: read d9/de/d20/d4f/d5b/f84 [2583761,104509] 0 2024-04-20T10:59:15.444 INFO:tasks.workunit.client.0.smithi027.stdout:4/327: dwrite d0/d14/d17/d1b/d2f/f82 [0,4194304] 0 2024-04-20T10:59:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:8/386: getdents d9/de/d64/d6c/d87/d3c/d4b 0 2024-04-20T10:59:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:8/387: stat d9/de 0 2024-04-20T10:59:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/300: truncate fc 1910048 0 2024-04-20T10:59:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/301: stat f0 0 2024-04-20T10:59:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/302: chown d19/d1b/c53 537 1 2024-04-20T10:59:15.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/388: dread d9/de/d64/d6c/d87/d3c/f44 [0,4194304] 0 2024-04-20T10:59:15.446 INFO:tasks.workunit.client.0.smithi027.stdout:4/328: mknod d0/d14/d17/d1b/d33/d5c/c86 0 2024-04-20T10:59:15.448 INFO:tasks.workunit.client.0.smithi027.stdout:3/386: dwrite d2/d18/f3f [0,4194304] 0 2024-04-20T10:59:15.448 INFO:tasks.workunit.client.0.smithi027.stdout:2/217: dwrite d5/dc/fe [0,4194304] 0 2024-04-20T10:59:15.448 INFO:tasks.workunit.client.0.smithi027.stdout:5/303: mknod d19/d1b/d1c/d1e/d3c/c63 0 2024-04-20T10:59:15.449 INFO:tasks.workunit.client.0.smithi027.stdout:8/389: getdents d9/de/d1e/d4a 0 2024-04-20T10:59:15.450 INFO:tasks.workunit.client.0.smithi027.stdout:4/329: mknod d0/d14/d17/d1b/d33/d5c/d6f/c87 0 2024-04-20T10:59:15.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/251: dwrite d4/d1b/d31/d23/f36 [0,4194304] 0 2024-04-20T10:59:15.457 INFO:tasks.workunit.client.0.smithi027.stdout:2/218: link f1 d5/d26/d3d/f48 0 2024-04-20T10:59:15.457 INFO:tasks.workunit.client.0.smithi027.stdout:5/304: mkdir d19/d1b/d1c/d1e/d3c/d64 0 2024-04-20T10:59:15.463 INFO:tasks.workunit.client.0.smithi027.stdout:9/252: symlink d4/d2a/l4d 0 2024-04-20T10:59:15.463 INFO:tasks.workunit.client.0.smithi027.stdout:2/219: symlink d5/d1e/d21/l49 0 2024-04-20T10:59:15.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/305: link d19/d1b/d1c/d1e/d22/c4b d19/d1b/d1c/d1e/d22/d47/c65 0 2024-04-20T10:59:15.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/306: write fb [1204813,107560] 0 2024-04-20T10:59:15.463 INFO:tasks.workunit.client.0.smithi027.stdout:6/289: sync 2024-04-20T10:59:15.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/307: mknod d19/d1b/d1c/c66 0 2024-04-20T10:59:15.463 INFO:tasks.workunit.client.0.smithi027.stdout:6/290: symlink da/dd/d3d/l68 0 2024-04-20T10:59:15.463 INFO:tasks.workunit.client.0.smithi027.stdout:6/291: write da/df/d19/d2e/d32/f55 [913432,82931] 0 2024-04-20T10:59:15.463 INFO:tasks.workunit.client.0.smithi027.stdout:6/292: creat da/dd/f69 x:0 0 0 2024-04-20T10:59:15.464 INFO:tasks.workunit.client.0.smithi027.stdout:6/293: creat da/df/d19/f6a x:0 0 0 2024-04-20T10:59:15.466 INFO:tasks.workunit.client.0.smithi027.stdout:6/294: rename da/df/d19/d2e/d42 to da/df/d19/d2e/d6b 0 2024-04-20T10:59:15.466 INFO:tasks.workunit.client.0.smithi027.stdout:6/295: write da/d21/d28/d2d/d53/f5e [1033791,109398] 0 2024-04-20T10:59:15.470 INFO:tasks.workunit.client.0.smithi027.stdout:9/253: dread d4/d1b/d31/f3c [0,4194304] 0 2024-04-20T10:59:15.471 INFO:tasks.workunit.client.0.smithi027.stdout:6/296: link da/dd/l50 da/l6c 0 2024-04-20T10:59:15.471 INFO:tasks.workunit.client.0.smithi027.stdout:9/254: mknod d4/d1b/d31/d23/c4e 0 2024-04-20T10:59:15.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/308: write d19/d1b/d1c/f45 [2164687,48600] 0 2024-04-20T10:59:15.471 INFO:tasks.workunit.client.0.smithi027.stdout:4/330: dwrite d0/d14/d17/d18/f4b [0,4194304] 0 2024-04-20T10:59:15.474 INFO:tasks.workunit.client.0.smithi027.stdout:5/309: dread d19/f27 [0,4194304] 0 2024-04-20T10:59:15.476 INFO:tasks.workunit.client.0.smithi027.stdout:9/255: mkdir d4/d14/d43/d4f 0 2024-04-20T10:59:15.476 INFO:tasks.workunit.client.0.smithi027.stdout:4/331: rename d0/d14/d17/d1b/d33/d5c/d6f/c7d to d0/d14/d17/d1a/c88 0 2024-04-20T10:59:15.476 INFO:tasks.workunit.client.0.smithi027.stdout:4/332: getdents d0/d14/d6b 0 2024-04-20T10:59:15.477 INFO:tasks.workunit.client.0.smithi027.stdout:9/256: readlink d4/d2a/l2b 0 2024-04-20T10:59:15.477 INFO:tasks.workunit.client.0.smithi027.stdout:9/257: truncate d4/d14/f49 173962 0 2024-04-20T10:59:15.477 INFO:tasks.workunit.client.0.smithi027.stdout:9/258: chown d4/d14/f49 130096508 1 2024-04-20T10:59:15.478 INFO:tasks.workunit.client.0.smithi027.stdout:2/220: dwrite d5/d26/f2c [0,4194304] 0 2024-04-20T10:59:15.488 INFO:tasks.workunit.client.0.smithi027.stdout:4/333: write d0/d14/d17/d18/f2d [963501,119122] 0 2024-04-20T10:59:15.488 INFO:tasks.workunit.client.0.smithi027.stdout:4/334: stat d0/d14/d17/d1b/d33/f57 0 2024-04-20T10:59:15.489 INFO:tasks.workunit.client.0.smithi027.stdout:0/282: sync 2024-04-20T10:59:15.489 INFO:tasks.workunit.client.0.smithi027.stdout:6/297: dwrite da/dd/f59 [0,4194304] 0 2024-04-20T10:59:15.491 INFO:tasks.workunit.client.0.smithi027.stdout:0/283: creat d0/d11/d2c/d2e/d55/f56 x:0 0 0 2024-04-20T10:59:15.491 INFO:tasks.workunit.client.0.smithi027.stdout:0/284: dread - d0/d7/d12/d25/f31 zero size 2024-04-20T10:59:15.495 INFO:tasks.workunit.client.0.smithi027.stdout:2/221: dread d5/dc/fe [0,4194304] 0 2024-04-20T10:59:15.496 INFO:tasks.workunit.client.0.smithi027.stdout:2/222: mkdir d5/dc/d16/d2a/d4a 0 2024-04-20T10:59:15.497 INFO:tasks.workunit.client.0.smithi027.stdout:9/259: dwrite d4/d14/f44 [0,4194304] 0 2024-04-20T10:59:15.502 INFO:tasks.workunit.client.0.smithi027.stdout:9/260: dread d4/f1e [0,4194304] 0 2024-04-20T10:59:15.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/261: mkdir d4/d1b/d50 0 2024-04-20T10:59:15.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/262: write d4/d14/f21 [1805099,20598] 0 2024-04-20T10:59:15.506 INFO:tasks.workunit.client.0.smithi027.stdout:9/263: rmdir d4/d14/d43 39 2024-04-20T10:59:15.508 INFO:tasks.workunit.client.0.smithi027.stdout:9/264: mkdir d4/d1b/d31/d51 0 2024-04-20T10:59:15.510 INFO:tasks.workunit.client.0.smithi027.stdout:9/265: dread d4/d14/f19 [0,4194304] 0 2024-04-20T10:59:15.512 INFO:tasks.workunit.client.0.smithi027.stdout:9/266: link d4/d1b/f24 d4/d14/d43/f52 0 2024-04-20T10:59:15.512 INFO:tasks.workunit.client.0.smithi027.stdout:9/267: chown d4/f1c 10 1 2024-04-20T10:59:15.512 INFO:tasks.workunit.client.0.smithi027.stdout:9/268: read d4/d14/d43/f52 [2270480,122077] 0 2024-04-20T10:59:15.512 INFO:tasks.workunit.client.0.smithi027.stdout:2/223: dwrite d5/d26/f2c [0,4194304] 0 2024-04-20T10:59:15.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/224: creat d5/f4b x:0 0 0 2024-04-20T10:59:15.520 INFO:tasks.workunit.client.0.smithi027.stdout:2/225: write d5/d26/f2c [2740611,30793] 0 2024-04-20T10:59:15.520 INFO:tasks.workunit.client.0.smithi027.stdout:2/226: write d5/dc/f1c [5172781,8579] 0 2024-04-20T10:59:15.525 INFO:tasks.workunit.client.0.smithi027.stdout:9/269: dread d4/d14/f38 [0,4194304] 0 2024-04-20T10:59:15.525 INFO:tasks.workunit.client.0.smithi027.stdout:9/270: write d4/d1b/d31/d23/d29/f35 [10602,129628] 0 2024-04-20T10:59:15.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/271: rmdir d4/d14 39 2024-04-20T10:59:15.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/272: chown d4/d1b/d31/d23 1802279 1 2024-04-20T10:59:15.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/273: write d4/d1b/d31/d23/f47 [201483,42180] 0 2024-04-20T10:59:15.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/274: fsync d4/f7 0 2024-04-20T10:59:15.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/275: fsync d4/d14/f27 0 2024-04-20T10:59:15.527 INFO:tasks.workunit.client.0.smithi027.stdout:9/276: creat d4/d1b/f53 x:0 0 0 2024-04-20T10:59:15.527 INFO:tasks.workunit.client.0.smithi027.stdout:9/277: truncate d4/d1b/d31/d23/d29/f4c 400829 0 2024-04-20T10:59:15.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/278: creat d4/d2a/f54 x:0 0 0 2024-04-20T10:59:15.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/279: readlink d4/d1b/d1f/l26 0 2024-04-20T10:59:15.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/280: creat d4/d14/d43/d4f/f55 x:0 0 0 2024-04-20T10:59:15.530 INFO:tasks.workunit.client.0.smithi027.stdout:4/335: dwrite d0/d14/d17/d1a/f70 [0,4194304] 0 2024-04-20T10:59:15.530 INFO:tasks.workunit.client.0.smithi027.stdout:4/336: fsync d0/d14/d17/d1b/f61 0 2024-04-20T10:59:15.531 INFO:tasks.workunit.client.0.smithi027.stdout:4/337: read d0/d14/d17/d1b/d2f/f3f [1307691,23759] 0 2024-04-20T10:59:15.531 INFO:tasks.workunit.client.0.smithi027.stdout:6/298: dwrite da/df/f4f [0,4194304] 0 2024-04-20T10:59:15.531 INFO:tasks.workunit.client.0.smithi027.stdout:6/299: chown da/df/d1f/f3f 1 1 2024-04-20T10:59:15.531 INFO:tasks.workunit.client.0.smithi027.stdout:9/281: creat d4/d1b/d50/f56 x:0 0 0 2024-04-20T10:59:15.531 INFO:tasks.workunit.client.0.smithi027.stdout:4/338: creat d0/d14/d17/d53/d79/f89 x:0 0 0 2024-04-20T10:59:15.531 INFO:tasks.workunit.client.0.smithi027.stdout:0/285: dwrite d0/d11/f54 [0,4194304] 0 2024-04-20T10:59:15.535 INFO:tasks.workunit.client.0.smithi027.stdout:9/282: mkdir d4/d1b/d1f/d57 0 2024-04-20T10:59:15.535 INFO:tasks.workunit.client.0.smithi027.stdout:9/283: truncate d4/d1b/d31/d23/f47 1226955 0 2024-04-20T10:59:15.535 INFO:tasks.workunit.client.0.smithi027.stdout:4/339: getdents d0/d14/d17/d1b/d33/d5c 0 2024-04-20T10:59:15.536 INFO:tasks.workunit.client.0.smithi027.stdout:2/227: dread d5/d1e/f42 [4194304,4194304] 0 2024-04-20T10:59:15.539 INFO:tasks.workunit.client.0.smithi027.stdout:9/284: mknod d4/d1b/c58 0 2024-04-20T10:59:15.544 INFO:tasks.workunit.client.0.smithi027.stdout:4/340: creat d0/d14/d17/d1a/f8a x:0 0 0 2024-04-20T10:59:15.544 INFO:tasks.workunit.client.0.smithi027.stdout:2/228: rename d5/dc/d16/c35 to d5/dc/d16/d2a/d4a/c4c 0 2024-04-20T10:59:15.545 INFO:tasks.workunit.client.0.smithi027.stdout:1/393: sync 2024-04-20T10:59:15.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/298: sync 2024-04-20T10:59:15.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/299: write d4/d5/d8/d21/f31 [2273486,3028] 0 2024-04-20T10:59:15.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/394: mknod d3/d7/d38/c93 0 2024-04-20T10:59:15.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/395: write d3/d7/d28/f8b [530743,98408] 0 2024-04-20T10:59:15.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/396: dread - d3/d7/d38/d62/f6e zero size 2024-04-20T10:59:15.550 INFO:tasks.workunit.client.0.smithi027.stdout:6/300: dwrite da/dd/f59 [0,4194304] 0 2024-04-20T10:59:15.555 INFO:tasks.workunit.client.0.smithi027.stdout:2/229: dread d5/f28 [0,4194304] 0 2024-04-20T10:59:15.555 INFO:tasks.workunit.client.0.smithi027.stdout:2/230: mkdir d5/dc/d16/d2a/d4d 0 2024-04-20T10:59:15.557 INFO:tasks.workunit.client.0.smithi027.stdout:2/231: truncate d5/dc/f1c 1664782 0 2024-04-20T10:59:15.557 INFO:tasks.workunit.client.0.smithi027.stdout:2/232: read - d5/d26/f3a zero size 2024-04-20T10:59:15.564 INFO:tasks.workunit.client.0.smithi027.stdout:8/390: sync 2024-04-20T10:59:15.565 INFO:tasks.workunit.client.0.smithi027.stdout:3/387: sync 2024-04-20T10:59:15.565 INFO:tasks.workunit.client.0.smithi027.stdout:3/388: chown d2/d18/d2e/c70 22906 1 2024-04-20T10:59:15.565 INFO:tasks.workunit.client.0.smithi027.stdout:2/233: creat d5/dc/d16/d37/f4e x:0 0 0 2024-04-20T10:59:15.565 INFO:tasks.workunit.client.0.smithi027.stdout:8/391: rmdir d9/de/d64 39 2024-04-20T10:59:15.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/234: mknod d5/dc/d16/d32/c4f 0 2024-04-20T10:59:15.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/235: write d5/dc/f19 [1150895,36016] 0 2024-04-20T10:59:15.567 INFO:tasks.workunit.client.0.smithi027.stdout:8/392: truncate f2 2810333 0 2024-04-20T10:59:15.567 INFO:tasks.workunit.client.0.smithi027.stdout:8/393: write d9/de/d1e/f65 [2826220,52220] 0 2024-04-20T10:59:15.567 INFO:tasks.workunit.client.0.smithi027.stdout:8/394: fdatasync d9/de/f1d 0 2024-04-20T10:59:15.567 INFO:tasks.workunit.client.0.smithi027.stdout:2/236: creat d5/dc/d16/d2a/d4a/f50 x:0 0 0 2024-04-20T10:59:15.570 INFO:tasks.workunit.client.0.smithi027.stdout:2/237: mknod d5/d1e/d21/c51 0 2024-04-20T10:59:15.570 INFO:tasks.workunit.client.0.smithi027.stdout:2/238: getdents d5/d44 0 2024-04-20T10:59:15.570 INFO:tasks.workunit.client.0.smithi027.stdout:9/285: dwrite d4/d1b/d50/f56 [0,4194304] 0 2024-04-20T10:59:15.571 INFO:tasks.workunit.client.0.smithi027.stdout:7/300: dwrite d4/d1b/d4a/f44 [0,4194304] 0 2024-04-20T10:59:15.571 INFO:tasks.workunit.client.0.smithi027.stdout:1/397: dwrite d3/d7/fc [0,4194304] 0 2024-04-20T10:59:15.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/398: dread - d3/d7/f7a zero size 2024-04-20T10:59:15.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/399: stat d3/d7/d38/f78 0 2024-04-20T10:59:15.572 INFO:tasks.workunit.client.0.smithi027.stdout:2/239: creat d5/dc/d16/d2a/f52 x:0 0 0 2024-04-20T10:59:15.572 INFO:tasks.workunit.client.0.smithi027.stdout:2/240: fdatasync f1 0 2024-04-20T10:59:15.572 INFO:tasks.workunit.client.0.smithi027.stdout:2/241: write d5/f4b [178339,106503] 0 2024-04-20T10:59:15.578 INFO:tasks.workunit.client.0.smithi027.stdout:7/301: rename d4/d1b/c16 to d4/d5/d8/d45/c57 0 2024-04-20T10:59:15.578 INFO:tasks.workunit.client.0.smithi027.stdout:7/302: write d4/d1b/f53 [648689,76633] 0 2024-04-20T10:59:15.581 INFO:tasks.workunit.client.0.smithi027.stdout:1/400: link d3/d7/ff d3/d7/d2d/d73/f94 0 2024-04-20T10:59:15.581 INFO:tasks.workunit.client.0.smithi027.stdout:1/401: fdatasync d3/d7/f23 0 2024-04-20T10:59:15.581 INFO:tasks.workunit.client.0.smithi027.stdout:1/402: write d3/d7/d38/f78 [1039525,77314] 0 2024-04-20T10:59:15.581 INFO:tasks.workunit.client.0.smithi027.stdout:1/403: chown d3/f1e 484916123 1 2024-04-20T10:59:15.581 INFO:tasks.workunit.client.0.smithi027.stdout:1/404: readlink d3/d7/d2d/d73/l89 0 2024-04-20T10:59:15.582 INFO:tasks.workunit.client.0.smithi027.stdout:6/301: dwrite da/df/d19/f6a [0,4194304] 0 2024-04-20T10:59:15.582 INFO:tasks.workunit.client.0.smithi027.stdout:6/302: truncate da/df/d19/f5d 919200 0 2024-04-20T10:59:15.582 INFO:tasks.workunit.client.0.smithi027.stdout:2/242: symlink d5/d26/l53 0 2024-04-20T10:59:15.583 INFO:tasks.workunit.client.0.smithi027.stdout:7/303: mkdir d4/d1b/d58 0 2024-04-20T10:59:15.584 INFO:tasks.workunit.client.0.smithi027.stdout:1/405: rename d3/c4 to d3/d41/c95 0 2024-04-20T10:59:15.584 INFO:tasks.workunit.client.0.smithi027.stdout:1/406: fdatasync d3/d20/d85/f86 0 2024-04-20T10:59:15.587 INFO:tasks.workunit.client.0.smithi027.stdout:6/303: truncate da/fb 40056 0 2024-04-20T10:59:15.587 INFO:tasks.workunit.client.0.smithi027.stdout:6/304: dread - da/dd/f69 zero size 2024-04-20T10:59:15.590 INFO:tasks.workunit.client.0.smithi027.stdout:2/243: creat d5/dc/d16/d2a/f54 x:0 0 0 2024-04-20T10:59:15.592 INFO:tasks.workunit.client.0.smithi027.stdout:5/310: sync 2024-04-20T10:59:15.593 INFO:tasks.workunit.client.0.smithi027.stdout:1/407: rmdir d3/d20 39 2024-04-20T10:59:15.594 INFO:tasks.workunit.client.0.smithi027.stdout:8/395: dwrite d9/de/d64/d6c/d87/d3c/f6f [0,4194304] 0 2024-04-20T10:59:15.595 INFO:tasks.workunit.client.0.smithi027.stdout:9/286: dwrite d4/d14/f49 [0,4194304] 0 2024-04-20T10:59:15.595 INFO:tasks.workunit.client.0.smithi027.stdout:1/408: creat d3/d20/f96 x:0 0 0 2024-04-20T10:59:15.597 INFO:tasks.workunit.client.0.smithi027.stdout:2/244: rename d5/dc/d16/d2a/f52 to d5/dc/f55 0 2024-04-20T10:59:15.599 INFO:tasks.workunit.client.0.smithi027.stdout:5/311: read d19/f27 [38842,107946] 0 2024-04-20T10:59:15.600 INFO:tasks.workunit.client.0.smithi027.stdout:1/409: write d3/d7/f34 [2251233,121168] 0 2024-04-20T10:59:15.600 INFO:tasks.workunit.client.0.smithi027.stdout:9/287: rmdir d4/d14/d43/d4f 39 2024-04-20T10:59:15.601 INFO:tasks.workunit.client.0.smithi027.stdout:7/304: dwrite d4/d5/fa [0,4194304] 0 2024-04-20T10:59:15.602 INFO:tasks.workunit.client.0.smithi027.stdout:2/245: creat d5/d1e/f56 x:0 0 0 2024-04-20T10:59:15.603 INFO:tasks.workunit.client.0.smithi027.stdout:1/410: mknod d3/d7/d28/c97 0 2024-04-20T10:59:15.604 INFO:tasks.workunit.client.0.smithi027.stdout:9/288: symlink d4/l59 0 2024-04-20T10:59:15.605 INFO:tasks.workunit.client.0.smithi027.stdout:2/246: dread d5/d26/f3b [0,4194304] 0 2024-04-20T10:59:15.605 INFO:tasks.workunit.client.0.smithi027.stdout:2/247: fsync d5/dc/f1c 0 2024-04-20T10:59:15.605 INFO:tasks.workunit.client.0.smithi027.stdout:6/305: truncate da/fb 426943 0 2024-04-20T10:59:15.606 INFO:tasks.workunit.client.0.smithi027.stdout:5/312: dread d19/d1b/d1c/d1e/d3c/f31 [0,4194304] 0 2024-04-20T10:59:15.607 INFO:tasks.workunit.client.0.smithi027.stdout:9/289: rename d4/d14/d43/d4f to d4/d1b/d1f/d5a 0 2024-04-20T10:59:15.607 INFO:tasks.workunit.client.0.smithi027.stdout:9/290: dread - d4/d1b/f53 zero size 2024-04-20T10:59:15.607 INFO:tasks.workunit.client.0.smithi027.stdout:7/305: dread d4/f2e [0,4194304] 0 2024-04-20T10:59:15.608 INFO:tasks.workunit.client.0.smithi027.stdout:2/248: mknod d5/d1e/d21/c57 0 2024-04-20T10:59:15.609 INFO:tasks.workunit.client.0.smithi027.stdout:6/306: symlink da/df/l6d 0 2024-04-20T10:59:15.610 INFO:tasks.workunit.client.0.smithi027.stdout:5/313: truncate d19/d1b/d1c/d1e/f56 332975 0 2024-04-20T10:59:15.610 INFO:tasks.workunit.client.0.smithi027.stdout:5/314: fdatasync d19/d1b/d1c/d1e/d3c/f31 0 2024-04-20T10:59:15.611 INFO:tasks.workunit.client.0.smithi027.stdout:7/306: mknod d4/d5/d8/c59 0 2024-04-20T10:59:15.612 INFO:tasks.workunit.client.0.smithi027.stdout:2/249: creat d5/f58 x:0 0 0 2024-04-20T10:59:15.613 INFO:tasks.workunit.client.0.smithi027.stdout:6/307: link da/df/d1f/c2c da/dd/d3d/c6e 0 2024-04-20T10:59:15.616 INFO:tasks.workunit.client.0.smithi027.stdout:8/396: dwrite d9/de/d1e/d4a/f35 [0,4194304] 0 2024-04-20T10:59:15.616 INFO:tasks.workunit.client.0.smithi027.stdout:8/397: truncate d9/de/f71 572712 0 2024-04-20T10:59:15.616 INFO:tasks.workunit.client.0.smithi027.stdout:8/398: readlink d9/de/l10 0 2024-04-20T10:59:15.616 INFO:tasks.workunit.client.0.smithi027.stdout:2/250: mkdir d5/dc/d16/d2a/d59 0 2024-04-20T10:59:15.617 INFO:tasks.workunit.client.0.smithi027.stdout:7/307: link d4/d5/f1f d4/f5a 0 2024-04-20T10:59:15.620 INFO:tasks.workunit.client.0.smithi027.stdout:7/308: write d4/f2e [588528,65343] 0 2024-04-20T10:59:15.620 INFO:tasks.workunit.client.0.smithi027.stdout:8/399: rename d9/de/d1e/d4a/f4e to d9/de/d20/d48/d59/f8e 0 2024-04-20T10:59:15.621 INFO:tasks.workunit.client.0.smithi027.stdout:9/291: dread d4/d1b/d31/d23/f36 [0,4194304] 0 2024-04-20T10:59:15.621 INFO:tasks.workunit.client.0.smithi027.stdout:2/251: creat d5/dc/d16/d32/f5a x:0 0 0 2024-04-20T10:59:15.621 INFO:tasks.workunit.client.0.smithi027.stdout:2/252: write d5/d26/f2c [2377917,41683] 0 2024-04-20T10:59:15.622 INFO:tasks.workunit.client.0.smithi027.stdout:8/400: rename d9/de/d1e/d4a/d6d/d75/f81 to d9/de/d1e/d4a/f8f 0 2024-04-20T10:59:15.625 INFO:tasks.workunit.client.0.smithi027.stdout:0/286: sync 2024-04-20T10:59:15.625 INFO:tasks.workunit.client.0.smithi027.stdout:0/287: readlink d0/d7/d12/l29 0 2024-04-20T10:59:15.625 INFO:tasks.workunit.client.0.smithi027.stdout:0/288: truncate d0/d7/d12/f3e 27270 0 2024-04-20T10:59:15.626 INFO:tasks.workunit.client.0.smithi027.stdout:0/289: dread d0/d7/d34/f3b [0,4194304] 0 2024-04-20T10:59:15.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/389: dread d2/d18/d2e/f35 [0,4194304] 0 2024-04-20T10:59:15.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/390: fdatasync d2/f66 0 2024-04-20T10:59:15.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/391: write d2/d18/d2c/f4f [1453825,24222] 0 2024-04-20T10:59:15.627 INFO:tasks.workunit.client.0.smithi027.stdout:7/309: truncate d4/f26 1189364 0 2024-04-20T10:59:15.627 INFO:tasks.workunit.client.0.smithi027.stdout:7/310: write d4/d5/d8/d21/f48 [1465369,69573] 0 2024-04-20T10:59:15.628 INFO:tasks.workunit.client.0.smithi027.stdout:7/311: readlink d4/d5/l55 0 2024-04-20T10:59:15.629 INFO:tasks.workunit.client.0.smithi027.stdout:0/290: creat d0/d7/d4e/f57 x:0 0 0 2024-04-20T10:59:15.629 INFO:tasks.workunit.client.0.smithi027.stdout:0/291: write d0/f42 [21361,27388] 0 2024-04-20T10:59:15.631 INFO:tasks.workunit.client.0.smithi027.stdout:0/292: mknod d0/c58 0 2024-04-20T10:59:15.631 INFO:tasks.workunit.client.0.smithi027.stdout:0/293: dread - d0/d11/d2c/d2e/d55/f56 zero size 2024-04-20T10:59:15.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/312: mkdir d4/d5b 0 2024-04-20T10:59:15.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/315: dwrite d19/d1b/f33 [0,4194304] 0 2024-04-20T10:59:15.634 INFO:tasks.workunit.client.0.smithi027.stdout:0/294: mkdir d0/d59 0 2024-04-20T10:59:15.634 INFO:tasks.workunit.client.0.smithi027.stdout:0/295: chown d0/d11/f32 299 1 2024-04-20T10:59:15.635 INFO:tasks.workunit.client.0.smithi027.stdout:8/401: dread d9/de/d64/d6c/d87/d3c/f6f [0,4194304] 0 2024-04-20T10:59:15.636 INFO:tasks.workunit.client.0.smithi027.stdout:8/402: truncate d9/de/f1d 2255768 0 2024-04-20T10:59:15.638 INFO:tasks.workunit.client.0.smithi027.stdout:5/316: rename d19/f27 to d19/d1b/d1c/d1e/d22/f67 0 2024-04-20T10:59:15.639 INFO:tasks.workunit.client.0.smithi027.stdout:3/392: dread d2/f14 [0,4194304] 0 2024-04-20T10:59:15.639 INFO:tasks.workunit.client.0.smithi027.stdout:3/393: readlink d2/d18/d2e/l37 0 2024-04-20T10:59:15.639 INFO:tasks.workunit.client.0.smithi027.stdout:4/341: sync 2024-04-20T10:59:15.639 INFO:tasks.workunit.client.0.smithi027.stdout:0/296: rmdir d0/d7/d4e 39 2024-04-20T10:59:15.641 INFO:tasks.workunit.client.0.smithi027.stdout:5/317: truncate d19/d1b/d1c/d1e/f56 225792 0 2024-04-20T10:59:15.641 INFO:tasks.workunit.client.0.smithi027.stdout:5/318: fsync d19/d1b/d1c/d1e/d3c/f46 0 2024-04-20T10:59:15.641 INFO:tasks.workunit.client.0.smithi027.stdout:1/411: dwrite d3/f1e [0,4194304] 0 2024-04-20T10:59:15.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/394: creat d2/d18/d2e/f75 x:0 0 0 2024-04-20T10:59:15.644 INFO:tasks.workunit.client.0.smithi027.stdout:0/297: rmdir d0/d7/d12/d1a 39 2024-04-20T10:59:15.644 INFO:tasks.workunit.client.0.smithi027.stdout:8/403: dread d9/de/f5e [0,4194304] 0 2024-04-20T10:59:15.645 INFO:tasks.workunit.client.0.smithi027.stdout:5/319: mkdir d19/d1b/d1c/d1e/d22/d68 0 2024-04-20T10:59:15.647 INFO:tasks.workunit.client.0.smithi027.stdout:1/412: creat d3/d7/d2d/d73/f98 x:0 0 0 2024-04-20T10:59:15.647 INFO:tasks.workunit.client.0.smithi027.stdout:5/320: dread d19/d1b/d1c/d1e/d3c/f3d [0,4194304] 0 2024-04-20T10:59:15.648 INFO:tasks.workunit.client.0.smithi027.stdout:5/321: fdatasync d19/d1b/f33 0 2024-04-20T10:59:15.648 INFO:tasks.workunit.client.0.smithi027.stdout:8/404: creat d9/de/d64/d6c/d87/d3c/f90 x:0 0 0 2024-04-20T10:59:15.649 INFO:tasks.workunit.client.0.smithi027.stdout:5/322: mknod d19/d1b/d1c/d1e/d22/d47/c69 0 2024-04-20T10:59:15.650 INFO:tasks.workunit.client.0.smithi027.stdout:5/323: read f18 [635832,120261] 0 2024-04-20T10:59:15.650 INFO:tasks.workunit.client.0.smithi027.stdout:8/405: symlink d9/de/d64/l91 0 2024-04-20T10:59:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:5/324: creat d19/d1b/d1c/f6a x:0 0 0 2024-04-20T10:59:15.651 INFO:tasks.workunit.client.0.smithi027.stdout:5/325: write f0 [8255036,2017] 0 2024-04-20T10:59:15.652 INFO:tasks.workunit.client.0.smithi027.stdout:7/313: dwrite d4/d5/fa [4194304,4194304] 0 2024-04-20T10:59:15.653 INFO:tasks.workunit.client.0.smithi027.stdout:5/326: creat d19/d1b/d1c/d1e/d22/d47/f6b x:0 0 0 2024-04-20T10:59:15.655 INFO:tasks.workunit.client.0.smithi027.stdout:7/314: rename d4/f26 to d4/d5/f5c 0 2024-04-20T10:59:15.656 INFO:tasks.workunit.client.0.smithi027.stdout:2/253: dwrite d5/d1e/f56 [0,4194304] 0 2024-04-20T10:59:15.658 INFO:tasks.workunit.client.0.smithi027.stdout:5/327: creat d19/d1b/d1c/d1e/d58/f6c x:0 0 0 2024-04-20T10:59:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:0/298: dread d0/d7/f1c [4194304,4194304] 0 2024-04-20T10:59:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/328: chown d19/c23 29651562 1 2024-04-20T10:59:15.659 INFO:tasks.workunit.client.0.smithi027.stdout:0/299: fdatasync d0/f42 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:2/254: unlink d5/d26/f3b 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:2/255: chown d5/dc 224 1 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:5/329: mknod d19/c6d 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:5/330: stat d19/d1b/d1c/d1e/d22/d47/f6b 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:5/331: readlink d19/d1b/d1c/d1e/l54 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:2/256: write d5/d1e/d21/f3e [354997,25398] 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:2/257: readlink d5/d26/l53 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:0/300: mkdir d0/d7/d34/d5a 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:5/332: mknod d19/d1b/d1c/d1e/c6e 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:5/333: fdatasync fb 0 2024-04-20T10:59:15.667 INFO:tasks.workunit.client.0.smithi027.stdout:5/334: chown d19/d1b/f33 196313549 1 2024-04-20T10:59:15.668 INFO:tasks.workunit.client.0.smithi027.stdout:7/315: write d4/d5/fa [3957540,119740] 0 2024-04-20T10:59:15.668 INFO:tasks.workunit.client.0.smithi027.stdout:0/301: mkdir d0/d11/d4b/d5b 0 2024-04-20T10:59:15.668 INFO:tasks.workunit.client.0.smithi027.stdout:7/316: mkdir d4/d4d/d5d 0 2024-04-20T10:59:15.668 INFO:tasks.workunit.client.0.smithi027.stdout:7/317: chown d4/d5/d8/d3b/f3e 117256 1 2024-04-20T10:59:15.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/302: truncate d0/d7/d12/d1a/f2d 2472563 0 2024-04-20T10:59:15.669 INFO:tasks.workunit.client.0.smithi027.stdout:0/303: chown d0/d11/d2c 19 1 2024-04-20T10:59:15.670 INFO:tasks.workunit.client.0.smithi027.stdout:3/395: write d2/dd/d22/f2f [3390167,102209] 0 2024-04-20T10:59:15.670 INFO:tasks.workunit.client.0.smithi027.stdout:3/396: fsync d2/d18/f63 0 2024-04-20T10:59:15.670 INFO:tasks.workunit.client.0.smithi027.stdout:3/397: stat d2/d18/f29 0 2024-04-20T10:59:15.671 INFO:tasks.workunit.client.0.smithi027.stdout:3/398: write d2/d18/d2e/f75 [510413,125184] 0 2024-04-20T10:59:15.671 INFO:tasks.workunit.client.0.smithi027.stdout:7/318: symlink d4/d5/l5e 0 2024-04-20T10:59:15.671 INFO:tasks.workunit.client.0.smithi027.stdout:7/319: rmdir d4/d4d/d5d 0 2024-04-20T10:59:15.671 INFO:tasks.workunit.client.0.smithi027.stdout:7/320: truncate d4/d5/f33 930495 0 2024-04-20T10:59:15.672 INFO:tasks.workunit.client.0.smithi027.stdout:0/304: write d0/d7/f26 [1398271,105012] 0 2024-04-20T10:59:15.673 INFO:tasks.workunit.client.0.smithi027.stdout:5/335: write d19/f4d [671862,111158] 0 2024-04-20T10:59:15.674 INFO:tasks.workunit.client.0.smithi027.stdout:0/305: mkdir d0/d11/d5c 0 2024-04-20T10:59:15.675 INFO:tasks.workunit.client.0.smithi027.stdout:5/336: symlink d19/d1b/d1c/d1e/d22/l6f 0 2024-04-20T10:59:15.676 INFO:tasks.workunit.client.0.smithi027.stdout:2/258: truncate d5/d1e/f56 984790 0 2024-04-20T10:59:15.676 INFO:tasks.workunit.client.0.smithi027.stdout:1/413: dwrite d3/d45/d5d/d6c/f90 [0,4194304] 0 2024-04-20T10:59:15.676 INFO:tasks.workunit.client.0.smithi027.stdout:1/414: write d3/d7/d2d/f4a [181650,108032] 0 2024-04-20T10:59:15.676 INFO:tasks.workunit.client.0.smithi027.stdout:1/415: fsync d3/d7/d38/d62/f81 0 2024-04-20T10:59:15.676 INFO:tasks.workunit.client.0.smithi027.stdout:0/306: mkdir d0/d7/d12/d1a/d47/d5d 0 2024-04-20T10:59:15.678 INFO:tasks.workunit.client.0.smithi027.stdout:2/259: symlink d5/dc/d16/l5b 0 2024-04-20T10:59:15.679 INFO:tasks.workunit.client.0.smithi027.stdout:8/406: dwrite d9/de/d20/d53/f77 [0,4194304] 0 2024-04-20T10:59:15.679 INFO:tasks.workunit.client.0.smithi027.stdout:8/407: fdatasync d9/de/d20/d4f/d5b/f88 0 2024-04-20T10:59:15.680 INFO:tasks.workunit.client.0.smithi027.stdout:1/416: creat d3/d7/d2d/f99 x:0 0 0 2024-04-20T10:59:15.681 INFO:tasks.workunit.client.0.smithi027.stdout:2/260: creat d5/d1e/f5c x:0 0 0 2024-04-20T10:59:15.682 INFO:tasks.workunit.client.0.smithi027.stdout:2/261: fsync d5/dc/d16/d2a/f54 0 2024-04-20T10:59:15.689 INFO:tasks.workunit.client.0.smithi027.stdout:2/262: dread d5/d26/d3d/f48 [0,4194304] 0 2024-04-20T10:59:15.690 INFO:tasks.workunit.client.0.smithi027.stdout:3/399: dwrite d2/d18/d2c/f30 [0,4194304] 0 2024-04-20T10:59:15.692 INFO:tasks.workunit.client.0.smithi027.stdout:0/307: dread d0/d7/d12/d1a/f2d [0,4194304] 0 2024-04-20T10:59:15.692 INFO:tasks.workunit.client.0.smithi027.stdout:1/417: creat d3/d45/f9a x:0 0 0 2024-04-20T10:59:15.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/308: dread d0/d7/d12/f3e [0,4194304] 0 2024-04-20T10:59:15.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/309: mknod d0/d11/d4b/c5e 0 2024-04-20T10:59:15.695 INFO:tasks.workunit.client.0.smithi027.stdout:0/310: getdents d0/d7/d12/d1a/d47 0 2024-04-20T10:59:15.695 INFO:tasks.workunit.client.0.smithi027.stdout:0/311: truncate d0/d7/d4e/f57 881244 0 2024-04-20T10:59:15.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/263: write d5/d1e/d21/f27 [1136458,122204] 0 2024-04-20T10:59:15.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/264: rename d5 to d5/d1e/d21/d5d 22 2024-04-20T10:59:15.707 INFO:tasks.workunit.client.0.smithi027.stdout:1/418: write d3/d7/f57 [1730068,525] 0 2024-04-20T10:59:15.707 INFO:tasks.workunit.client.0.smithi027.stdout:8/408: dwrite d9/de/d20/d48/d59/f60 [0,4194304] 0 2024-04-20T10:59:15.709 INFO:tasks.workunit.client.0.smithi027.stdout:1/419: getdents d3/d41 0 2024-04-20T10:59:15.710 INFO:tasks.workunit.client.0.smithi027.stdout:8/409: unlink d9/f39 0 2024-04-20T10:59:15.711 INFO:tasks.workunit.client.0.smithi027.stdout:8/410: stat d9/de/d20/d48/d59/f8e 0 2024-04-20T10:59:15.711 INFO:tasks.workunit.client.0.smithi027.stdout:1/420: mkdir d3/d7/d2d/d9b 0 2024-04-20T10:59:15.711 INFO:tasks.workunit.client.0.smithi027.stdout:1/421: write d3/d7e/f83 [368483,78098] 0 2024-04-20T10:59:15.711 INFO:tasks.workunit.client.0.smithi027.stdout:0/312: dwrite d0/d7/f26 [4194304,4194304] 0 2024-04-20T10:59:15.712 INFO:tasks.workunit.client.0.smithi027.stdout:8/411: creat d9/de/d1e/d4a/d6d/d79/f92 x:0 0 0 2024-04-20T10:59:15.712 INFO:tasks.workunit.client.0.smithi027.stdout:8/412: chown d9/de/d18/f2d 2 1 2024-04-20T10:59:15.713 INFO:tasks.workunit.client.0.smithi027.stdout:1/422: mkdir d3/d7/d38/d9c 0 2024-04-20T10:59:15.715 INFO:tasks.workunit.client.0.smithi027.stdout:0/313: symlink d0/d7/d12/d1a/d47/l5f 0 2024-04-20T10:59:15.715 INFO:tasks.workunit.client.0.smithi027.stdout:0/314: read d0/f42 [41406,103080] 0 2024-04-20T10:59:15.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/413: creat d9/de/d64/d6c/f93 x:0 0 0 2024-04-20T10:59:15.718 INFO:tasks.workunit.client.0.smithi027.stdout:1/423: creat d3/d20/d85/f9d x:0 0 0 2024-04-20T10:59:15.718 INFO:tasks.workunit.client.0.smithi027.stdout:1/424: fdatasync d3/d7/d28/f3d 0 2024-04-20T10:59:15.720 INFO:tasks.workunit.client.0.smithi027.stdout:2/265: dwrite d5/dc/f41 [0,4194304] 0 2024-04-20T10:59:15.720 INFO:tasks.workunit.client.0.smithi027.stdout:2/266: stat d5/dc/d16/d2a/f40 0 2024-04-20T10:59:15.720 INFO:tasks.workunit.client.0.smithi027.stdout:2/267: dread - d5/dc/d16/d2a/d4a/f50 zero size 2024-04-20T10:59:15.720 INFO:tasks.workunit.client.0.smithi027.stdout:0/315: unlink d0/d7/d12/d1a/d47/c51 0 2024-04-20T10:59:15.720 INFO:tasks.workunit.client.0.smithi027.stdout:0/316: fdatasync d0/d7/d12/f3e 0 2024-04-20T10:59:15.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/414: rename d9/de/d20/d4f/c56 to d9/de/d1e/d4a/d6d/d75/c94 0 2024-04-20T10:59:15.724 INFO:tasks.workunit.client.0.smithi027.stdout:1/425: symlink d3/d20/d85/l9e 0 2024-04-20T10:59:15.724 INFO:tasks.workunit.client.0.smithi027.stdout:8/415: dread d9/de/f1d [0,4194304] 0 2024-04-20T10:59:15.726 INFO:tasks.workunit.client.0.smithi027.stdout:2/268: unlink d5/d1e/d21/f27 0 2024-04-20T10:59:15.728 INFO:tasks.workunit.client.0.smithi027.stdout:0/317: unlink d0/d7/d12/d1a/l3f 0 2024-04-20T10:59:15.728 INFO:tasks.workunit.client.0.smithi027.stdout:0/318: stat d0/d7/d12/d25/f31 0 2024-04-20T10:59:15.729 INFO:tasks.workunit.client.0.smithi027.stdout:8/416: mkdir d9/de/d20/d95 0 2024-04-20T10:59:15.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/269: mkdir d5/d26/d47/d5e 0 2024-04-20T10:59:15.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/270: write d5/d26/f3a [989314,93105] 0 2024-04-20T10:59:15.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/271: fsync d5/dc/f1c 0 2024-04-20T10:59:15.732 INFO:tasks.workunit.client.0.smithi027.stdout:0/319: mknod d0/d7/c60 0 2024-04-20T10:59:15.733 INFO:tasks.workunit.client.0.smithi027.stdout:8/417: mknod d9/de/d20/c96 0 2024-04-20T10:59:15.734 INFO:tasks.workunit.client.0.smithi027.stdout:0/320: dread d0/d7/d12/d1a/f50 [0,4194304] 0 2024-04-20T10:59:15.734 INFO:tasks.workunit.client.0.smithi027.stdout:0/321: getdents d0/d7/d12/d1a/d47/d5d 0 2024-04-20T10:59:15.738 INFO:tasks.workunit.client.0.smithi027.stdout:0/322: mkdir d0/d11/d2c/d2e/d61 0 2024-04-20T10:59:15.739 INFO:tasks.workunit.client.0.smithi027.stdout:0/323: rename d0/d7/d4e to d0/d11/d62 0 2024-04-20T10:59:15.740 INFO:tasks.workunit.client.0.smithi027.stdout:8/418: dread d9/de/d1e/d49/f5c [0,4194304] 0 2024-04-20T10:59:15.740 INFO:tasks.workunit.client.0.smithi027.stdout:8/419: dread - d9/de/d20/d48/f85 zero size 2024-04-20T10:59:15.740 INFO:tasks.workunit.client.0.smithi027.stdout:0/324: rmdir d0/d11/d2c/d2e/d2b/d3d 39 2024-04-20T10:59:15.741 INFO:tasks.workunit.client.0.smithi027.stdout:0/325: creat d0/d11/d5c/f63 x:0 0 0 2024-04-20T10:59:15.743 INFO:tasks.workunit.client.0.smithi027.stdout:6/308: sync 2024-04-20T10:59:15.744 INFO:tasks.workunit.client.0.smithi027.stdout:9/292: sync 2024-04-20T10:59:15.745 INFO:tasks.workunit.client.0.smithi027.stdout:9/293: getdents d4/d14 0 2024-04-20T10:59:15.745 INFO:tasks.workunit.client.0.smithi027.stdout:9/294: read d4/d1b/f22 [519796,42864] 0 2024-04-20T10:59:15.746 INFO:tasks.workunit.client.0.smithi027.stdout:9/295: truncate d4/d1b/d31/d23/d29/f35 814694 0 2024-04-20T10:59:15.746 INFO:tasks.workunit.client.0.smithi027.stdout:9/296: truncate d4/d1b/f22 1049810 0 2024-04-20T10:59:15.746 INFO:tasks.workunit.client.0.smithi027.stdout:9/297: stat d4/d2a 0 2024-04-20T10:59:15.746 INFO:tasks.workunit.client.0.smithi027.stdout:9/298: mknod d4/d1b/d31/d51/c5b 0 2024-04-20T10:59:15.748 INFO:tasks.workunit.client.0.smithi027.stdout:9/299: link d4/d14/l37 d4/d14/l5c 0 2024-04-20T10:59:15.751 INFO:tasks.workunit.client.0.smithi027.stdout:0/326: dread d0/d7/fa [4194304,4194304] 0 2024-04-20T10:59:15.752 INFO:tasks.workunit.client.0.smithi027.stdout:0/327: mkdir d0/d7/d64 0 2024-04-20T10:59:15.754 INFO:tasks.workunit.client.0.smithi027.stdout:0/328: creat d0/d11/f65 x:0 0 0 2024-04-20T10:59:15.755 INFO:tasks.workunit.client.0.smithi027.stdout:9/300: getdents d4/d14 0 2024-04-20T10:59:15.756 INFO:tasks.workunit.client.0.smithi027.stdout:9/301: write d4/d1b/d1f/d45/f3f [112175,122469] 0 2024-04-20T10:59:15.757 INFO:tasks.workunit.client.0.smithi027.stdout:9/302: mkdir d4/d1b/d50/d5d 0 2024-04-20T10:59:15.766 INFO:tasks.workunit.client.0.smithi027.stdout:8/420: dwrite d9/de/d64/d6c/d87/f30 [0,4194304] 0 2024-04-20T10:59:15.767 INFO:tasks.workunit.client.0.smithi027.stdout:6/309: dwrite da/d21/d28/f34 [0,4194304] 0 2024-04-20T10:59:15.768 INFO:tasks.workunit.client.0.smithi027.stdout:6/310: dread da/df/d19/f5d [0,4194304] 0 2024-04-20T10:59:15.768 INFO:tasks.workunit.client.0.smithi027.stdout:6/311: fdatasync da/dd/d3d/f64 0 2024-04-20T10:59:15.768 INFO:tasks.workunit.client.0.smithi027.stdout:6/312: write da/df/d19/d2e/f45 [806223,10654] 0 2024-04-20T10:59:15.768 INFO:tasks.workunit.client.0.smithi027.stdout:6/313: write da/df/d1f/f3f [1603229,51507] 0 2024-04-20T10:59:15.768 INFO:tasks.workunit.client.0.smithi027.stdout:8/421: mkdir d9/de/d20/d4f/d97 0 2024-04-20T10:59:15.770 INFO:tasks.workunit.client.0.smithi027.stdout:6/314: creat da/d21/d28/d2d/d53/f6f x:0 0 0 2024-04-20T10:59:15.771 INFO:tasks.workunit.client.0.smithi027.stdout:8/422: mknod d9/de/d20/d95/c98 0 2024-04-20T10:59:15.772 INFO:tasks.workunit.client.0.smithi027.stdout:6/315: truncate da/f1b 980180 0 2024-04-20T10:59:15.772 INFO:tasks.workunit.client.0.smithi027.stdout:6/316: write da/df/d19/d2e/d6b/f62 [116199,19291] 0 2024-04-20T10:59:15.772 INFO:tasks.workunit.client.0.smithi027.stdout:6/317: write da/dd/f69 [521255,72135] 0 2024-04-20T10:59:15.772 INFO:tasks.workunit.client.0.smithi027.stdout:0/329: dwrite d0/d7/d12/f44 [0,4194304] 0 2024-04-20T10:59:15.772 INFO:tasks.workunit.client.0.smithi027.stdout:8/423: creat d9/de/d20/d48/d59/f99 x:0 0 0 2024-04-20T10:59:15.779 INFO:tasks.workunit.client.0.smithi027.stdout:0/330: mknod d0/d11/d2c/d2e/d2b/c66 0 2024-04-20T10:59:15.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/331: write d0/d11/d62/f57 [789079,5539] 0 2024-04-20T10:59:15.780 INFO:tasks.workunit.client.0.smithi027.stdout:8/424: symlink d9/de/d20/d4f/l9a 0 2024-04-20T10:59:15.780 INFO:tasks.workunit.client.0.smithi027.stdout:8/425: chown d9/de/d20/d4f/d97 69109034 1 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:8/426: truncate d9/de/d1e/d49/f5c 4932214 0 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:8/427: chown d9/de/d64/c76 129070 1 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:8/428: chown d9/de/d64/c6e 118371 1 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:5/337: sync 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/342: sync 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/321: sync 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/322: read - d4/d5/d8/f4c zero size 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/323: readlink d4/d1b/l1c 0 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/324: getdents d4/d1b/d58 0 2024-04-20T10:59:15.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/343: write d0/d14/d17/d53/d6a/f7c [4988841,105686] 0 2024-04-20T10:59:15.782 INFO:tasks.workunit.client.0.smithi027.stdout:0/332: mkdir d0/d7/d12/d1a/d47/d5d/d67 0 2024-04-20T10:59:15.783 INFO:tasks.workunit.client.0.smithi027.stdout:8/429: creat d9/de/d1e/d4a/d6d/f9b x:0 0 0 2024-04-20T10:59:15.783 INFO:tasks.workunit.client.0.smithi027.stdout:5/338: truncate f18 777160 0 2024-04-20T10:59:15.783 INFO:tasks.workunit.client.0.smithi027.stdout:5/339: write d19/d1b/d1c/d1e/d58/f61 [820697,82609] 0 2024-04-20T10:59:15.783 INFO:tasks.workunit.client.0.smithi027.stdout:5/340: chown d19/c1a 16546671 1 2024-04-20T10:59:15.792 INFO:tasks.workunit.client.0.smithi027.stdout:7/325: creat d4/d1b/d58/f5f x:0 0 0 2024-04-20T10:59:15.792 INFO:tasks.workunit.client.0.smithi027.stdout:4/344: creat d0/d14/d71/f8b x:0 0 0 2024-04-20T10:59:15.792 INFO:tasks.workunit.client.0.smithi027.stdout:4/345: chown d0/l46 95605326 1 2024-04-20T10:59:15.792 INFO:tasks.workunit.client.0.smithi027.stdout:5/341: mknod d19/d1b/d1c/d1e/d25/d40/c70 0 2024-04-20T10:59:15.792 INFO:tasks.workunit.client.0.smithi027.stdout:5/342: stat d19/d1b/d1c/d1e/d22/c59 0 2024-04-20T10:59:15.792 INFO:tasks.workunit.client.0.smithi027.stdout:0/333: creat d0/d59/f68 x:0 0 0 2024-04-20T10:59:15.795 INFO:tasks.workunit.client.0.smithi027.stdout:0/334: creat d0/d7/d12/d1a/d47/f69 x:0 0 0 2024-04-20T10:59:15.799 INFO:tasks.workunit.client.0.smithi027.stdout:7/326: write d4/d1b/f35 [986208,101679] 0 2024-04-20T10:59:15.800 INFO:tasks.workunit.client.0.smithi027.stdout:6/318: dwrite da/dd/f25 [0,4194304] 0 2024-04-20T10:59:15.800 INFO:tasks.workunit.client.0.smithi027.stdout:7/327: chown d4/d5/d8/d21/l2c 6136 1 2024-04-20T10:59:15.800 INFO:tasks.workunit.client.0.smithi027.stdout:7/328: dread - d4/d5/d8/f4c zero size 2024-04-20T10:59:15.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/272: write d5/d1e/f56 [694325,80225] 0 2024-04-20T10:59:15.800 INFO:tasks.workunit.client.0.smithi027.stdout:2/273: dread - d5/d1e/f5c zero size 2024-04-20T10:59:15.803 INFO:tasks.workunit.client.0.smithi027.stdout:2/274: dread d5/d26/f3a [0,4194304] 0 2024-04-20T10:59:15.804 INFO:tasks.workunit.client.0.smithi027.stdout:8/430: dwrite d9/de/d64/d6c/d87/d3c/f44 [0,4194304] 0 2024-04-20T10:59:15.804 INFO:tasks.workunit.client.0.smithi027.stdout:8/431: write d9/de/d1e/d4a/d6d/d75/f83 [301560,50523] 0 2024-04-20T10:59:15.810 INFO:tasks.workunit.client.0.smithi027.stdout:5/343: read d19/d1b/d1c/d1e/d58/f5e [99066,10456] 0 2024-04-20T10:59:15.833 INFO:tasks.workunit.client.0.smithi027.stdout:8/432: dwrite d9/de/d64/d6c/d87/f7f [0,4194304] 0 2024-04-20T10:59:15.835 INFO:tasks.workunit.client.0.smithi027.stdout:8/433: rename d9/de/d20/d48/d59/f8e to d9/de/d64/d6c/d87/d3c/f9c 0 2024-04-20T10:59:15.836 INFO:tasks.workunit.client.0.smithi027.stdout:8/434: creat d9/de/d20/f9d x:0 0 0 2024-04-20T10:59:15.837 INFO:tasks.workunit.client.0.smithi027.stdout:2/275: dwrite d5/dc/d16/d2a/f54 [0,4194304] 0 2024-04-20T10:59:15.837 INFO:tasks.workunit.client.0.smithi027.stdout:2/276: stat d5/d1e/d21/c57 0 2024-04-20T10:59:15.838 INFO:tasks.workunit.client.0.smithi027.stdout:4/346: dwrite d0/d14/d17/d1b/d2f/f73 [0,4194304] 0 2024-04-20T10:59:15.838 INFO:tasks.workunit.client.0.smithi027.stdout:4/347: truncate d0/d14/d17/f31 176109 0 2024-04-20T10:59:15.838 INFO:tasks.workunit.client.0.smithi027.stdout:8/435: truncate d9/de/d1e/d4a/f45 2226479 0 2024-04-20T10:59:15.838 INFO:tasks.workunit.client.0.smithi027.stdout:0/335: dwrite d0/d11/d2c/d2e/d55/f56 [0,4194304] 0 2024-04-20T10:59:15.838 INFO:tasks.workunit.client.0.smithi027.stdout:0/336: dread - d0/d7/d12/d1a/d47/f69 zero size 2024-04-20T10:59:15.838 INFO:tasks.workunit.client.0.smithi027.stdout:6/319: dwrite da/d21/d28/d2d/d53/f6f [0,4194304] 0 2024-04-20T10:59:15.839 INFO:tasks.workunit.client.0.smithi027.stdout:7/329: dwrite d4/d5/d8/d3b/f3e [0,4194304] 0 2024-04-20T10:59:15.840 INFO:tasks.workunit.client.0.smithi027.stdout:2/277: rename d5/dc/f19 to d5/d26/d47/f5f 0 2024-04-20T10:59:15.840 INFO:tasks.workunit.client.0.smithi027.stdout:2/278: write d5/dc/f1c [2204849,47536] 0 2024-04-20T10:59:15.841 INFO:tasks.workunit.client.0.smithi027.stdout:2/279: fsync d5/dc/d16/d2a/f40 0 2024-04-20T10:59:15.841 INFO:tasks.workunit.client.0.smithi027.stdout:2/280: dread - d5/dc/d16/d32/f5a zero size 2024-04-20T10:59:15.843 INFO:tasks.workunit.client.0.smithi027.stdout:6/320: fdatasync da/df/d19/d2e/f45 0 2024-04-20T10:59:15.843 INFO:tasks.workunit.client.0.smithi027.stdout:6/321: chown da/df/l6d 7646 1 2024-04-20T10:59:15.843 INFO:tasks.workunit.client.0.smithi027.stdout:4/348: truncate d0/f28 2496009 0 2024-04-20T10:59:15.843 INFO:tasks.workunit.client.0.smithi027.stdout:6/322: write da/df/d19/d2e/d32/f55 [1138434,129061] 0 2024-04-20T10:59:15.843 INFO:tasks.workunit.client.0.smithi027.stdout:8/436: getdents d9/de/d64/d6c 0 2024-04-20T10:59:15.843 INFO:tasks.workunit.client.0.smithi027.stdout:8/437: chown d9/de/d64/d6c/d87/d3c/f61 615287 1 2024-04-20T10:59:15.843 INFO:tasks.workunit.client.0.smithi027.stdout:4/349: dread d0/d14/d17/d1b/d33/f35 [0,4194304] 0 2024-04-20T10:59:15.859 INFO:tasks.workunit.client.0.smithi027.stdout:7/330: creat d4/d4d/f60 x:0 0 0 2024-04-20T10:59:15.859 INFO:tasks.workunit.client.0.smithi027.stdout:7/331: chown d4/f2e 4165 1 2024-04-20T10:59:15.859 INFO:tasks.workunit.client.0.smithi027.stdout:7/332: chown d4/d1b/d4a 306 1 2024-04-20T10:59:15.859 INFO:tasks.workunit.client.0.smithi027.stdout:7/333: fsync d4/d1b/d58/f5f 0 2024-04-20T10:59:15.859 INFO:tasks.workunit.client.0.smithi027.stdout:8/438: creat d9/de/d64/d6c/f9e x:0 0 0 2024-04-20T10:59:15.859 INFO:tasks.workunit.client.0.smithi027.stdout:4/350: unlink d0/c2c 0 2024-04-20T10:59:15.870 INFO:tasks.workunit.client.0.smithi027.stdout:0/337: dwrite d0/d7/d12/d25/f31 [0,4194304] 0 2024-04-20T10:59:15.870 INFO:tasks.workunit.client.0.smithi027.stdout:2/281: dwrite d5/d1e/f5c [0,4194304] 0 2024-04-20T10:59:15.870 INFO:tasks.workunit.client.0.smithi027.stdout:8/439: dread d9/de/d64/d6c/d87/f30 [0,4194304] 0 2024-04-20T10:59:15.871 INFO:tasks.workunit.client.0.smithi027.stdout:8/440: chown d9/de/d20/d53 15063441 1 2024-04-20T10:59:15.871 INFO:tasks.workunit.client.0.smithi027.stdout:0/338: dread d0/d7/d12/f3e [0,4194304] 0 2024-04-20T10:59:15.871 INFO:tasks.workunit.client.0.smithi027.stdout:0/339: fsync d0/f42 0 2024-04-20T10:59:15.872 INFO:tasks.workunit.client.0.smithi027.stdout:1/426: sync 2024-04-20T10:59:15.872 INFO:tasks.workunit.client.0.smithi027.stdout:9/303: sync 2024-04-20T10:59:15.872 INFO:tasks.workunit.client.0.smithi027.stdout:3/400: sync 2024-04-20T10:59:15.873 INFO:tasks.workunit.client.0.smithi027.stdout:8/441: unlink d9/de/d20/d4f/d5b/f88 0 2024-04-20T10:59:15.873 INFO:tasks.workunit.client.0.smithi027.stdout:8/442: read d9/de/f3a [149473,47318] 0 2024-04-20T10:59:15.873 INFO:tasks.workunit.client.0.smithi027.stdout:3/401: read d2/dd/d57/f59 [176491,107837] 0 2024-04-20T10:59:15.874 INFO:tasks.workunit.client.0.smithi027.stdout:6/323: dwrite da/df/d19/d2e/d32/f44 [0,4194304] 0 2024-04-20T10:59:15.874 INFO:tasks.workunit.client.0.smithi027.stdout:1/427: creat d3/d7e/f9f x:0 0 0 2024-04-20T10:59:15.874 INFO:tasks.workunit.client.0.smithi027.stdout:1/428: dread - d3/d7/d2d/d73/f98 zero size 2024-04-20T10:59:15.875 INFO:tasks.workunit.client.0.smithi027.stdout:9/304: symlink d4/d1b/d31/d23/d29/d4b/l5e 0 2024-04-20T10:59:15.877 INFO:tasks.workunit.client.0.smithi027.stdout:4/351: dwrite d0/d14/d17/d1a/f70 [0,4194304] 0 2024-04-20T10:59:15.878 INFO:tasks.workunit.client.0.smithi027.stdout:8/443: creat d9/de/d1e/d4a/d6d/d7d/f9f x:0 0 0 2024-04-20T10:59:15.878 INFO:tasks.workunit.client.0.smithi027.stdout:8/444: chown d9/de/d1e/d4a/c36 276 1 2024-04-20T10:59:15.879 INFO:tasks.workunit.client.0.smithi027.stdout:3/402: rmdir d2/d18/d2c/d72 0 2024-04-20T10:59:15.879 INFO:tasks.workunit.client.0.smithi027.stdout:3/403: truncate d2/d18/d2c/f69 358544 0 2024-04-20T10:59:15.879 INFO:tasks.workunit.client.0.smithi027.stdout:3/404: chown d2/dd/df/d23/d68 56082579 1 2024-04-20T10:59:15.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/340: dread d0/d11/f32 [0,4194304] 0 2024-04-20T10:59:15.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/341: chown d0/d11/d2c/d2e/d2b/d3d 49790 1 2024-04-20T10:59:15.880 INFO:tasks.workunit.client.0.smithi027.stdout:7/334: dwrite d4/d5/f14 [0,4194304] 0 2024-04-20T10:59:15.882 INFO:tasks.workunit.client.0.smithi027.stdout:2/282: dread d5/dc/fe [0,4194304] 0 2024-04-20T10:59:15.882 INFO:tasks.workunit.client.0.smithi027.stdout:2/283: read - d5/dc/f55 zero size 2024-04-20T10:59:15.882 INFO:tasks.workunit.client.0.smithi027.stdout:6/324: creat da/d21/f70 x:0 0 0 2024-04-20T10:59:15.883 INFO:tasks.workunit.client.0.smithi027.stdout:9/305: mknod d4/d1b/d31/d23/d29/d4b/c5f 0 2024-04-20T10:59:15.885 INFO:tasks.workunit.client.0.smithi027.stdout:8/445: mkdir d9/de/d20/d48/d59/da0 0 2024-04-20T10:59:15.885 INFO:tasks.workunit.client.0.smithi027.stdout:6/325: dread da/fb [0,4194304] 0 2024-04-20T10:59:15.885 INFO:tasks.workunit.client.0.smithi027.stdout:3/405: creat d2/d18/f76 x:0 0 0 2024-04-20T10:59:15.888 INFO:tasks.workunit.client.0.smithi027.stdout:1/429: read d3/d7/d28/f48 [2846385,42368] 0 2024-04-20T10:59:15.889 INFO:tasks.workunit.client.0.smithi027.stdout:6/326: mknod da/df/d1f/c71 0 2024-04-20T10:59:15.890 INFO:tasks.workunit.client.0.smithi027.stdout:3/406: creat d2/dd/f77 x:0 0 0 2024-04-20T10:59:15.890 INFO:tasks.workunit.client.0.smithi027.stdout:0/342: dread d0/f20 [0,4194304] 0 2024-04-20T10:59:15.892 INFO:tasks.workunit.client.0.smithi027.stdout:1/430: creat d3/d7/d28/d35/fa0 x:0 0 0 2024-04-20T10:59:15.893 INFO:tasks.workunit.client.0.smithi027.stdout:6/327: unlink da/df/f33 0 2024-04-20T10:59:15.895 INFO:tasks.workunit.client.0.smithi027.stdout:3/407: creat d2/d18/d2c/f78 x:0 0 0 2024-04-20T10:59:15.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/343: getdents d0/d11/d2c/d2e/d2b 0 2024-04-20T10:59:15.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/344: chown d0/d7/d12/d1a/d47/l5f 1985684 1 2024-04-20T10:59:15.898 INFO:tasks.workunit.client.0.smithi027.stdout:1/431: mknod d3/d7/d28/ca1 0 2024-04-20T10:59:15.900 INFO:tasks.workunit.client.0.smithi027.stdout:4/352: dwrite d0/d14/d17/d18/f4b [0,4194304] 0 2024-04-20T10:59:15.900 INFO:tasks.workunit.client.0.smithi027.stdout:9/306: dread d4/f1c [0,4194304] 0 2024-04-20T10:59:15.900 INFO:tasks.workunit.client.0.smithi027.stdout:6/328: rmdir da/df/d19/d2e/d32 39 2024-04-20T10:59:15.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/408: symlink d2/d18/d2e/l79 0 2024-04-20T10:59:15.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/409: truncate d2/dd/f4b 1369342 0 2024-04-20T10:59:15.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/410: readlink d2/dd/d57/l74 0 2024-04-20T10:59:15.901 INFO:tasks.workunit.client.0.smithi027.stdout:0/345: creat d0/d59/f6a x:0 0 0 2024-04-20T10:59:15.902 INFO:tasks.workunit.client.0.smithi027.stdout:1/432: mknod d3/d45/ca2 0 2024-04-20T10:59:15.904 INFO:tasks.workunit.client.0.smithi027.stdout:4/353: unlink d0/d14/d17/d18/f39 0 2024-04-20T10:59:15.904 INFO:tasks.workunit.client.0.smithi027.stdout:9/307: symlink d4/d14/d43/l60 0 2024-04-20T10:59:15.905 INFO:tasks.workunit.client.0.smithi027.stdout:6/329: truncate da/dd/f25 1133408 0 2024-04-20T10:59:15.906 INFO:tasks.workunit.client.0.smithi027.stdout:3/411: rmdir d2/d18/d2c 39 2024-04-20T10:59:15.907 INFO:tasks.workunit.client.0.smithi027.stdout:2/284: dwrite d5/d1e/f5c [0,4194304] 0 2024-04-20T10:59:15.907 INFO:tasks.workunit.client.0.smithi027.stdout:3/412: write d2/dd/f4b [98780,78266] 0 2024-04-20T10:59:15.907 INFO:tasks.workunit.client.0.smithi027.stdout:0/346: creat d0/d11/d2c/d2e/d55/f6b x:0 0 0 2024-04-20T10:59:15.910 INFO:tasks.workunit.client.0.smithi027.stdout:9/308: dread d4/d1b/d1f/d45/f3f [0,4194304] 0 2024-04-20T10:59:15.910 INFO:tasks.workunit.client.0.smithi027.stdout:9/309: chown l3 35 1 2024-04-20T10:59:15.910 INFO:tasks.workunit.client.0.smithi027.stdout:3/413: mknod d2/d18/d2c/d4c/c7a 0 2024-04-20T10:59:15.910 INFO:tasks.workunit.client.0.smithi027.stdout:8/446: dwrite d9/de/d1e/d4a/d6d/d75/f83 [0,4194304] 0 2024-04-20T10:59:15.912 INFO:tasks.workunit.client.0.smithi027.stdout:0/347: creat d0/d7/d12/d1a/f6c x:0 0 0 2024-04-20T10:59:15.914 INFO:tasks.workunit.client.0.smithi027.stdout:9/310: creat d4/d2a/f61 x:0 0 0 2024-04-20T10:59:15.920 INFO:tasks.workunit.client.0.smithi027.stdout:9/311: symlink d4/d1b/d31/d51/l62 0 2024-04-20T10:59:15.920 INFO:tasks.workunit.client.0.smithi027.stdout:9/312: write d4/d14/f19 [3741230,97465] 0 2024-04-20T10:59:15.920 INFO:tasks.workunit.client.0.smithi027.stdout:9/313: truncate d4/d1b/d31/d23/f47 1714231 0 2024-04-20T10:59:15.920 INFO:tasks.workunit.client.0.smithi027.stdout:9/314: chown d4/f4a 495212 1 2024-04-20T10:59:15.923 INFO:tasks.workunit.client.0.smithi027.stdout:9/315: unlink d4/d1b/d1f/l33 0 2024-04-20T10:59:15.924 INFO:tasks.workunit.client.0.smithi027.stdout:9/316: readlink d4/d1b/d31/d23/d29/d4b/l5e 0 2024-04-20T10:59:15.924 INFO:tasks.workunit.client.0.smithi027.stdout:9/317: fsync d4/d1b/d31/d23/f36 0 2024-04-20T10:59:15.925 INFO:tasks.workunit.client.0.smithi027.stdout:9/318: unlink d4/f4a 0 2024-04-20T10:59:15.928 INFO:tasks.workunit.client.0.smithi027.stdout:9/319: rename d4/d2a/f54 to d4/d1b/d50/d5d/f63 0 2024-04-20T10:59:15.930 INFO:tasks.workunit.client.0.smithi027.stdout:1/433: dwrite d3/d7/d2d/d73/f7d [0,4194304] 0 2024-04-20T10:59:15.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/434: mkdir d3/d7/da3 0 2024-04-20T10:59:15.935 INFO:tasks.workunit.client.0.smithi027.stdout:1/435: stat d3/d7/d28/d35 0 2024-04-20T10:59:15.935 INFO:tasks.workunit.client.0.smithi027.stdout:1/436: getdents d3/d45/d5d 0 2024-04-20T10:59:15.936 INFO:tasks.workunit.client.0.smithi027.stdout:1/437: truncate d3/d7/ff 943961 0 2024-04-20T10:59:15.936 INFO:tasks.workunit.client.0.smithi027.stdout:9/320: dread d4/d1b/d31/d23/f36 [0,4194304] 0 2024-04-20T10:59:15.936 INFO:tasks.workunit.client.0.smithi027.stdout:1/438: mkdir d3/d7/d38/da4 0 2024-04-20T10:59:15.936 INFO:tasks.workunit.client.0.smithi027.stdout:1/439: truncate d3/d7/f7a 167790 0 2024-04-20T10:59:15.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/321: rename d4/d1b/d31/d23/d29/d4b/c5f to d4/d1b/d1f/c64 0 2024-04-20T10:59:15.938 INFO:tasks.workunit.client.0.smithi027.stdout:9/322: chown d4/d1b 220176873 1 2024-04-20T10:59:15.939 INFO:tasks.workunit.client.0.smithi027.stdout:1/440: rename d3/d20/c6a to d3/d41/ca5 0 2024-04-20T10:59:15.939 INFO:tasks.workunit.client.0.smithi027.stdout:9/323: symlink d4/d1b/d1f/l65 0 2024-04-20T10:59:15.939 INFO:tasks.workunit.client.0.smithi027.stdout:9/324: fsync d4/d1b/f24 0 2024-04-20T10:59:15.940 INFO:tasks.workunit.client.0.smithi027.stdout:1/441: symlink d3/d7/la6 0 2024-04-20T10:59:15.940 INFO:tasks.workunit.client.0.smithi027.stdout:1/442: chown d3/d7/fa 236168 1 2024-04-20T10:59:15.940 INFO:tasks.workunit.client.0.smithi027.stdout:9/325: creat d4/d2a/f66 x:0 0 0 2024-04-20T10:59:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:1/443: rename d3/d7/d28/f5c to d3/d7/d38/fa7 0 2024-04-20T10:59:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:1/444: dread - d3/d7e/f9f zero size 2024-04-20T10:59:15.942 INFO:tasks.workunit.client.0.smithi027.stdout:9/326: creat d4/d1b/d31/f67 x:0 0 0 2024-04-20T10:59:15.944 INFO:tasks.workunit.client.0.smithi027.stdout:7/335: dwrite d4/d1b/f35 [0,4194304] 0 2024-04-20T10:59:15.944 INFO:tasks.workunit.client.0.smithi027.stdout:7/336: mknod d4/d5/d8/c61 0 2024-04-20T10:59:15.944 INFO:tasks.workunit.client.0.smithi027.stdout:7/337: stat d4/d5 0 2024-04-20T10:59:15.945 INFO:tasks.workunit.client.0.smithi027.stdout:7/338: stat d4/d1b/f53 0 2024-04-20T10:59:15.946 INFO:tasks.workunit.client.0.smithi027.stdout:7/339: unlink d4/d1b/f28 0 2024-04-20T10:59:15.956 INFO:tasks.workunit.client.0.smithi027.stdout:8/447: dwrite d9/de/d64/d6c/d87/d3c/f61 [0,4194304] 0 2024-04-20T10:59:15.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/330: dwrite da/d21/f70 [0,4194304] 0 2024-04-20T10:59:15.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/331: fsync da/dd/f25 0 2024-04-20T10:59:15.956 INFO:tasks.workunit.client.0.smithi027.stdout:3/414: dwrite d2/f14 [0,4194304] 0 2024-04-20T10:59:15.956 INFO:tasks.workunit.client.0.smithi027.stdout:3/415: truncate d2/d18/d2e/f75 968869 0 2024-04-20T10:59:15.957 INFO:tasks.workunit.client.0.smithi027.stdout:3/416: write d2/d18/d2e/d41/f5a [689361,42185] 0 2024-04-20T10:59:15.958 INFO:tasks.workunit.client.0.smithi027.stdout:8/448: getdents d9/de/d64 0 2024-04-20T10:59:15.958 INFO:tasks.workunit.client.0.smithi027.stdout:0/348: dwrite d0/d11/f32 [0,4194304] 0 2024-04-20T10:59:15.958 INFO:tasks.workunit.client.0.smithi027.stdout:7/340: dread d4/d5/f14 [0,4194304] 0 2024-04-20T10:59:15.958 INFO:tasks.workunit.client.0.smithi027.stdout:4/354: dwrite d0/f27 [0,4194304] 0 2024-04-20T10:59:15.958 INFO:tasks.workunit.client.0.smithi027.stdout:0/349: readlink d0/d7/l10 0 2024-04-20T10:59:15.958 INFO:tasks.workunit.client.0.smithi027.stdout:0/350: read d0/d7/f26 [2048453,29775] 0 2024-04-20T10:59:15.958 INFO:tasks.workunit.client.0.smithi027.stdout:0/351: truncate d0/d11/f54 4648308 0 2024-04-20T10:59:15.961 INFO:tasks.workunit.client.0.smithi027.stdout:3/417: rename d2/dd/f4b to d2/d18/f7b 0 2024-04-20T10:59:15.962 INFO:tasks.workunit.client.0.smithi027.stdout:5/344: dread d19/d1b/d1c/d1e/d22/f32 [0,4194304] 0 2024-04-20T10:59:15.963 INFO:tasks.workunit.client.0.smithi027.stdout:0/352: mknod d0/d11/d2c/c6d 0 2024-04-20T10:59:15.963 INFO:tasks.workunit.client.0.smithi027.stdout:0/353: chown d0/d7/d12/d25/c49 16318587 1 2024-04-20T10:59:15.964 INFO:tasks.workunit.client.0.smithi027.stdout:2/285: dwrite d5/dc/fe [0,4194304] 0 2024-04-20T10:59:15.965 INFO:tasks.workunit.client.0.smithi027.stdout:5/345: mknod d19/d1b/d1c/d1e/d25/d40/c71 0 2024-04-20T10:59:15.965 INFO:tasks.workunit.client.0.smithi027.stdout:5/346: chown d19/d1b/d1c/d1e/d3c/c28 2 1 2024-04-20T10:59:15.967 INFO:tasks.workunit.client.0.smithi027.stdout:1/445: dwrite d3/d7/d28/d35/f5e [4194304,4194304] 0 2024-04-20T10:59:15.967 INFO:tasks.workunit.client.0.smithi027.stdout:0/354: creat d0/d11/d2c/d2e/d2b/f6e x:0 0 0 2024-04-20T10:59:15.969 INFO:tasks.workunit.client.0.smithi027.stdout:2/286: mkdir d5/dc/d16/d60 0 2024-04-20T10:59:15.969 INFO:tasks.workunit.client.0.smithi027.stdout:4/355: dread d0/d14/d17/d1b/f21 [0,4194304] 0 2024-04-20T10:59:15.971 INFO:tasks.workunit.client.0.smithi027.stdout:5/347: rmdir d19/d1b/d1c/d1e/d3c/d64 0 2024-04-20T10:59:15.971 INFO:tasks.workunit.client.0.smithi027.stdout:5/348: readlink d19/l4a 0 2024-04-20T10:59:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:9/327: dwrite d4/d1b/d31/f3c [0,4194304] 0 2024-04-20T10:59:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:9/328: write d4/d1b/d31/d23/d29/f4c [909802,33843] 0 2024-04-20T10:59:15.974 INFO:tasks.workunit.client.0.smithi027.stdout:9/329: fdatasync d4/d14/f44 0 2024-04-20T10:59:15.975 INFO:tasks.workunit.client.0.smithi027.stdout:1/446: symlink d3/d45/d5d/la8 0 2024-04-20T10:59:15.977 INFO:tasks.workunit.client.0.smithi027.stdout:0/355: unlink d0/d11/d2c/d2e/d35/c38 0 2024-04-20T10:59:15.977 INFO:tasks.workunit.client.0.smithi027.stdout:0/356: fdatasync d0/d7/d12/f44 0 2024-04-20T10:59:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/287: rename d5/d26/d30/l39 to d5/dc/d16/d32/l61 0 2024-04-20T10:59:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/288: chown d5/dc/d16/d2a/f40 44 1 2024-04-20T10:59:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/289: getdents d5/dc/d16/d2a/d4d 0 2024-04-20T10:59:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/290: fdatasync d5/d1e/f56 0 2024-04-20T10:59:15.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/291: dread - d5/d26/d30/f3c zero size 2024-04-20T10:59:15.981 INFO:tasks.workunit.client.0.smithi027.stdout:8/449: dwrite d9/de/d64/d6a/f67 [0,4194304] 0 2024-04-20T10:59:15.988 INFO:tasks.workunit.client.0.smithi027.stdout:6/332: dwrite da/df/d19/d2e/d32/f55 [0,4194304] 0 2024-04-20T10:59:15.988 INFO:tasks.workunit.client.0.smithi027.stdout:6/333: readlink da/df/d19/d2e/l57 0 2024-04-20T10:59:15.990 INFO:tasks.workunit.client.0.smithi027.stdout:5/349: dwrite d19/d1b/d1c/d1e/d58/f5e [0,4194304] 0 2024-04-20T10:59:15.991 INFO:tasks.workunit.client.0.smithi027.stdout:8/450: dread d9/f34 [0,4194304] 0 2024-04-20T10:59:15.993 INFO:tasks.workunit.client.0.smithi027.stdout:9/330: mkdir d4/d1b/d1f/d45/d68 0 2024-04-20T10:59:15.994 INFO:tasks.workunit.client.0.smithi027.stdout:1/447: creat d3/d7/d38/fa9 x:0 0 0 2024-04-20T10:59:15.995 INFO:tasks.workunit.client.0.smithi027.stdout:0/357: mkdir d0/d11/d4b/d5b/d6f 0 2024-04-20T10:59:15.998 INFO:tasks.workunit.client.0.smithi027.stdout:8/451: creat d9/de/d20/d4f/d97/fa1 x:0 0 0 2024-04-20T10:59:15.999 INFO:tasks.workunit.client.0.smithi027.stdout:9/331: creat d4/d2a/f69 x:0 0 0 2024-04-20T10:59:16.001 INFO:tasks.workunit.client.0.smithi027.stdout:0/358: creat d0/d11/d2c/d2e/d61/f70 x:0 0 0 2024-04-20T10:59:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:8/452: symlink d9/de/d64/d6c/la2 0 2024-04-20T10:59:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:8/453: fsync d9/de/f37 0 2024-04-20T10:59:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:8/454: chown d9/de/d1e/d4a 7 1 2024-04-20T10:59:16.002 INFO:tasks.workunit.client.0.smithi027.stdout:8/455: chown d9/de/d20/d48/d59/f60 3 1 2024-04-20T10:59:16.003 INFO:tasks.workunit.client.0.smithi027.stdout:6/334: dread da/d21/d28/f3a [0,4194304] 0 2024-04-20T10:59:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:9/332: creat d4/d1b/d1f/f6a x:0 0 0 2024-04-20T10:59:16.004 INFO:tasks.workunit.client.0.smithi027.stdout:9/333: write d4/f1e [3929907,83196] 0 2024-04-20T10:59:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:0/359: mkdir d0/d7/d34/d5a/d71 0 2024-04-20T10:59:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:0/360: chown d0/d11/d2c/d2e/d2b/d3d 258462 1 2024-04-20T10:59:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:0/361: write d0/d11/d2c/f52 [96823,72726] 0 2024-04-20T10:59:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:0/362: truncate d0/d11/f65 913335 0 2024-04-20T10:59:16.005 INFO:tasks.workunit.client.0.smithi027.stdout:0/363: truncate d0/d11/d2c/d2e/d2b/f6e 919161 0 2024-04-20T10:59:16.006 INFO:tasks.workunit.client.0.smithi027.stdout:8/456: rename d9/de/d64/d6c/d87 to d9/de/da3 0 2024-04-20T10:59:16.006 INFO:tasks.workunit.client.0.smithi027.stdout:6/335: readlink da/l20 0 2024-04-20T10:59:16.007 INFO:tasks.workunit.client.0.smithi027.stdout:9/334: rmdir d4/d1b/d1f/d5a 39 2024-04-20T10:59:16.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/364: symlink d0/d11/d62/l72 0 2024-04-20T10:59:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:2/292: dwrite d5/dc/d16/d37/f3f [0,4194304] 0 2024-04-20T10:59:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/336: symlink da/df/l72 0 2024-04-20T10:59:16.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/337: chown da/d21/d28/d2d/d51/c52 174256315 1 2024-04-20T10:59:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/338: stat da/df/d19/l27 0 2024-04-20T10:59:16.012 INFO:tasks.workunit.client.0.smithi027.stdout:9/335: creat d4/f6b x:0 0 0 2024-04-20T10:59:16.013 INFO:tasks.workunit.client.0.smithi027.stdout:0/365: unlink d0/d7/d12/d1a/l40 0 2024-04-20T10:59:16.015 INFO:tasks.workunit.client.0.smithi027.stdout:2/293: unlink d5/c7 0 2024-04-20T10:59:16.015 INFO:tasks.workunit.client.0.smithi027.stdout:8/457: write d9/de/d20/d48/d59/f60 [1884016,78819] 0 2024-04-20T10:59:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:6/339: rename da/dd/d3c/d58/c60 to da/dd/c73 0 2024-04-20T10:59:16.017 INFO:tasks.workunit.client.0.smithi027.stdout:9/336: unlink d4/d1b/f24 0 2024-04-20T10:59:16.018 INFO:tasks.workunit.client.0.smithi027.stdout:2/294: truncate d5/dc/f41 334270 0 2024-04-20T10:59:16.019 INFO:tasks.workunit.client.0.smithi027.stdout:2/295: stat d5/dc/l12 0 2024-04-20T10:59:16.019 INFO:tasks.workunit.client.0.smithi027.stdout:2/296: fdatasync d5/dc/d16/d32/f5a 0 2024-04-20T10:59:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:5/350: dwrite d19/d1b/d1c/d1e/d22/d47/f55 [0,4194304] 0 2024-04-20T10:59:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:1/448: dwrite d3/d7/d28/d35/f52 [0,4194304] 0 2024-04-20T10:59:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:5/351: write fb [2186630,117225] 0 2024-04-20T10:59:16.020 INFO:tasks.workunit.client.0.smithi027.stdout:5/352: readlink l12 0 2024-04-20T10:59:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:8/458: link d9/de/d1e/d4a/f8f d9/de/d20/d4f/d97/fa4 0 2024-04-20T10:59:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:8/459: fdatasync d9/de/f5e 0 2024-04-20T10:59:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:8/460: chown d9/de/f5e 0 1 2024-04-20T10:59:16.021 INFO:tasks.workunit.client.0.smithi027.stdout:8/461: truncate d9/de/d20/d48/d59/d74/f80 398760 0 2024-04-20T10:59:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:6/340: creat da/df/d19/f74 x:0 0 0 2024-04-20T10:59:16.022 INFO:tasks.workunit.client.0.smithi027.stdout:6/341: fdatasync da/dd/d3d/f64 0 2024-04-20T10:59:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:2/297: symlink d5/dc/d16/d2a/d4d/l62 0 2024-04-20T10:59:16.024 INFO:tasks.workunit.client.0.smithi027.stdout:2/298: fsync d5/d26/d47/f5f 0 2024-04-20T10:59:16.025 INFO:tasks.workunit.client.0.smithi027.stdout:0/366: dread d0/ff [0,4194304] 0 2024-04-20T10:59:16.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/367: write d0/d11/d2c/d2e/f4d [156858,13909] 0 2024-04-20T10:59:16.028 INFO:tasks.workunit.client.0.smithi027.stdout:1/449: mknod d3/d7/d28/caa 0 2024-04-20T10:59:16.029 INFO:tasks.workunit.client.0.smithi027.stdout:9/337: mknod d4/d2a/c6c 0 2024-04-20T10:59:16.030 INFO:tasks.workunit.client.0.smithi027.stdout:2/299: getdents d5/dc/d16/d32 0 2024-04-20T10:59:16.031 INFO:tasks.workunit.client.0.smithi027.stdout:0/368: rmdir d0/d11/d4b/d5b 39 2024-04-20T10:59:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:0/369: dread - d0/d59/f6a zero size 2024-04-20T10:59:16.032 INFO:tasks.workunit.client.0.smithi027.stdout:0/370: read - d0/d7/d12/d1a/f6c zero size 2024-04-20T10:59:16.034 INFO:tasks.workunit.client.0.smithi027.stdout:1/450: symlink d3/d7/d38/d9c/lab 0 2024-04-20T10:59:16.034 INFO:tasks.workunit.client.0.smithi027.stdout:1/451: write d3/d7/d2d/d73/f98 [951386,123091] 0 2024-04-20T10:59:16.034 INFO:tasks.workunit.client.0.smithi027.stdout:1/452: chown d3/d7/f1f 66332 1 2024-04-20T10:59:16.034 INFO:tasks.workunit.client.0.smithi027.stdout:2/300: symlink d5/dc/d16/d2a/d4a/l63 0 2024-04-20T10:59:16.034 INFO:tasks.workunit.client.0.smithi027.stdout:6/342: write da/d21/d28/d2d/d53/f6f [1372,116476] 0 2024-04-20T10:59:16.034 INFO:tasks.workunit.client.0.smithi027.stdout:2/301: dread - d5/d1e/d21/f33 zero size 2024-04-20T10:59:16.036 INFO:tasks.workunit.client.0.smithi027.stdout:6/343: dread da/df/f4d [0,4194304] 0 2024-04-20T10:59:16.038 INFO:tasks.workunit.client.0.smithi027.stdout:0/371: rename d0/d11/d2c/d2e/f4d to d0/d59/f73 0 2024-04-20T10:59:16.039 INFO:tasks.workunit.client.0.smithi027.stdout:2/302: link d5/d26/c38 d5/d26/d47/c64 0 2024-04-20T10:59:16.040 INFO:tasks.workunit.client.0.smithi027.stdout:1/453: dread d3/d7/d28/f3d [0,4194304] 0 2024-04-20T10:59:16.050 INFO:tasks.workunit.client.0.smithi027.stdout:8/462: dwrite d9/f24 [0,4194304] 0 2024-04-20T10:59:16.050 INFO:tasks.workunit.client.0.smithi027.stdout:9/338: dwrite d4/d1b/d31/d23/f47 [0,4194304] 0 2024-04-20T10:59:16.271 INFO:tasks.workunit.client.0.smithi027.stdout:4/356: write d0/d14/d17/d1b/d2f/f67 [2062105,33245] 0 2024-04-20T10:59:16.271 INFO:tasks.workunit.client.0.smithi027.stdout:4/357: fdatasync d0/d14/d71/f8b 0 2024-04-20T10:59:16.297 INFO:tasks.workunit.client.0.smithi027.stdout:2/303: truncate d5/d1e/f5c 1661975 0 2024-04-20T10:59:16.298 INFO:tasks.workunit.client.0.smithi027.stdout:6/344: rename da/df/f49 to da/df/d19/d2e/d32/f75 0 2024-04-20T10:59:16.298 INFO:tasks.workunit.client.0.smithi027.stdout:2/304: write d5/dc/f1c [4312325,46929] 0 2024-04-20T10:59:16.299 INFO:tasks.workunit.client.0.smithi027.stdout:1/454: mkdir d3/d7/d38/da4/dac 0 2024-04-20T10:59:16.299 INFO:tasks.workunit.client.0.smithi027.stdout:1/455: truncate d3/d7/d28/f70 177822 0 2024-04-20T10:59:16.300 INFO:tasks.workunit.client.0.smithi027.stdout:1/456: dread d3/d7/d38/f56 [0,4194304] 0 2024-04-20T10:59:16.302 INFO:tasks.workunit.client.0.smithi027.stdout:8/463: mkdir d9/de/d20/d4f/d97/da5 0 2024-04-20T10:59:16.302 INFO:tasks.workunit.client.0.smithi027.stdout:8/464: read - d9/de/d1e/d4a/d6d/d79/f92 zero size 2024-04-20T10:59:16.303 INFO:tasks.workunit.client.0.smithi027.stdout:9/339: mknod d4/d1b/c6d 0 2024-04-20T10:59:16.303 INFO:tasks.workunit.client.0.smithi027.stdout:9/340: write d4/d2a/f69 [778274,88207] 0 2024-04-20T10:59:16.303 INFO:tasks.workunit.client.0.smithi027.stdout:9/341: dread - d4/d1b/d50/d5d/f63 zero size 2024-04-20T10:59:16.307 INFO:tasks.workunit.client.0.smithi027.stdout:4/358: link d0/d14/d17/l25 d0/d14/d17/d1b/d33/d5c/d6f/l8c 0 2024-04-20T10:59:16.307 INFO:tasks.workunit.client.0.smithi027.stdout:4/359: write d0/d14/d17/d1b/f30 [1382570,73436] 0 2024-04-20T10:59:16.308 INFO:tasks.workunit.client.0.smithi027.stdout:0/372: rename d0/ff to d0/d11/d5c/f74 0 2024-04-20T10:59:16.309 INFO:tasks.workunit.client.0.smithi027.stdout:7/341: sync 2024-04-20T10:59:16.309 INFO:tasks.workunit.client.0.smithi027.stdout:3/418: sync 2024-04-20T10:59:16.309 INFO:tasks.workunit.client.0.smithi027.stdout:7/342: read - d4/d1b/d58/f5f zero size 2024-04-20T10:59:16.310 INFO:tasks.workunit.client.0.smithi027.stdout:6/345: dread da/dd/f59 [0,4194304] 0 2024-04-20T10:59:16.310 INFO:tasks.workunit.client.0.smithi027.stdout:2/305: truncate d5/d26/d3d/f48 2358586 0 2024-04-20T10:59:16.310 INFO:tasks.workunit.client.0.smithi027.stdout:2/306: chown d5/dc/d16/c18 2 1 2024-04-20T10:59:16.312 INFO:tasks.workunit.client.0.smithi027.stdout:5/353: sync 2024-04-20T10:59:16.313 INFO:tasks.workunit.client.0.smithi027.stdout:1/457: mkdir d3/d7/d28/d92/dad 0 2024-04-20T10:59:16.315 INFO:tasks.workunit.client.0.smithi027.stdout:9/342: creat d4/d1b/d31/d23/d29/d4b/f6e x:0 0 0 2024-04-20T10:59:16.322 INFO:tasks.workunit.client.0.smithi027.stdout:4/360: symlink d0/d14/d17/d1b/d33/d5c/d6f/d7b/l8d 0 2024-04-20T10:59:16.322 INFO:tasks.workunit.client.0.smithi027.stdout:0/373: dread d0/d7/d12/d25/f31 [0,4194304] 0 2024-04-20T10:59:16.322 INFO:tasks.workunit.client.0.smithi027.stdout:5/354: write fc [1867219,74064] 0 2024-04-20T10:59:16.323 INFO:tasks.workunit.client.0.smithi027.stdout:8/465: rename d9/de/d18/c23 to d9/de/d1e/d4a/d6d/d75/d7c/ca6 0 2024-04-20T10:59:16.324 INFO:tasks.workunit.client.0.smithi027.stdout:4/361: link d0/d14/d17/d1b/c69 d0/d14/d6b/c8e 0 2024-04-20T10:59:16.325 INFO:tasks.workunit.client.0.smithi027.stdout:0/374: creat d0/d7/d34/d5a/f75 x:0 0 0 2024-04-20T10:59:16.327 INFO:tasks.workunit.client.0.smithi027.stdout:4/362: symlink d0/d14/d17/d53/d6a/l8f 0 2024-04-20T10:59:16.328 INFO:tasks.workunit.client.0.smithi027.stdout:0/375: symlink d0/d7/d34/l76 0 2024-04-20T10:59:16.328 INFO:tasks.workunit.client.0.smithi027.stdout:0/376: chown d0/d11/d2c/d2e/l45 57577031 1 2024-04-20T10:59:16.328 INFO:tasks.workunit.client.0.smithi027.stdout:0/377: readlink d0/d7/d12/d1a/d47/l5f 0 2024-04-20T10:59:16.330 INFO:tasks.workunit.client.0.smithi027.stdout:4/363: mknod d0/d14/d71/c90 0 2024-04-20T10:59:16.331 INFO:tasks.workunit.client.0.smithi027.stdout:9/343: read d4/d1b/d31/d23/f36 [2813710,34007] 0 2024-04-20T10:59:16.331 INFO:tasks.workunit.client.0.smithi027.stdout:2/307: dread d5/dc/fe [0,4194304] 0 2024-04-20T10:59:16.332 INFO:tasks.workunit.client.0.smithi027.stdout:2/308: readlink d5/dc/d16/l2f 0 2024-04-20T10:59:16.334 INFO:tasks.workunit.client.0.smithi027.stdout:4/364: creat d0/d48/f91 x:0 0 0 2024-04-20T10:59:16.334 INFO:tasks.workunit.client.0.smithi027.stdout:4/365: dread - d0/d48/f91 zero size 2024-04-20T10:59:16.334 INFO:tasks.workunit.client.0.smithi027.stdout:4/366: chown d0/d14/d71/f8b 29279 1 2024-04-20T10:59:16.334 INFO:tasks.workunit.client.0.smithi027.stdout:2/309: dread d5/d26/d47/f5f [0,4194304] 0 2024-04-20T10:59:16.339 INFO:tasks.workunit.client.0.smithi027.stdout:8/466: write d9/de/d64/d6c/f8c [2153441,19676] 0 2024-04-20T10:59:16.340 INFO:tasks.workunit.client.0.smithi027.stdout:0/378: rename d0/d7 to d0/d11/d2c/d2e/d61/d77 0 2024-04-20T10:59:16.341 INFO:tasks.workunit.client.0.smithi027.stdout:2/310: creat d5/d44/f65 x:0 0 0 2024-04-20T10:59:16.342 INFO:tasks.workunit.client.0.smithi027.stdout:8/467: mknod d9/ca7 0 2024-04-20T10:59:16.342 INFO:tasks.workunit.client.0.smithi027.stdout:8/468: stat d9/de/da3/d3c/d4b/c72 0 2024-04-20T10:59:16.342 INFO:tasks.workunit.client.0.smithi027.stdout:3/419: dwrite d2/d18/d2c/f69 [0,4194304] 0 2024-04-20T10:59:16.343 INFO:tasks.workunit.client.0.smithi027.stdout:0/379: creat d0/d11/d2c/d2e/d61/f78 x:0 0 0 2024-04-20T10:59:16.344 INFO:tasks.workunit.client.0.smithi027.stdout:8/469: creat d9/de/d20/d4f/d5b/fa8 x:0 0 0 2024-04-20T10:59:16.344 INFO:tasks.workunit.client.0.smithi027.stdout:7/343: dwrite d4/d5/f40 [0,4194304] 0 2024-04-20T10:59:16.348 INFO:tasks.workunit.client.0.smithi027.stdout:5/355: dwrite d19/d1b/d1c/d1e/d3c/f46 [8388608,4194304] 0 2024-04-20T10:59:16.354 INFO:tasks.workunit.client.0.smithi027.stdout:8/470: dread d9/de/f1a [0,4194304] 0 2024-04-20T10:59:16.355 INFO:tasks.workunit.client.0.smithi027.stdout:5/356: getdents d19/d1b/d1c/d1e/d25 0 2024-04-20T10:59:16.355 INFO:tasks.workunit.client.0.smithi027.stdout:8/471: symlink d9/de/d64/d6c/la9 0 2024-04-20T10:59:16.357 INFO:tasks.workunit.client.0.smithi027.stdout:5/357: mknod d19/d1b/d1c/d1e/d3c/c72 0 2024-04-20T10:59:16.360 INFO:tasks.workunit.client.0.smithi027.stdout:8/472: symlink d9/de/d1e/d4a/d6d/laa 0 2024-04-20T10:59:16.361 INFO:tasks.workunit.client.0.smithi027.stdout:1/458: dwrite d3/d7/d28/f48 [0,4194304] 0 2024-04-20T10:59:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:5/358: mknod d19/d1b/d1c/d1e/d22/c73 0 2024-04-20T10:59:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:5/359: write d19/d1b/d1c/d1e/d25/f2f [986941,87502] 0 2024-04-20T10:59:16.362 INFO:tasks.workunit.client.0.smithi027.stdout:7/344: dread d4/d5/d8/f38 [0,4194304] 0 2024-04-20T10:59:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:7/345: write d4/d5/d8/d21/f31 [1512357,44230] 0 2024-04-20T10:59:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:2/311: dread d5/d1e/f5c [0,4194304] 0 2024-04-20T10:59:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:4/367: dwrite d0/d14/d17/d18/f4b [0,4194304] 0 2024-04-20T10:59:16.364 INFO:tasks.workunit.client.0.smithi027.stdout:6/346: dwrite da/df/d19/d2e/d6b/f62 [0,4194304] 0 2024-04-20T10:59:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:9/344: dwrite d4/f1c [0,4194304] 0 2024-04-20T10:59:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:4/368: chown d0/d14/c5d 1685 1 2024-04-20T10:59:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:1/459: rmdir d3/d7/da3 0 2024-04-20T10:59:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:1/460: chown d3/d7 408 1 2024-04-20T10:59:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:8/473: read d9/de/d1e/f68 [75711,65402] 0 2024-04-20T10:59:16.365 INFO:tasks.workunit.client.0.smithi027.stdout:0/380: dwrite d0/d11/d2c/d2e/d61/d77/d12/f3e [0,4194304] 0 2024-04-20T10:59:16.366 INFO:tasks.workunit.client.0.smithi027.stdout:2/312: getdents d5/d1e 0 2024-04-20T10:59:16.366 INFO:tasks.workunit.client.0.smithi027.stdout:2/313: fdatasync d5/dc/d16/d2a/f40 0 2024-04-20T10:59:16.366 INFO:tasks.workunit.client.0.smithi027.stdout:2/314: write d5/dc/f55 [384759,100305] 0 2024-04-20T10:59:16.369 INFO:tasks.workunit.client.0.smithi027.stdout:6/347: creat da/d21/d28/d2d/d53/f76 x:0 0 0 2024-04-20T10:59:16.370 INFO:tasks.workunit.client.0.smithi027.stdout:5/360: write d19/d1b/f43 [2451880,7410] 0 2024-04-20T10:59:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:9/345: unlink d4/d1b/d1f/f6a 0 2024-04-20T10:59:16.371 INFO:tasks.workunit.client.0.smithi027.stdout:9/346: write d4/f6b [776811,80521] 0 2024-04-20T10:59:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:4/369: mkdir d0/d14/d6b/d92 0 2024-04-20T10:59:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:4/370: dread - d0/d14/d17/d1b/f7f zero size 2024-04-20T10:59:16.372 INFO:tasks.workunit.client.0.smithi027.stdout:4/371: dread - d0/d14/d17/d18/f54 zero size 2024-04-20T10:59:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:0/381: write d0/d11/d5c/f74 [1984741,123431] 0 2024-04-20T10:59:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:0/382: fdatasync d0/d11/d2c/d2e/d55/f56 0 2024-04-20T10:59:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/461: symlink d3/d7/lae 0 2024-04-20T10:59:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/462: fdatasync d3/d7/d2d/f4a 0 2024-04-20T10:59:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/346: dread d4/d1b/f35 [0,4194304] 0 2024-04-20T10:59:16.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/347: write d4/d5/d8/f38 [961446,68230] 0 2024-04-20T10:59:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:7/348: truncate d4/d1b/d58/f5f 8661 0 2024-04-20T10:59:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:7/349: getdents d4/d1b/d4a 0 2024-04-20T10:59:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/420: dread d2/f9 [4194304,4194304] 0 2024-04-20T10:59:16.374 INFO:tasks.workunit.client.0.smithi027.stdout:8/474: mkdir d9/de/da3/dab 0 2024-04-20T10:59:16.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/347: dread d4/d1b/f22 [0,4194304] 0 2024-04-20T10:59:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:3/421: read d2/d18/d2c/f4f [629259,103900] 0 2024-04-20T10:59:16.376 INFO:tasks.workunit.client.0.smithi027.stdout:3/422: chown d2/d18/f3f 101 1 2024-04-20T10:59:16.377 INFO:tasks.workunit.client.0.smithi027.stdout:6/348: mknod da/df/d19/d2e/d6b/d65/c77 0 2024-04-20T10:59:16.378 INFO:tasks.workunit.client.0.smithi027.stdout:4/372: symlink d0/d14/d17/d53/d79/l93 0 2024-04-20T10:59:16.383 INFO:tasks.workunit.client.0.smithi027.stdout:0/383: symlink d0/d11/d2c/d2e/d61/d77/d12/d1a/d47/d5d/l79 0 2024-04-20T10:59:16.383 INFO:tasks.workunit.client.0.smithi027.stdout:1/463: rmdir d3/d7/d2d/d9b 0 2024-04-20T10:59:16.383 INFO:tasks.workunit.client.0.smithi027.stdout:1/464: dread - d3/d7/d2d/f84 zero size 2024-04-20T10:59:16.384 INFO:tasks.workunit.client.0.smithi027.stdout:4/373: dread d0/d14/d17/d1b/d2f/f67 [0,4194304] 0 2024-04-20T10:59:16.385 INFO:tasks.workunit.client.0.smithi027.stdout:7/350: rename d4/d52 to d4/d1b/d4a/d62 0 2024-04-20T10:59:16.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/423: symlink d2/dd/d22/l7c 0 2024-04-20T10:59:16.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/424: readlink d2/d18/d2e/l37 0 2024-04-20T10:59:16.388 INFO:tasks.workunit.client.0.smithi027.stdout:6/349: getdents da/df 0 2024-04-20T10:59:16.389 INFO:tasks.workunit.client.0.smithi027.stdout:6/350: stat da/d21/d28/d2d 0 2024-04-20T10:59:16.389 INFO:tasks.workunit.client.0.smithi027.stdout:0/384: mknod d0/d11/d4b/c7a 0 2024-04-20T10:59:16.390 INFO:tasks.workunit.client.0.smithi027.stdout:5/361: dread f3 [0,4194304] 0 2024-04-20T10:59:16.391 INFO:tasks.workunit.client.0.smithi027.stdout:9/348: write d4/d1b/f22 [815696,88618] 0 2024-04-20T10:59:16.391 INFO:tasks.workunit.client.0.smithi027.stdout:9/349: chown d4/d1b/d31/f3c 27679 1 2024-04-20T10:59:16.392 INFO:tasks.workunit.client.0.smithi027.stdout:4/374: symlink d0/d48/d68/l94 0 2024-04-20T10:59:16.395 INFO:tasks.workunit.client.0.smithi027.stdout:5/362: dread d19/d1b/d1c/d1e/d22/f30 [0,4194304] 0 2024-04-20T10:59:16.396 INFO:tasks.workunit.client.0.smithi027.stdout:0/385: dread d0/d11/d2c/d2e/d61/d77/f1c [4194304,4194304] 0 2024-04-20T10:59:16.398 INFO:tasks.workunit.client.0.smithi027.stdout:9/350: symlink d4/d1b/d1f/d5a/l6f 0 2024-04-20T10:59:16.403 INFO:tasks.workunit.client.0.smithi027.stdout:7/351: dwrite d4/d5/f40 [0,4194304] 0 2024-04-20T10:59:16.403 INFO:tasks.workunit.client.0.smithi027.stdout:7/352: readlink d4/d5/d8/d21/l41 0 2024-04-20T10:59:16.403 INFO:tasks.workunit.client.0.smithi027.stdout:6/351: write da/df/d19/d2e/d32/f55 [3076577,47094] 0 2024-04-20T10:59:16.404 INFO:tasks.workunit.client.0.smithi027.stdout:9/351: creat d4/d1b/d31/d23/d29/f70 x:0 0 0 2024-04-20T10:59:16.405 INFO:tasks.workunit.client.0.smithi027.stdout:3/425: write d2/d18/f53 [2222078,80095] 0 2024-04-20T10:59:16.406 INFO:tasks.workunit.client.0.smithi027.stdout:7/353: creat d4/d1b/f63 x:0 0 0 2024-04-20T10:59:16.406 INFO:tasks.workunit.client.0.smithi027.stdout:7/354: chown d4/d1b/l1c 1183 1 2024-04-20T10:59:16.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/352: truncate da/df/d19/d2e/d32/f44 4040334 0 2024-04-20T10:59:16.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/353: fdatasync da/d21/d28/d2d/d53/f76 0 2024-04-20T10:59:16.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/354: dread - da/d21/d28/d2d/f43 zero size 2024-04-20T10:59:16.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/355: write da/d21/d28/f34 [4716249,23774] 0 2024-04-20T10:59:16.407 INFO:tasks.workunit.client.0.smithi027.stdout:7/355: dread d4/d5/f5c [0,4194304] 0 2024-04-20T10:59:16.407 INFO:tasks.workunit.client.0.smithi027.stdout:9/352: creat d4/d1b/d1f/d45/f71 x:0 0 0 2024-04-20T10:59:16.418 INFO:tasks.workunit.client.0.smithi027.stdout:1/465: dwrite d3/d7/d38/f65 [0,4194304] 0 2024-04-20T10:59:16.418 INFO:tasks.workunit.client.0.smithi027.stdout:1/466: write d3/d7/d28/d35/fa0 [801739,40852] 0 2024-04-20T10:59:16.418 INFO:tasks.workunit.client.0.smithi027.stdout:1/467: dread - d3/d7/d2d/f99 zero size 2024-04-20T10:59:16.418 INFO:tasks.workunit.client.0.smithi027.stdout:3/426: write d2/fe [6018158,119757] 0 2024-04-20T10:59:16.418 INFO:tasks.workunit.client.0.smithi027.stdout:3/427: fsync d2/f1f 0 2024-04-20T10:59:16.419 INFO:tasks.workunit.client.0.smithi027.stdout:8/475: dwrite d9/de/f1b [0,4194304] 0 2024-04-20T10:59:16.421 INFO:tasks.workunit.client.0.smithi027.stdout:1/468: creat d3/d20/faf x:0 0 0 2024-04-20T10:59:16.422 INFO:tasks.workunit.client.0.smithi027.stdout:8/476: symlink d9/de/d20/d95/lac 0 2024-04-20T10:59:16.423 INFO:tasks.workunit.client.0.smithi027.stdout:0/386: rename d0/d11/d2c/d2e/d61/d77 to d0/d11/d2c/d2e/d2b/d7b 0 2024-04-20T10:59:16.423 INFO:tasks.workunit.client.0.smithi027.stdout:3/428: rename d2 to d2/dd/d7d 22 2024-04-20T10:59:16.423 INFO:tasks.workunit.client.0.smithi027.stdout:0/387: write d0/d11/d2c/d2e/d2b/d7b/f27 [198606,112797] 0 2024-04-20T10:59:16.424 INFO:tasks.workunit.client.0.smithi027.stdout:4/375: dwrite d0/d14/d17/d1b/d2f/f73 [0,4194304] 0 2024-04-20T10:59:16.426 INFO:tasks.workunit.client.0.smithi027.stdout:1/469: mkdir d3/d7/d28/db0 0 2024-04-20T10:59:16.426 INFO:tasks.workunit.client.0.smithi027.stdout:1/470: write d3/d7/d38/fa9 [210938,87727] 0 2024-04-20T10:59:16.427 INFO:tasks.workunit.client.0.smithi027.stdout:1/471: read d3/d7/d2d/d73/f98 [227443,119994] 0 2024-04-20T10:59:16.427 INFO:tasks.workunit.client.0.smithi027.stdout:5/363: dwrite d19/d1b/d1c/d1e/f57 [0,4194304] 0 2024-04-20T10:59:16.428 INFO:tasks.workunit.client.0.smithi027.stdout:9/353: dwrite d4/d2a/f69 [0,4194304] 0 2024-04-20T10:59:16.429 INFO:tasks.workunit.client.0.smithi027.stdout:8/477: mkdir d9/de/d20/d48/d59/dad 0 2024-04-20T10:59:16.429 INFO:tasks.workunit.client.0.smithi027.stdout:8/478: read f3 [2228591,15764] 0 2024-04-20T10:59:16.430 INFO:tasks.workunit.client.0.smithi027.stdout:3/429: creat d2/d18/d2c/d4c/f7e x:0 0 0 2024-04-20T10:59:16.431 INFO:tasks.workunit.client.0.smithi027.stdout:7/356: dwrite d4/d5/d8/d3b/f42 [0,4194304] 0 2024-04-20T10:59:16.432 INFO:tasks.workunit.client.0.smithi027.stdout:0/388: link d0/d11/d2c/d2e/d2b/d7b/d12/d25/f31 d0/d11/d2c/d2e/d2b/d7b/d64/f7c 0 2024-04-20T10:59:16.432 INFO:tasks.workunit.client.0.smithi027.stdout:1/472: stat d3/c64 0 2024-04-20T10:59:16.435 INFO:tasks.workunit.client.0.smithi027.stdout:5/364: link d19/d1b/d1c/f6a d19/d1b/d1c/d1e/d3c/f74 0 2024-04-20T10:59:16.437 INFO:tasks.workunit.client.0.smithi027.stdout:3/430: dread d2/d18/f26 [0,4194304] 0 2024-04-20T10:59:16.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/354: mkdir d4/d1b/d1f/d45/d68/d72 0 2024-04-20T10:59:16.439 INFO:tasks.workunit.client.0.smithi027.stdout:8/479: symlink d9/de/d20/d4f/d5b/lae 0 2024-04-20T10:59:16.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/376: write d0/d14/d17/d1b/f61 [3903813,36438] 0 2024-04-20T10:59:16.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/357: mknod d4/d5/d8/d45/c64 0 2024-04-20T10:59:16.444 INFO:tasks.workunit.client.0.smithi027.stdout:0/389: link d0/d11/d2c/d2e/d2b/d7b/d12/l29 d0/d11/d2c/l7d 0 2024-04-20T10:59:16.445 INFO:tasks.workunit.client.0.smithi027.stdout:1/473: mknod d3/d7e/cb1 0 2024-04-20T10:59:16.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/480: dread d9/de/d20/d48/d59/f60 [0,4194304] 0 2024-04-20T10:59:16.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/481: stat d9/de/d20/f73 0 2024-04-20T10:59:16.446 INFO:tasks.workunit.client.0.smithi027.stdout:5/365: mkdir d19/d1b/d1c/d1e/d22/d68/d75 0 2024-04-20T10:59:16.448 INFO:tasks.workunit.client.0.smithi027.stdout:3/431: mkdir d2/d7f 0 2024-04-20T10:59:16.448 INFO:tasks.workunit.client.0.smithi027.stdout:3/432: dread - d2/d18/d2c/f78 zero size 2024-04-20T10:59:16.449 INFO:tasks.workunit.client.0.smithi027.stdout:4/377: getdents d0/d14 0 2024-04-20T10:59:16.450 INFO:tasks.workunit.client.0.smithi027.stdout:7/358: creat d4/f65 x:0 0 0 2024-04-20T10:59:16.451 INFO:tasks.workunit.client.0.smithi027.stdout:0/390: creat d0/d11/d2c/d2e/d2b/d3d/f7e x:0 0 0 2024-04-20T10:59:16.452 INFO:tasks.workunit.client.0.smithi027.stdout:1/474: mknod d3/d7/d38/da4/cb2 0 2024-04-20T10:59:16.452 INFO:tasks.workunit.client.0.smithi027.stdout:1/475: read d3/d7/f7a [114795,60791] 0 2024-04-20T10:59:16.454 INFO:tasks.workunit.client.0.smithi027.stdout:8/482: creat d9/de/faf x:0 0 0 2024-04-20T10:59:16.454 INFO:tasks.workunit.client.0.smithi027.stdout:5/366: mknod d19/c76 0 2024-04-20T10:59:16.455 INFO:tasks.workunit.client.0.smithi027.stdout:9/355: dwrite d4/d1b/f22 [0,4194304] 0 2024-04-20T10:59:16.471 INFO:tasks.workunit.client.0.smithi027.stdout:0/391: dwrite d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f2d [0,4194304] 0 2024-04-20T10:59:16.480 INFO:tasks.workunit.client.0.smithi027.stdout:1/476: dwrite d3/d20/f33 [0,4194304] 0 2024-04-20T10:59:16.487 INFO:tasks.workunit.client.0.smithi027.stdout:9/356: dwrite d4/d1b/d31/d23/d29/f4c [0,4194304] 0 2024-04-20T10:59:16.713 INFO:tasks.workunit.client.0.smithi027.stdout:3/433: rename d2/d18/d2e/d41 to d2/d80 0 2024-04-20T10:59:16.713 INFO:tasks.workunit.client.0.smithi027.stdout:4/378: mknod d0/d5e/c95 0 2024-04-20T10:59:16.713 INFO:tasks.workunit.client.0.smithi027.stdout:7/359: getdents d4/d5/d8/d3b 0 2024-04-20T10:59:16.714 INFO:tasks.workunit.client.0.smithi027.stdout:8/483: link d9/de/l3b d9/de/d1e/d4a/d6d/d78/lb0 0 2024-04-20T10:59:16.718 INFO:tasks.workunit.client.0.smithi027.stdout:0/392: mkdir d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f 0 2024-04-20T10:59:16.724 INFO:tasks.workunit.client.0.smithi027.stdout:1/477: link d3/d20/d85/f9d d3/d7/d38/da4/dac/fb3 0 2024-04-20T10:59:16.727 INFO:tasks.workunit.client.0.smithi027.stdout:9/357: creat d4/d1b/d1f/d45/d68/d72/f73 x:0 0 0 2024-04-20T10:59:16.729 INFO:tasks.workunit.client.0.smithi027.stdout:5/367: rename d19/f52 to d19/d1b/d1c/d1e/d22/d68/d75/f77 0 2024-04-20T10:59:16.731 INFO:tasks.workunit.client.0.smithi027.stdout:4/379: symlink d0/d14/d17/d1b/d33/l96 0 2024-04-20T10:59:16.732 INFO:tasks.workunit.client.0.smithi027.stdout:7/360: creat d4/d1b/d4a/d62/f66 x:0 0 0 2024-04-20T10:59:16.735 INFO:tasks.workunit.client.0.smithi027.stdout:0/393: symlink d0/d11/d2c/d2e/d2b/d7b/l80 0 2024-04-20T10:59:16.736 INFO:tasks.workunit.client.0.smithi027.stdout:5/368: dread fc [0,4194304] 0 2024-04-20T10:59:16.736 INFO:tasks.workunit.client.0.smithi027.stdout:5/369: read - d19/d1b/d1c/d1e/d22/f5a zero size 2024-04-20T10:59:16.736 INFO:tasks.workunit.client.0.smithi027.stdout:5/370: write d19/d1b/d1c/d1e/d3c/f31 [5020976,7290] 0 2024-04-20T10:59:16.738 INFO:tasks.workunit.client.0.smithi027.stdout:1/478: rename d3/d7/d28/d35/f52 to d3/d7/d28/d92/fb4 0 2024-04-20T10:59:16.738 INFO:tasks.workunit.client.0.smithi027.stdout:3/434: dwrite d2/d18/d2c/d4c/f7e [0,4194304] 0 2024-04-20T10:59:16.738 INFO:tasks.workunit.client.0.smithi027.stdout:0/394: rmdir d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d 39 2024-04-20T10:59:16.740 INFO:tasks.workunit.client.0.smithi027.stdout:5/371: mkdir d19/d1b/d1c/d1e/d25/d40/d78 0 2024-04-20T10:59:16.740 INFO:tasks.workunit.client.0.smithi027.stdout:5/372: readlink d19/d1b/d1c/l2e 0 2024-04-20T10:59:16.740 INFO:tasks.workunit.client.0.smithi027.stdout:1/479: rmdir d3/d45/d5d 39 2024-04-20T10:59:16.740 INFO:tasks.workunit.client.0.smithi027.stdout:2/315: sync 2024-04-20T10:59:16.743 INFO:tasks.workunit.client.0.smithi027.stdout:3/435: rmdir d2/dd/df/d23 39 2024-04-20T10:59:16.744 INFO:tasks.workunit.client.0.smithi027.stdout:6/356: sync 2024-04-20T10:59:16.744 INFO:tasks.workunit.client.0.smithi027.stdout:0/395: creat d0/d11/d62/f81 x:0 0 0 2024-04-20T10:59:16.745 INFO:tasks.workunit.client.0.smithi027.stdout:5/373: unlink c4 0 2024-04-20T10:59:16.746 INFO:tasks.workunit.client.0.smithi027.stdout:2/316: mkdir d5/dc/d16/d2a/d59/d66 0 2024-04-20T10:59:16.748 INFO:tasks.workunit.client.0.smithi027.stdout:2/317: dread f1 [0,4194304] 0 2024-04-20T10:59:16.753 INFO:tasks.workunit.client.0.smithi027.stdout:2/318: dread d5/f4b [0,4194304] 0 2024-04-20T10:59:16.754 INFO:tasks.workunit.client.0.smithi027.stdout:8/484: dwrite d9/de/d1e/d4a/d6d/d7d/f9f [0,4194304] 0 2024-04-20T10:59:16.754 INFO:tasks.workunit.client.0.smithi027.stdout:8/485: getdents d9/de/d20/d48/d62 0 2024-04-20T10:59:16.754 INFO:tasks.workunit.client.0.smithi027.stdout:8/486: read - d9/de/d20/d48/f85 zero size 2024-04-20T10:59:16.754 INFO:tasks.workunit.client.0.smithi027.stdout:0/396: mkdir d0/d11/d4b/d5b/d6f/d82 0 2024-04-20T10:59:16.754 INFO:tasks.workunit.client.0.smithi027.stdout:0/397: chown d0/d11/d2c/d2e/f1d 15489 1 2024-04-20T10:59:16.755 INFO:tasks.workunit.client.0.smithi027.stdout:5/374: creat d19/d1b/d1c/d1e/f79 x:0 0 0 2024-04-20T10:59:16.756 INFO:tasks.workunit.client.0.smithi027.stdout:5/375: stat d19/d1b/d1c/d1e/d3c/c28 0 2024-04-20T10:59:16.756 INFO:tasks.workunit.client.0.smithi027.stdout:5/376: write d19/d1b/d1c/d1e/d58/f6c [204418,55090] 0 2024-04-20T10:59:16.756 INFO:tasks.workunit.client.0.smithi027.stdout:5/377: chown d19/d1b/d1c/d1e/f79 1464 1 2024-04-20T10:59:16.756 INFO:tasks.workunit.client.0.smithi027.stdout:5/378: chown d19/d1b/d1c/d1e/d3c/c72 474 1 2024-04-20T10:59:16.757 INFO:tasks.workunit.client.0.smithi027.stdout:2/319: rename d5/d1e/f56 to d5/dc/d16/d2a/d4a/f67 0 2024-04-20T10:59:16.757 INFO:tasks.workunit.client.0.smithi027.stdout:2/320: write d5/f4b [705734,24447] 0 2024-04-20T10:59:16.757 INFO:tasks.workunit.client.0.smithi027.stdout:2/321: chown d5/dc/c2e 190710 1 2024-04-20T10:59:16.758 INFO:tasks.workunit.client.0.smithi027.stdout:1/480: write d3/f17 [2019772,81998] 0 2024-04-20T10:59:16.758 INFO:tasks.workunit.client.0.smithi027.stdout:1/481: dread - d3/d7/d2d/f99 zero size 2024-04-20T10:59:16.758 INFO:tasks.workunit.client.0.smithi027.stdout:9/358: dwrite d4/d14/f19 [0,4194304] 0 2024-04-20T10:59:16.759 INFO:tasks.workunit.client.0.smithi027.stdout:8/487: creat d9/de/d20/fb1 x:0 0 0 2024-04-20T10:59:16.759 INFO:tasks.workunit.client.0.smithi027.stdout:3/436: dwrite d2/d18/d2c/f54 [0,4194304] 0 2024-04-20T10:59:16.760 INFO:tasks.workunit.client.0.smithi027.stdout:0/398: symlink d0/d11/d2c/d2e/d2b/d7b/d12/l83 0 2024-04-20T10:59:16.760 INFO:tasks.workunit.client.0.smithi027.stdout:0/399: write d0/d59/f73 [165936,129536] 0 2024-04-20T10:59:16.760 INFO:tasks.workunit.client.0.smithi027.stdout:0/400: write d0/d59/f73 [261231,99656] 0 2024-04-20T10:59:16.761 INFO:tasks.workunit.client.0.smithi027.stdout:5/379: rename d19/c5f to d19/c7a 0 2024-04-20T10:59:16.763 INFO:tasks.workunit.client.0.smithi027.stdout:1/482: creat d3/d7/d28/d92/fb5 x:0 0 0 2024-04-20T10:59:16.769 INFO:tasks.workunit.client.0.smithi027.stdout:6/357: dwrite da/df/d19/d2e/f38 [0,4194304] 0 2024-04-20T10:59:16.770 INFO:tasks.workunit.client.0.smithi027.stdout:8/488: dread d9/de/d20/d48/d59/f60 [0,4194304] 0 2024-04-20T10:59:16.770 INFO:tasks.workunit.client.0.smithi027.stdout:8/489: readlink d9/de/l10 0 2024-04-20T10:59:16.771 INFO:tasks.workunit.client.0.smithi027.stdout:3/437: symlink d2/d18/d2e/l81 0 2024-04-20T10:59:16.775 INFO:tasks.workunit.client.0.smithi027.stdout:8/490: write d9/de/f1a [3279043,20041] 0 2024-04-20T10:59:16.778 INFO:tasks.workunit.client.0.smithi027.stdout:0/401: creat d0/d11/d2c/d2e/d2b/d3d/f84 x:0 0 0 2024-04-20T10:59:16.779 INFO:tasks.workunit.client.0.smithi027.stdout:9/359: dread d4/d14/f44 [0,4194304] 0 2024-04-20T10:59:16.785 INFO:tasks.workunit.client.0.smithi027.stdout:1/483: dwrite d3/d7/d28/f3d [0,4194304] 0 2024-04-20T10:59:16.785 INFO:tasks.workunit.client.0.smithi027.stdout:1/484: truncate d3/d41/f8c 674979 0 2024-04-20T10:59:16.785 INFO:tasks.workunit.client.0.smithi027.stdout:1/485: chown d3/d41/c87 30181 1 2024-04-20T10:59:16.785 INFO:tasks.workunit.client.0.smithi027.stdout:2/322: dwrite d5/d26/f3a [0,4194304] 0 2024-04-20T10:59:16.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/323: readlink d5/dc/l45 0 2024-04-20T10:59:16.786 INFO:tasks.workunit.client.0.smithi027.stdout:5/380: dwrite f0 [8388608,4194304] 0 2024-04-20T10:59:16.787 INFO:tasks.workunit.client.0.smithi027.stdout:8/491: dread d9/de/da3/d3c/d4b/f69 [0,4194304] 0 2024-04-20T10:59:16.787 INFO:tasks.workunit.client.0.smithi027.stdout:8/492: dread - d9/de/d64/d6c/f9e zero size 2024-04-20T10:59:16.787 INFO:tasks.workunit.client.0.smithi027.stdout:3/438: rename d2/dd/c4a to d2/d18/d2c/d4c/d67/d6c/c82 0 2024-04-20T10:59:16.791 INFO:tasks.workunit.client.0.smithi027.stdout:1/486: truncate d3/d7/f1f 712712 0 2024-04-20T10:59:16.793 INFO:tasks.workunit.client.0.smithi027.stdout:5/381: read f3 [1229117,113171] 0 2024-04-20T10:59:16.799 INFO:tasks.workunit.client.0.smithi027.stdout:2/324: dread d5/f4b [0,4194304] 0 2024-04-20T10:59:16.799 INFO:tasks.workunit.client.0.smithi027.stdout:3/439: dread d2/dd/df/d33/f3c [0,4194304] 0 2024-04-20T10:59:16.800 INFO:tasks.workunit.client.0.smithi027.stdout:6/358: dwrite da/df/d19/d2e/f41 [0,4194304] 0 2024-04-20T10:59:16.801 INFO:tasks.workunit.client.0.smithi027.stdout:6/359: readlink da/dd/l50 0 2024-04-20T10:59:16.802 INFO:tasks.workunit.client.0.smithi027.stdout:9/360: dwrite d4/d1b/d50/f56 [0,4194304] 0 2024-04-20T10:59:16.802 INFO:tasks.workunit.client.0.smithi027.stdout:0/402: rename d0/d11/c28 to d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/c85 0 2024-04-20T10:59:16.805 INFO:tasks.workunit.client.0.smithi027.stdout:3/440: link d2/d80/f5a d2/dd/d57/f83 0 2024-04-20T10:59:16.808 INFO:tasks.workunit.client.0.smithi027.stdout:8/493: dwrite d9/de/da3/d3c/f6f [0,4194304] 0 2024-04-20T10:59:16.809 INFO:tasks.workunit.client.0.smithi027.stdout:6/360: link da/d21/d28/d2d/c4a da/df/d19/c78 0 2024-04-20T10:59:16.812 INFO:tasks.workunit.client.0.smithi027.stdout:1/487: rename d3/d20/d85/f9d to d3/d45/fb6 0 2024-04-20T10:59:16.814 INFO:tasks.workunit.client.0.smithi027.stdout:9/361: dread d4/f7 [4194304,4194304] 0 2024-04-20T10:59:16.814 INFO:tasks.workunit.client.0.smithi027.stdout:0/403: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f86 x:0 0 0 2024-04-20T10:59:16.815 INFO:tasks.workunit.client.0.smithi027.stdout:5/382: creat d19/d1b/d1c/d1e/d58/f7b x:0 0 0 2024-04-20T10:59:16.815 INFO:tasks.workunit.client.0.smithi027.stdout:2/325: dwrite d5/dc/d16/d2a/f40 [0,4194304] 0 2024-04-20T10:59:16.815 INFO:tasks.workunit.client.0.smithi027.stdout:5/383: write d19/d1b/d1c/d1e/d22/d47/f6b [934567,86930] 0 2024-04-20T10:59:16.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/361: mkdir da/dd/d79 0 2024-04-20T10:59:16.817 INFO:tasks.workunit.client.0.smithi027.stdout:3/441: rename d2/d18/d2c/l62 to d2/d18/d2c/d4c/d67/l84 0 2024-04-20T10:59:16.819 INFO:tasks.workunit.client.0.smithi027.stdout:0/404: link d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/l5f d0/d11/d2c/d2e/d2b/d7b/l87 0 2024-04-20T10:59:16.823 INFO:tasks.workunit.client.0.smithi027.stdout:2/326: rmdir d5/d26/d47 39 2024-04-20T10:59:16.824 INFO:tasks.workunit.client.0.smithi027.stdout:5/384: mknod d19/d1b/d1c/d1e/d25/d40/d78/c7c 0 2024-04-20T10:59:16.825 INFO:tasks.workunit.client.0.smithi027.stdout:6/362: rmdir da/d21/d28/d2d/d51 39 2024-04-20T10:59:16.825 INFO:tasks.workunit.client.0.smithi027.stdout:8/494: dwrite d9/de/f1d [0,4194304] 0 2024-04-20T10:59:16.826 INFO:tasks.workunit.client.0.smithi027.stdout:9/362: rename d4/d1b/d1f/d45/d68 to d4/d74 0 2024-04-20T10:59:16.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/442: getdents d2/dd/df 0 2024-04-20T10:59:16.829 INFO:tasks.workunit.client.0.smithi027.stdout:5/385: symlink d19/d1b/d1c/d1e/d25/d40/d78/l7d 0 2024-04-20T10:59:16.829 INFO:tasks.workunit.client.0.smithi027.stdout:5/386: fdatasync d19/d1b/d1c/d1e/d22/d47/f55 0 2024-04-20T10:59:16.831 INFO:tasks.workunit.client.0.smithi027.stdout:2/327: rename d5/d1e/f5c to d5/dc/d16/d2a/f68 0 2024-04-20T10:59:16.833 INFO:tasks.workunit.client.0.smithi027.stdout:9/363: creat d4/d1b/d31/f75 x:0 0 0 2024-04-20T10:59:16.833 INFO:tasks.workunit.client.0.smithi027.stdout:9/364: readlink d4/d2a/l2b 0 2024-04-20T10:59:16.833 INFO:tasks.workunit.client.0.smithi027.stdout:9/365: fdatasync d4/d1b/f53 0 2024-04-20T10:59:16.835 INFO:tasks.workunit.client.0.smithi027.stdout:4/380: sync 2024-04-20T10:59:16.835 INFO:tasks.workunit.client.0.smithi027.stdout:7/361: sync 2024-04-20T10:59:16.836 INFO:tasks.workunit.client.0.smithi027.stdout:3/443: truncate d2/dd/f10 7438948 0 2024-04-20T10:59:16.837 INFO:tasks.workunit.client.0.smithi027.stdout:5/387: creat d19/d1b/d1c/f7e x:0 0 0 2024-04-20T10:59:16.837 INFO:tasks.workunit.client.0.smithi027.stdout:1/488: dwrite d3/d41/f4e [0,4194304] 0 2024-04-20T10:59:16.838 INFO:tasks.workunit.client.0.smithi027.stdout:6/363: dread da/df/d19/d2e/d6b/f62 [0,4194304] 0 2024-04-20T10:59:16.838 INFO:tasks.workunit.client.0.smithi027.stdout:6/364: truncate da/d21/f66 785655 0 2024-04-20T10:59:16.838 INFO:tasks.workunit.client.0.smithi027.stdout:4/381: truncate d0/d14/d17/d18/f4b 2481331 0 2024-04-20T10:59:16.838 INFO:tasks.workunit.client.0.smithi027.stdout:0/405: dwrite d0/d11/d2c/d2e/d2b/d7b/d12/f3e [0,4194304] 0 2024-04-20T10:59:16.839 INFO:tasks.workunit.client.0.smithi027.stdout:7/362: creat d4/d5b/f67 x:0 0 0 2024-04-20T10:59:16.840 INFO:tasks.workunit.client.0.smithi027.stdout:5/388: mknod d19/d1b/d1c/d1e/d22/c7f 0 2024-04-20T10:59:16.845 INFO:tasks.workunit.client.0.smithi027.stdout:1/489: symlink d3/d20/lb7 0 2024-04-20T10:59:16.846 INFO:tasks.workunit.client.0.smithi027.stdout:4/382: dread - d0/d14/d17/d1a/f8a zero size 2024-04-20T10:59:16.846 INFO:tasks.workunit.client.0.smithi027.stdout:6/365: getdents da/df/d19/d2e/d6b 0 2024-04-20T10:59:16.848 INFO:tasks.workunit.client.0.smithi027.stdout:7/363: link d4/d4d/f4f d4/d1b/d4a/f68 0 2024-04-20T10:59:16.849 INFO:tasks.workunit.client.0.smithi027.stdout:7/364: readlink d4/d5/d8/d21/l41 0 2024-04-20T10:59:16.849 INFO:tasks.workunit.client.0.smithi027.stdout:7/365: write d4/d1b/d58/f5f [451837,117405] 0 2024-04-20T10:59:16.851 INFO:tasks.workunit.client.0.smithi027.stdout:5/389: symlink d19/d1b/d1c/d1e/d22/d68/l80 0 2024-04-20T10:59:16.851 INFO:tasks.workunit.client.0.smithi027.stdout:5/390: chown d19/d1b/d1c/d1e/d25/d40/c71 438217087 1 2024-04-20T10:59:16.851 INFO:tasks.workunit.client.0.smithi027.stdout:5/391: write fb [1093744,89461] 0 2024-04-20T10:59:16.853 INFO:tasks.workunit.client.0.smithi027.stdout:1/490: mkdir d3/d7/d38/db8 0 2024-04-20T10:59:16.854 INFO:tasks.workunit.client.0.smithi027.stdout:0/406: write d0/d11/d2c/d2e/f1d [3248255,85592] 0 2024-04-20T10:59:16.857 INFO:tasks.workunit.client.0.smithi027.stdout:6/366: symlink da/df/d19/d2e/d32/l7a 0 2024-04-20T10:59:16.860 INFO:tasks.workunit.client.0.smithi027.stdout:5/392: creat d19/d1b/d1c/d1e/d25/d40/f81 x:0 0 0 2024-04-20T10:59:16.861 INFO:tasks.workunit.client.0.smithi027.stdout:1/491: link d3/d20/f33 d3/d7/d38/d62/fb9 0 2024-04-20T10:59:16.861 INFO:tasks.workunit.client.0.smithi027.stdout:1/492: write d3/d7/f1f [1127082,107238] 0 2024-04-20T10:59:16.862 INFO:tasks.workunit.client.0.smithi027.stdout:2/328: dwrite d5/dc/d16/d2a/d4a/f50 [0,4194304] 0 2024-04-20T10:59:16.862 INFO:tasks.workunit.client.0.smithi027.stdout:4/383: unlink d0/d14/d17/d1a/c43 0 2024-04-20T10:59:16.864 INFO:tasks.workunit.client.0.smithi027.stdout:2/329: dread d5/d1e/d21/f3e [0,4194304] 0 2024-04-20T10:59:16.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/407: unlink d0/d11/d2c/d2e/d2b/d7b/l10 0 2024-04-20T10:59:16.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/408: truncate d0/f42 679391 0 2024-04-20T10:59:16.865 INFO:tasks.workunit.client.0.smithi027.stdout:6/367: mknod da/df/c7b 0 2024-04-20T10:59:16.865 INFO:tasks.workunit.client.0.smithi027.stdout:6/368: readlink da/l6c 0 2024-04-20T10:59:16.865 INFO:tasks.workunit.client.0.smithi027.stdout:6/369: chown da/dd/d3c 83119 1 2024-04-20T10:59:16.865 INFO:tasks.workunit.client.0.smithi027.stdout:6/370: truncate da/f1b 1014905 0 2024-04-20T10:59:16.865 INFO:tasks.workunit.client.0.smithi027.stdout:6/371: write da/df/d19/f5d [1939533,103352] 0 2024-04-20T10:59:16.865 INFO:tasks.workunit.client.0.smithi027.stdout:5/393: mkdir d19/d1b/d1c/d1e/d25/d40/d82 0 2024-04-20T10:59:16.866 INFO:tasks.workunit.client.0.smithi027.stdout:5/394: chown d19/d1b/d1c/d1e/d25/d40/c71 36 1 2024-04-20T10:59:16.866 INFO:tasks.workunit.client.0.smithi027.stdout:1/493: mkdir d3/d45/dba 0 2024-04-20T10:59:16.867 INFO:tasks.workunit.client.0.smithi027.stdout:4/384: mkdir d0/d14/d17/d1b/d33/d97 0 2024-04-20T10:59:16.867 INFO:tasks.workunit.client.0.smithi027.stdout:4/385: chown d0/fb 914249 1 2024-04-20T10:59:16.868 INFO:tasks.workunit.client.0.smithi027.stdout:2/330: creat d5/dc/d16/d60/f69 x:0 0 0 2024-04-20T10:59:16.869 INFO:tasks.workunit.client.0.smithi027.stdout:5/395: chown d19/l2d 102864298 1 2024-04-20T10:59:16.869 INFO:tasks.workunit.client.0.smithi027.stdout:5/396: read - d19/d1b/d1c/d1e/d58/f60 zero size 2024-04-20T10:59:16.869 INFO:tasks.workunit.client.0.smithi027.stdout:5/397: truncate f2 12646908 0 2024-04-20T10:59:16.869 INFO:tasks.workunit.client.0.smithi027.stdout:3/444: dwrite d2/f21 [4194304,4194304] 0 2024-04-20T10:59:16.869 INFO:tasks.workunit.client.0.smithi027.stdout:8/495: dwrite d9/de/f1a [4194304,4194304] 0 2024-04-20T10:59:16.870 INFO:tasks.workunit.client.0.smithi027.stdout:1/494: rename d3/d20/f33 to d3/d7/d38/db8/fbb 0 2024-04-20T10:59:16.871 INFO:tasks.workunit.client.0.smithi027.stdout:1/495: write d3/d7/f23 [551052,42711] 0 2024-04-20T10:59:16.871 INFO:tasks.workunit.client.0.smithi027.stdout:1/496: fdatasync d3/d41/f8c 0 2024-04-20T10:59:16.871 INFO:tasks.workunit.client.0.smithi027.stdout:1/497: readlink d3/d7/d28/l31 0 2024-04-20T10:59:16.871 INFO:tasks.workunit.client.0.smithi027.stdout:1/498: dread - d3/f11 zero size 2024-04-20T10:59:16.871 INFO:tasks.workunit.client.0.smithi027.stdout:4/386: creat d0/d14/d17/d1b/d33/d5c/f98 x:0 0 0 2024-04-20T10:59:16.874 INFO:tasks.workunit.client.0.smithi027.stdout:6/372: dread da/df/f10 [0,4194304] 0 2024-04-20T10:59:16.874 INFO:tasks.workunit.client.0.smithi027.stdout:0/409: fdatasync d0/d11/d2c/d2e/f1d 0 2024-04-20T10:59:16.875 INFO:tasks.workunit.client.0.smithi027.stdout:7/366: dwrite d4/d1b/ff [0,4194304] 0 2024-04-20T10:59:16.875 INFO:tasks.workunit.client.0.smithi027.stdout:0/410: dread - d0/d11/d2c/d2e/d55/f6b zero size 2024-04-20T10:59:16.875 INFO:tasks.workunit.client.0.smithi027.stdout:2/331: mknod d5/d1e/d21/c6a 0 2024-04-20T10:59:16.875 INFO:tasks.workunit.client.0.smithi027.stdout:2/332: stat d5/d26/d3d 0 2024-04-20T10:59:16.875 INFO:tasks.workunit.client.0.smithi027.stdout:5/398: mkdir d19/d1b/d1c/d1e/d83 0 2024-04-20T10:59:16.877 INFO:tasks.workunit.client.0.smithi027.stdout:3/445: rename d2/d18/d2c/f47 to d2/dd/df/f85 0 2024-04-20T10:59:16.877 INFO:tasks.workunit.client.0.smithi027.stdout:3/446: fsync d2/d80/f6f 0 2024-04-20T10:59:16.878 INFO:tasks.workunit.client.0.smithi027.stdout:1/499: mkdir d3/d7/d38/da4/dbc 0 2024-04-20T10:59:16.878 INFO:tasks.workunit.client.0.smithi027.stdout:1/500: write d3/d7/f1f [1785512,115036] 0 2024-04-20T10:59:16.879 INFO:tasks.workunit.client.0.smithi027.stdout:4/387: unlink d0/f28 0 2024-04-20T10:59:16.880 INFO:tasks.workunit.client.0.smithi027.stdout:6/373: rmdir da/df/d19/d2e/d6b 39 2024-04-20T10:59:16.880 INFO:tasks.workunit.client.0.smithi027.stdout:6/374: getdents da/dd/d79 0 2024-04-20T10:59:16.882 INFO:tasks.workunit.client.0.smithi027.stdout:7/367: mknod d4/d1b/d4a/c69 0 2024-04-20T10:59:16.883 INFO:tasks.workunit.client.0.smithi027.stdout:0/411: symlink d0/d11/d2c/d2e/d2b/d7b/d34/l88 0 2024-04-20T10:59:16.883 INFO:tasks.workunit.client.0.smithi027.stdout:0/412: fdatasync d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f86 0 2024-04-20T10:59:16.884 INFO:tasks.workunit.client.0.smithi027.stdout:2/333: creat d5/dc/d16/d37/f6b x:0 0 0 2024-04-20T10:59:16.887 INFO:tasks.workunit.client.0.smithi027.stdout:3/447: unlink d2/c56 0 2024-04-20T10:59:16.888 INFO:tasks.workunit.client.0.smithi027.stdout:1/501: creat d3/d20/fbd x:0 0 0 2024-04-20T10:59:16.888 INFO:tasks.workunit.client.0.smithi027.stdout:1/502: fsync d3/d45/f9a 0 2024-04-20T10:59:16.889 INFO:tasks.workunit.client.0.smithi027.stdout:1/503: dread d3/d7/d38/f56 [0,4194304] 0 2024-04-20T10:59:16.895 INFO:tasks.workunit.client.0.smithi027.stdout:4/388: dread d0/f27 [0,4194304] 0 2024-04-20T10:59:16.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/334: creat d5/f6c x:0 0 0 2024-04-20T10:59:16.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/335: chown d5/dc/d16 1005 1 2024-04-20T10:59:16.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/336: fdatasync d5/d1e/f42 0 2024-04-20T10:59:16.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/337: readlink d5/dc/d16/d2a/d4d/l62 0 2024-04-20T10:59:16.895 INFO:tasks.workunit.client.0.smithi027.stdout:5/399: rename d19/d1b/d1c/d1e/d22/d47/c65 to d19/c84 0 2024-04-20T10:59:16.895 INFO:tasks.workunit.client.0.smithi027.stdout:3/448: dread d2/f16 [0,4194304] 0 2024-04-20T10:59:16.895 INFO:tasks.workunit.client.0.smithi027.stdout:0/413: mknod d0/d11/d2c/d2e/d35/c89 0 2024-04-20T10:59:16.897 INFO:tasks.workunit.client.0.smithi027.stdout:8/496: dwrite d9/de/faf [0,4194304] 0 2024-04-20T10:59:16.899 INFO:tasks.workunit.client.0.smithi027.stdout:7/368: dwrite d4/d5/f54 [0,4194304] 0 2024-04-20T10:59:16.899 INFO:tasks.workunit.client.0.smithi027.stdout:2/338: symlink d5/d26/d47/l6d 0 2024-04-20T10:59:16.899 INFO:tasks.workunit.client.0.smithi027.stdout:6/375: rmdir da/dd/d3c/d58 0 2024-04-20T10:59:16.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/449: creat d2/dd/df/d23/f86 x:0 0 0 2024-04-20T10:59:16.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/450: chown d2/dd/d57/l74 28638 1 2024-04-20T10:59:16.901 INFO:tasks.workunit.client.0.smithi027.stdout:3/451: write d2/dd/d57/f59 [279763,46433] 0 2024-04-20T10:59:16.901 INFO:tasks.workunit.client.0.smithi027.stdout:0/414: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/f8a x:0 0 0 2024-04-20T10:59:16.905 INFO:tasks.workunit.client.0.smithi027.stdout:8/497: rename d9/de/d20/d48/l8a to d9/de/d1e/lb2 0 2024-04-20T10:59:16.906 INFO:tasks.workunit.client.0.smithi027.stdout:7/369: creat d4/d1b/d4a/d62/f6a x:0 0 0 2024-04-20T10:59:16.907 INFO:tasks.workunit.client.0.smithi027.stdout:5/400: mknod d19/c85 0 2024-04-20T10:59:16.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/339: creat d5/dc/d16/d37/f6e x:0 0 0 2024-04-20T10:59:16.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/340: write d5/d26/d47/f5f [1603265,65139] 0 2024-04-20T10:59:16.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/341: fsync d5/dc/d16/d32/f5a 0 2024-04-20T10:59:16.912 INFO:tasks.workunit.client.0.smithi027.stdout:6/376: rename da/dd/d3d/f64 to da/d21/d28/d54/f7c 0 2024-04-20T10:59:16.912 INFO:tasks.workunit.client.0.smithi027.stdout:8/498: unlink d9/de/l14 0 2024-04-20T10:59:16.913 INFO:tasks.workunit.client.0.smithi027.stdout:5/401: write d19/d1b/d1c/d1e/d22/d47/f6b [350237,103] 0 2024-04-20T10:59:16.915 INFO:tasks.workunit.client.0.smithi027.stdout:4/389: dwrite d0/fb [4194304,4194304] 0 2024-04-20T10:59:16.921 INFO:tasks.workunit.client.0.smithi027.stdout:1/504: dwrite d3/d41/f4e [0,4194304] 0 2024-04-20T10:59:16.921 INFO:tasks.workunit.client.0.smithi027.stdout:1/505: stat d3/d7e/c88 0 2024-04-20T10:59:16.921 INFO:tasks.workunit.client.0.smithi027.stdout:1/506: write d3/d7/fa [303179,80415] 0 2024-04-20T10:59:16.923 INFO:tasks.workunit.client.0.smithi027.stdout:6/377: dread da/d21/d28/d2d/d53/f6f [0,4194304] 0 2024-04-20T10:59:16.931 INFO:tasks.workunit.client.0.smithi027.stdout:0/415: dwrite d0/f1f [0,4194304] 0 2024-04-20T10:59:16.931 INFO:tasks.workunit.client.0.smithi027.stdout:0/416: fdatasync d0/d11/f21 0 2024-04-20T10:59:16.932 INFO:tasks.workunit.client.0.smithi027.stdout:2/342: dwrite d5/d26/d3d/f48 [0,4194304] 0 2024-04-20T10:59:17.169 INFO:tasks.workunit.client.0.smithi027.stdout:8/499: getdents d9/de/d1e/d4a/d6d/d75 0 2024-04-20T10:59:17.170 INFO:tasks.workunit.client.0.smithi027.stdout:7/370: mknod d4/c6b 0 2024-04-20T10:59:17.171 INFO:tasks.workunit.client.0.smithi027.stdout:5/402: creat d19/d1b/f86 x:0 0 0 2024-04-20T10:59:17.173 INFO:tasks.workunit.client.0.smithi027.stdout:4/390: getdents d0/d14/d17/d1b 0 2024-04-20T10:59:17.175 INFO:tasks.workunit.client.0.smithi027.stdout:1/507: creat d3/d7/d38/d62/fbe x:0 0 0 2024-04-20T10:59:17.175 INFO:tasks.workunit.client.0.smithi027.stdout:1/508: chown d3/d41/f80 6256407 1 2024-04-20T10:59:17.178 INFO:tasks.workunit.client.0.smithi027.stdout:0/417: symlink d0/d11/d2c/d2e/d2b/d7b/d12/d25/l8b 0 2024-04-20T10:59:17.179 INFO:tasks.workunit.client.0.smithi027.stdout:2/343: mknod d5/d26/d3d/c6f 0 2024-04-20T10:59:17.185 INFO:tasks.workunit.client.0.smithi027.stdout:5/403: link d19/d1b/c53 d19/d1b/d1c/d1e/d83/c87 0 2024-04-20T10:59:17.187 INFO:tasks.workunit.client.0.smithi027.stdout:1/509: creat d3/d7/d28/d92/dad/fbf x:0 0 0 2024-04-20T10:59:17.189 INFO:tasks.workunit.client.0.smithi027.stdout:7/371: dwrite d4/d1b/f35 [0,4194304] 0 2024-04-20T10:59:17.189 INFO:tasks.workunit.client.0.smithi027.stdout:6/378: rename da/df/d1f/c71 to da/c7d 0 2024-04-20T10:59:17.190 INFO:tasks.workunit.client.0.smithi027.stdout:0/418: write d0/d11/d2c/d2e/d2b/d7b/f26 [6245766,42850] 0 2024-04-20T10:59:17.190 INFO:tasks.workunit.client.0.smithi027.stdout:5/404: dread d19/d1b/d1c/d1e/f56 [0,4194304] 0 2024-04-20T10:59:17.190 INFO:tasks.workunit.client.0.smithi027.stdout:2/344: creat d5/d1e/f70 x:0 0 0 2024-04-20T10:59:17.193 INFO:tasks.workunit.client.0.smithi027.stdout:1/510: creat d3/d7/d38/da4/dac/fc0 x:0 0 0 2024-04-20T10:59:17.194 INFO:tasks.workunit.client.0.smithi027.stdout:7/372: creat d4/d5/f6c x:0 0 0 2024-04-20T10:59:17.195 INFO:tasks.workunit.client.0.smithi027.stdout:6/379: mkdir da/df/d7e 0 2024-04-20T10:59:17.196 INFO:tasks.workunit.client.0.smithi027.stdout:8/500: dwrite d9/de/d20/d4f/d97/fa4 [0,4194304] 0 2024-04-20T10:59:17.198 INFO:tasks.workunit.client.0.smithi027.stdout:4/391: dwrite d0/d48/f91 [0,4194304] 0 2024-04-20T10:59:17.198 INFO:tasks.workunit.client.0.smithi027.stdout:5/405: creat d19/d1b/d1c/d1e/f88 x:0 0 0 2024-04-20T10:59:17.198 INFO:tasks.workunit.client.0.smithi027.stdout:4/392: write d0/d14/d17/d1b/f7f [538428,38963] 0 2024-04-20T10:59:17.199 INFO:tasks.workunit.client.0.smithi027.stdout:2/345: rename d5/dc/d16/d2a/f68 to d5/d26/d47/f71 0 2024-04-20T10:59:17.201 INFO:tasks.workunit.client.0.smithi027.stdout:1/511: write d3/d7/d28/d92/fb4 [2087988,64664] 0 2024-04-20T10:59:17.201 INFO:tasks.workunit.client.0.smithi027.stdout:6/380: dread da/d21/d28/d54/f7c [0,4194304] 0 2024-04-20T10:59:17.201 INFO:tasks.workunit.client.0.smithi027.stdout:1/512: dread - d3/d7/d38/fa7 zero size 2024-04-20T10:59:17.201 INFO:tasks.workunit.client.0.smithi027.stdout:6/381: fdatasync da/df/f1a 0 2024-04-20T10:59:17.201 INFO:tasks.workunit.client.0.smithi027.stdout:1/513: write d3/d7/f34 [357448,55976] 0 2024-04-20T10:59:17.201 INFO:tasks.workunit.client.0.smithi027.stdout:6/382: write da/df/f17 [2074223,61586] 0 2024-04-20T10:59:17.202 INFO:tasks.workunit.client.0.smithi027.stdout:9/366: sync 2024-04-20T10:59:17.202 INFO:tasks.workunit.client.0.smithi027.stdout:7/373: mknod d4/d5/d8/d46/c6d 0 2024-04-20T10:59:17.203 INFO:tasks.workunit.client.0.smithi027.stdout:8/501: read - d9/de/d64/d6c/f93 zero size 2024-04-20T10:59:17.204 INFO:tasks.workunit.client.0.smithi027.stdout:0/419: mkdir d0/d11/d2c/d2e/d35/d8c 0 2024-04-20T10:59:17.205 INFO:tasks.workunit.client.0.smithi027.stdout:5/406: getdents d19/d1b/d1c/d1e/d22/d68/d75 0 2024-04-20T10:59:17.206 INFO:tasks.workunit.client.0.smithi027.stdout:5/407: fdatasync fb 0 2024-04-20T10:59:17.206 INFO:tasks.workunit.client.0.smithi027.stdout:5/408: chown l13 756360 1 2024-04-20T10:59:17.206 INFO:tasks.workunit.client.0.smithi027.stdout:4/393: getdents d0/d14/d6b 0 2024-04-20T10:59:17.209 INFO:tasks.workunit.client.0.smithi027.stdout:1/514: rename d3/d7/d28/d35/fa0 to d3/d7/d38/da4/fc1 0 2024-04-20T10:59:17.209 INFO:tasks.workunit.client.0.smithi027.stdout:3/452: sync 2024-04-20T10:59:17.209 INFO:tasks.workunit.client.0.smithi027.stdout:6/383: creat da/dd/d3d/f7f x:0 0 0 2024-04-20T10:59:17.211 INFO:tasks.workunit.client.0.smithi027.stdout:8/502: readlink d9/de/d1e/d4a/l3d 0 2024-04-20T10:59:17.211 INFO:tasks.workunit.client.0.smithi027.stdout:9/367: dread d4/d1b/d31/d23/d29/f4c [0,4194304] 0 2024-04-20T10:59:17.212 INFO:tasks.workunit.client.0.smithi027.stdout:0/420: mknod d0/d11/d2c/d2e/c8d 0 2024-04-20T10:59:17.214 INFO:tasks.workunit.client.0.smithi027.stdout:4/394: symlink d0/d14/d17/d53/d79/l99 0 2024-04-20T10:59:17.214 INFO:tasks.workunit.client.0.smithi027.stdout:4/395: write d0/d14/d17/f16 [2761748,26736] 0 2024-04-20T10:59:17.216 INFO:tasks.workunit.client.0.smithi027.stdout:1/515: dread d3/d7/d38/da4/fc1 [0,4194304] 0 2024-04-20T10:59:17.216 INFO:tasks.workunit.client.0.smithi027.stdout:7/374: rename d4/d1b/d4a/d62/f66 to d4/d5/d8/d21/f6e 0 2024-04-20T10:59:17.218 INFO:tasks.workunit.client.0.smithi027.stdout:8/503: fdatasync d9/de/f47 0 2024-04-20T10:59:17.218 INFO:tasks.workunit.client.0.smithi027.stdout:8/504: readlink d9/de/d1e/lb2 0 2024-04-20T10:59:17.218 INFO:tasks.workunit.client.0.smithi027.stdout:9/368: symlink d4/d1b/d1f/d45/l76 0 2024-04-20T10:59:17.219 INFO:tasks.workunit.client.0.smithi027.stdout:6/384: write da/f2a [2903402,106583] 0 2024-04-20T10:59:17.220 INFO:tasks.workunit.client.0.smithi027.stdout:0/421: rmdir d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/d67 0 2024-04-20T10:59:17.220 INFO:tasks.workunit.client.0.smithi027.stdout:0/422: read - d0/d11/d62/f81 zero size 2024-04-20T10:59:17.222 INFO:tasks.workunit.client.0.smithi027.stdout:4/396: unlink d0/d14/l44 0 2024-04-20T10:59:17.222 INFO:tasks.workunit.client.0.smithi027.stdout:5/409: dwrite d19/d1b/d1c/f45 [0,4194304] 0 2024-04-20T10:59:17.223 INFO:tasks.workunit.client.0.smithi027.stdout:1/516: truncate d3/d7/ff 877122 0 2024-04-20T10:59:17.223 INFO:tasks.workunit.client.0.smithi027.stdout:1/517: fdatasync d3/d20/fbd 0 2024-04-20T10:59:17.224 INFO:tasks.workunit.client.0.smithi027.stdout:7/375: mkdir d4/d5/d6f 0 2024-04-20T10:59:17.225 INFO:tasks.workunit.client.0.smithi027.stdout:6/385: dread da/df/d19/d2e/f45 [0,4194304] 0 2024-04-20T10:59:17.225 INFO:tasks.workunit.client.0.smithi027.stdout:0/423: mknod d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/c8e 0 2024-04-20T10:59:17.226 INFO:tasks.workunit.client.0.smithi027.stdout:4/397: creat d0/d14/d17/d1b/d33/f9a x:0 0 0 2024-04-20T10:59:17.227 INFO:tasks.workunit.client.0.smithi027.stdout:4/398: write d0/d14/d17/d1b/f4c [1553359,80249] 0 2024-04-20T10:59:17.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/518: mknod d3/cc2 0 2024-04-20T10:59:17.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/519: chown d3/d20/c8a 44260929 1 2024-04-20T10:59:17.231 INFO:tasks.workunit.client.0.smithi027.stdout:8/505: mkdir d9/de/d20/d48/db3 0 2024-04-20T10:59:17.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/424: mknod d0/d11/d2c/d2e/d2b/d7b/d34/c8f 0 2024-04-20T10:59:17.233 INFO:tasks.workunit.client.0.smithi027.stdout:0/425: fdatasync d0/d11/d2c/d2e/d2b/d7b/d34/d5a/f75 0 2024-04-20T10:59:17.233 INFO:tasks.workunit.client.0.smithi027.stdout:0/426: chown d0/d11/d2c/d2e/d61/f78 12 1 2024-04-20T10:59:17.233 INFO:tasks.workunit.client.0.smithi027.stdout:4/399: creat d0/d14/d17/d53/d6a/f9b x:0 0 0 2024-04-20T10:59:17.236 INFO:tasks.workunit.client.0.smithi027.stdout:7/376: rename d4/f65 to d4/f70 0 2024-04-20T10:59:17.244 INFO:tasks.workunit.client.0.smithi027.stdout:5/410: dwrite f18 [0,4194304] 0 2024-04-20T10:59:17.252 INFO:tasks.workunit.client.0.smithi027.stdout:6/386: dwrite da/df/d19/d2e/d32/f55 [0,4194304] 0 2024-04-20T10:59:17.253 INFO:tasks.workunit.client.0.smithi027.stdout:6/387: write da/d21/d28/d2d/f43 [621341,115521] 0 2024-04-20T10:59:17.261 INFO:tasks.workunit.client.0.smithi027.stdout:5/411: dwrite d19/d1b/d1c/f7e [0,4194304] 0 2024-04-20T10:59:17.261 INFO:tasks.workunit.client.0.smithi027.stdout:5/412: read - d19/d1b/d1c/d1e/d22/d68/d75/f77 zero size 2024-04-20T10:59:17.262 INFO:tasks.workunit.client.0.smithi027.stdout:5/413: dread d19/d1b/d1c/d1e/f56 [0,4194304] 0 2024-04-20T10:59:17.269 INFO:tasks.workunit.client.0.smithi027.stdout:5/414: dread d19/d1b/d1c/f3b [0,4194304] 0 2024-04-20T10:59:17.490 INFO:tasks.workunit.client.0.smithi027.stdout:0/427: rename d0/d11/d62/f57 to d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f90 0 2024-04-20T10:59:17.490 INFO:tasks.workunit.client.0.smithi027.stdout:4/400: creat d0/d14/d17/d1b/f9c x:0 0 0 2024-04-20T10:59:17.490 INFO:tasks.workunit.client.0.smithi027.stdout:1/520: unlink d3/d7/d2d/d73/f94 0 2024-04-20T10:59:17.490 INFO:tasks.workunit.client.0.smithi027.stdout:7/377: creat d4/d5/d8/d45/f71 x:0 0 0 2024-04-20T10:59:17.490 INFO:tasks.workunit.client.0.smithi027.stdout:7/378: chown d4/d5/d8/d21 148283 1 2024-04-20T10:59:17.490 INFO:tasks.workunit.client.0.smithi027.stdout:6/388: truncate da/df/d1f/f3f 653088 0 2024-04-20T10:59:17.490 INFO:tasks.workunit.client.0.smithi027.stdout:6/389: write da/f1b [821562,20181] 0 2024-04-20T10:59:17.495 INFO:tasks.workunit.client.0.smithi027.stdout:5/415: getdents d19/d1b/d1c/d1e/d58 0 2024-04-20T10:59:17.499 INFO:tasks.workunit.client.0.smithi027.stdout:0/428: symlink d0/d11/d2c/d2e/d2b/l91 0 2024-04-20T10:59:17.499 INFO:tasks.workunit.client.0.smithi027.stdout:0/429: stat d0/d11/d2c/d2e/d2b/d7b/f1c 0 2024-04-20T10:59:17.502 INFO:tasks.workunit.client.0.smithi027.stdout:1/521: symlink d3/lc3 0 2024-04-20T10:59:17.506 INFO:tasks.workunit.client.0.smithi027.stdout:7/379: rename d4/d5/f54 to d4/d1b/f72 0 2024-04-20T10:59:17.506 INFO:tasks.workunit.client.0.smithi027.stdout:5/416: creat d19/d1b/d1c/d1e/d22/f89 x:0 0 0 2024-04-20T10:59:17.506 INFO:tasks.workunit.client.0.smithi027.stdout:0/430: mkdir d0/d11/d2c/d2e/d2b/d7b/d34/d92 0 2024-04-20T10:59:17.508 INFO:tasks.workunit.client.0.smithi027.stdout:7/380: symlink d4/d4d/l73 0 2024-04-20T10:59:17.508 INFO:tasks.workunit.client.0.smithi027.stdout:7/381: chown d4/d1b/f72 1638985452 1 2024-04-20T10:59:17.508 INFO:tasks.workunit.client.0.smithi027.stdout:7/382: write d4/f2e [276984,38209] 0 2024-04-20T10:59:17.511 INFO:tasks.workunit.client.0.smithi027.stdout:3/453: sync 2024-04-20T10:59:17.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/346: sync 2024-04-20T10:59:17.511 INFO:tasks.workunit.client.0.smithi027.stdout:9/369: sync 2024-04-20T10:59:17.512 INFO:tasks.workunit.client.0.smithi027.stdout:8/506: dwrite d9/de/f1a [0,4194304] 0 2024-04-20T10:59:17.512 INFO:tasks.workunit.client.0.smithi027.stdout:8/507: dread - d9/de/d64/d6c/f9e zero size 2024-04-20T10:59:17.513 INFO:tasks.workunit.client.0.smithi027.stdout:7/383: creat d4/d5/d8/d3b/f74 x:0 0 0 2024-04-20T10:59:17.513 INFO:tasks.workunit.client.0.smithi027.stdout:7/384: chown d4/d5/d8/d46 15170921 1 2024-04-20T10:59:17.515 INFO:tasks.workunit.client.0.smithi027.stdout:3/454: creat d2/d18/d2c/d4c/d67/d6c/f87 x:0 0 0 2024-04-20T10:59:17.516 INFO:tasks.workunit.client.0.smithi027.stdout:2/347: mknod d5/dc/d16/d2a/d4a/c72 0 2024-04-20T10:59:17.516 INFO:tasks.workunit.client.0.smithi027.stdout:2/348: chown d5/d1e/d21/l49 0 1 2024-04-20T10:59:17.520 INFO:tasks.workunit.client.0.smithi027.stdout:8/508: rmdir d9/de/d1e/d4a/d6d 39 2024-04-20T10:59:17.520 INFO:tasks.workunit.client.0.smithi027.stdout:4/401: dwrite d0/d14/d17/d1b/d33/d5c/f98 [0,4194304] 0 2024-04-20T10:59:17.521 INFO:tasks.workunit.client.0.smithi027.stdout:3/455: mkdir d2/dd/df/d23/d68/d88 0 2024-04-20T10:59:17.522 INFO:tasks.workunit.client.0.smithi027.stdout:6/390: dwrite da/d21/d28/d54/f7c [0,4194304] 0 2024-04-20T10:59:17.523 INFO:tasks.workunit.client.0.smithi027.stdout:2/349: mknod d5/dc/d16/d2a/d59/d66/c73 0 2024-04-20T10:59:17.525 INFO:tasks.workunit.client.0.smithi027.stdout:4/402: fdatasync d0/d14/d17/d1a/f70 0 2024-04-20T10:59:17.525 INFO:tasks.workunit.client.0.smithi027.stdout:3/456: creat d2/dd/df/d23/d68/d88/f89 x:0 0 0 2024-04-20T10:59:17.528 INFO:tasks.workunit.client.0.smithi027.stdout:6/391: unlink da/d21/d28/f3e 0 2024-04-20T10:59:17.531 INFO:tasks.workunit.client.0.smithi027.stdout:2/350: rename d5/dc/d16/d2a/f54 to d5/d26/f74 0 2024-04-20T10:59:17.532 INFO:tasks.workunit.client.0.smithi027.stdout:3/457: symlink d2/d7f/l8a 0 2024-04-20T10:59:17.532 INFO:tasks.workunit.client.0.smithi027.stdout:1/522: dwrite d3/d7/d38/d62/f81 [0,4194304] 0 2024-04-20T10:59:17.534 INFO:tasks.workunit.client.0.smithi027.stdout:4/403: rename d0/d14/d17/d18/f36 to d0/d48/f9d 0 2024-04-20T10:59:17.534 INFO:tasks.workunit.client.0.smithi027.stdout:4/404: fsync d0/d14/d17/d1b/d33/f57 0 2024-04-20T10:59:17.535 INFO:tasks.workunit.client.0.smithi027.stdout:0/431: dwrite d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f6c [0,4194304] 0 2024-04-20T10:59:17.536 INFO:tasks.workunit.client.0.smithi027.stdout:6/392: symlink da/l80 0 2024-04-20T10:59:17.538 INFO:tasks.workunit.client.0.smithi027.stdout:1/523: rename d3/d7/d2d/f99 to d3/d7/d2d/d73/fc4 0 2024-04-20T10:59:17.540 INFO:tasks.workunit.client.0.smithi027.stdout:5/417: dwrite f18 [0,4194304] 0 2024-04-20T10:59:17.540 INFO:tasks.workunit.client.0.smithi027.stdout:5/418: truncate f16 1712558 0 2024-04-20T10:59:17.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/351: write d5/dc/d16/d37/f3f [2606870,4741] 0 2024-04-20T10:59:17.541 INFO:tasks.workunit.client.0.smithi027.stdout:0/432: creat d0/d11/d2c/d2e/d2b/d7b/d64/f93 x:0 0 0 2024-04-20T10:59:17.542 INFO:tasks.workunit.client.0.smithi027.stdout:6/393: rename da/df/d19/d2e/d32/f55 to da/df/d7e/f81 0 2024-04-20T10:59:17.543 INFO:tasks.workunit.client.0.smithi027.stdout:1/524: stat d3/d7/c15 0 2024-04-20T10:59:17.543 INFO:tasks.workunit.client.0.smithi027.stdout:1/525: chown d3/d20/c8a 754217 1 2024-04-20T10:59:17.544 INFO:tasks.workunit.client.0.smithi027.stdout:4/405: dread d0/d14/d17/d1b/f61 [0,4194304] 0 2024-04-20T10:59:17.544 INFO:tasks.workunit.client.0.smithi027.stdout:4/406: stat d0/d14/d17/d1b/d33/f57 0 2024-04-20T10:59:17.545 INFO:tasks.workunit.client.0.smithi027.stdout:5/419: mknod d19/d1b/d1c/d1e/d3c/c8a 0 2024-04-20T10:59:17.546 INFO:tasks.workunit.client.0.smithi027.stdout:7/385: dwrite d4/d5/d8/f43 [0,4194304] 0 2024-04-20T10:59:17.547 INFO:tasks.workunit.client.0.smithi027.stdout:4/407: creat d0/d14/d17/f9e x:0 0 0 2024-04-20T10:59:17.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/352: dread d5/dc/f1c [0,4194304] 0 2024-04-20T10:59:17.551 INFO:tasks.workunit.client.0.smithi027.stdout:8/509: dwrite d9/de/d20/f25 [0,4194304] 0 2024-04-20T10:59:17.551 INFO:tasks.workunit.client.0.smithi027.stdout:7/386: creat d4/d1b/d58/f75 x:0 0 0 2024-04-20T10:59:17.551 INFO:tasks.workunit.client.0.smithi027.stdout:4/408: link d0/d48/l4f d0/d14/d17/d1b/d33/d97/l9f 0 2024-04-20T10:59:17.551 INFO:tasks.workunit.client.0.smithi027.stdout:4/409: fsync d0/d14/d17/d1b/d2f/f82 0 2024-04-20T10:59:17.551 INFO:tasks.workunit.client.0.smithi027.stdout:7/387: write d4/d1b/f63 [783554,63292] 0 2024-04-20T10:59:17.552 INFO:tasks.workunit.client.0.smithi027.stdout:5/420: symlink d19/l8b 0 2024-04-20T10:59:17.552 INFO:tasks.workunit.client.0.smithi027.stdout:5/421: fsync d19/d1b/d1c/d1e/d22/d68/d75/f77 0 2024-04-20T10:59:17.552 INFO:tasks.workunit.client.0.smithi027.stdout:5/422: write d19/d1b/d1c/d1e/d25/f2f [1260149,57677] 0 2024-04-20T10:59:17.553 INFO:tasks.workunit.client.0.smithi027.stdout:2/353: creat d5/dc/d16/d2a/f75 x:0 0 0 2024-04-20T10:59:17.553 INFO:tasks.workunit.client.0.smithi027.stdout:2/354: dread - d5/dc/d16/d37/f46 zero size 2024-04-20T10:59:17.553 INFO:tasks.workunit.client.0.smithi027.stdout:6/394: write da/d21/f70 [3265319,113024] 0 2024-04-20T10:59:17.554 INFO:tasks.workunit.client.0.smithi027.stdout:8/510: creat d9/de/d1e/d4a/d6d/d79/fb4 x:0 0 0 2024-04-20T10:59:17.555 INFO:tasks.workunit.client.0.smithi027.stdout:8/511: chown d9/de/d20/f73 152190960 1 2024-04-20T10:59:17.555 INFO:tasks.workunit.client.0.smithi027.stdout:8/512: chown d9/de/d20/d48/d62 1807034 1 2024-04-20T10:59:17.557 INFO:tasks.workunit.client.0.smithi027.stdout:4/410: creat d0/d14/d17/d1b/d33/d5c/d6f/fa0 x:0 0 0 2024-04-20T10:59:17.557 INFO:tasks.workunit.client.0.smithi027.stdout:4/411: write d0/d14/d17/d1b/f9c [86773,39571] 0 2024-04-20T10:59:17.557 INFO:tasks.workunit.client.0.smithi027.stdout:4/412: write d0/f13 [4123339,114094] 0 2024-04-20T10:59:17.558 INFO:tasks.workunit.client.0.smithi027.stdout:7/388: symlink d4/d5/d8/d3b/l76 0 2024-04-20T10:59:17.558 INFO:tasks.workunit.client.0.smithi027.stdout:7/389: fsync d4/d5/d8/d3b/f74 0 2024-04-20T10:59:17.558 INFO:tasks.workunit.client.0.smithi027.stdout:7/390: write d4/d1b/d4a/d62/f6a [431954,56323] 0 2024-04-20T10:59:17.560 INFO:tasks.workunit.client.0.smithi027.stdout:3/458: dwrite d2/d18/f5b [0,4194304] 0 2024-04-20T10:59:17.561 INFO:tasks.workunit.client.0.smithi027.stdout:5/423: mknod d19/d1b/d1c/d1e/d25/d40/c8c 0 2024-04-20T10:59:17.561 INFO:tasks.workunit.client.0.smithi027.stdout:5/424: dread - d19/d1b/d1c/d1e/d3c/f74 zero size 2024-04-20T10:59:17.561 INFO:tasks.workunit.client.0.smithi027.stdout:5/425: readlink l13 0 2024-04-20T10:59:17.562 INFO:tasks.workunit.client.0.smithi027.stdout:2/355: rename d5/dc/d16/d2a/d59/d66/c73 to d5/c76 0 2024-04-20T10:59:17.563 INFO:tasks.workunit.client.0.smithi027.stdout:2/356: chown d5/d26/d3d/f48 855334498 1 2024-04-20T10:59:17.563 INFO:tasks.workunit.client.0.smithi027.stdout:6/395: dread da/d21/d28/d54/f7c [0,4194304] 0 2024-04-20T10:59:17.563 INFO:tasks.workunit.client.0.smithi027.stdout:2/357: readlink d5/dc/d16/d32/l61 0 2024-04-20T10:59:17.563 INFO:tasks.workunit.client.0.smithi027.stdout:8/513: write d9/de/da3/d3c/f9c [7279270,14171] 0 2024-04-20T10:59:17.563 INFO:tasks.workunit.client.0.smithi027.stdout:2/358: dread - d5/dc/d16/d32/f5a zero size 2024-04-20T10:59:17.563 INFO:tasks.workunit.client.0.smithi027.stdout:0/433: dwrite d0/d59/f73 [0,4194304] 0 2024-04-20T10:59:17.563 INFO:tasks.workunit.client.0.smithi027.stdout:1/526: dread d3/d45/d5d/d6c/f90 [0,4194304] 0 2024-04-20T10:59:17.564 INFO:tasks.workunit.client.0.smithi027.stdout:4/413: mkdir d0/d14/d17/d53/d6a/da1 0 2024-04-20T10:59:17.564 INFO:tasks.workunit.client.0.smithi027.stdout:7/391: mkdir d4/d5/d8/d46/d77 0 2024-04-20T10:59:17.565 INFO:tasks.workunit.client.0.smithi027.stdout:7/392: fsync d4/d5/f40 0 2024-04-20T10:59:17.565 INFO:tasks.workunit.client.0.smithi027.stdout:7/393: readlink d4/d5/l55 0 2024-04-20T10:59:17.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/359: creat d5/dc/d16/d2a/f77 x:0 0 0 2024-04-20T10:59:17.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/360: chown d5/dc/d16/d37/f4e 5009 1 2024-04-20T10:59:17.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/361: fdatasync d5/dc/d16/d2a/f75 0 2024-04-20T10:59:17.567 INFO:tasks.workunit.client.0.smithi027.stdout:2/362: fsync d5/d26/d47/f5f 0 2024-04-20T10:59:17.567 INFO:tasks.workunit.client.0.smithi027.stdout:2/363: dread - d5/dc/d16/d32/f5a zero size 2024-04-20T10:59:17.568 INFO:tasks.workunit.client.0.smithi027.stdout:0/434: mknod d0/d11/d4b/c94 0 2024-04-20T10:59:17.569 INFO:tasks.workunit.client.0.smithi027.stdout:1/527: symlink d3/d41/lc5 0 2024-04-20T10:59:17.570 INFO:tasks.workunit.client.0.smithi027.stdout:2/364: rename d5/dc/d16/d2a/d4d to d5/d26/d78 0 2024-04-20T10:59:17.571 INFO:tasks.workunit.client.0.smithi027.stdout:1/528: mkdir d3/d41/dc6 0 2024-04-20T10:59:17.580 INFO:tasks.workunit.client.0.smithi027.stdout:3/459: dread d2/f1f [0,4194304] 0 2024-04-20T10:59:17.580 INFO:tasks.workunit.client.0.smithi027.stdout:3/460: truncate d2/dd/d57/f59 1703932 0 2024-04-20T10:59:17.580 INFO:tasks.workunit.client.0.smithi027.stdout:3/461: symlink d2/dd/d57/l8b 0 2024-04-20T10:59:17.582 INFO:tasks.workunit.client.0.smithi027.stdout:3/462: mkdir d2/dd/d57/d8c 0 2024-04-20T10:59:17.582 INFO:tasks.workunit.client.0.smithi027.stdout:0/435: dread d0/d11/d2c/d2e/d2b/d7b/d12/f3e [0,4194304] 0 2024-04-20T10:59:17.583 INFO:tasks.workunit.client.0.smithi027.stdout:3/463: getdents d2/d18/d2c/d4c/d67/d6c 0 2024-04-20T10:59:17.583 INFO:tasks.workunit.client.0.smithi027.stdout:3/464: fdatasync d2/dd/df/d23/d68/d88/f89 0 2024-04-20T10:59:17.585 INFO:tasks.workunit.client.0.smithi027.stdout:1/529: write d3/d45/d5d/d6c/f90 [1235622,8673] 0 2024-04-20T10:59:17.585 INFO:tasks.workunit.client.0.smithi027.stdout:0/436: creat d0/d11/d5c/f95 x:0 0 0 2024-04-20T10:59:17.585 INFO:tasks.workunit.client.0.smithi027.stdout:0/437: write d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/f8a [558591,102956] 0 2024-04-20T10:59:17.585 INFO:tasks.workunit.client.0.smithi027.stdout:0/438: readlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/l79 0 2024-04-20T10:59:17.586 INFO:tasks.workunit.client.0.smithi027.stdout:0/439: dread - d0/d11/d2c/d2e/d61/f70 zero size 2024-04-20T10:59:17.587 INFO:tasks.workunit.client.0.smithi027.stdout:0/440: rename d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f6c to d0/d11/d2c/d2e/f96 0 2024-04-20T10:59:17.594 INFO:tasks.workunit.client.0.smithi027.stdout:6/396: dwrite da/d21/d28/d2d/d53/f76 [0,4194304] 0 2024-04-20T10:59:17.594 INFO:tasks.workunit.client.0.smithi027.stdout:7/394: dwrite d4/d5/f1f [0,4194304] 0 2024-04-20T10:59:17.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/426: dwrite d19/d1b/d1c/d1e/f56 [0,4194304] 0 2024-04-20T10:59:17.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/427: write d19/d1b/f86 [375865,65133] 0 2024-04-20T10:59:17.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/428: write f3 [4722569,2815] 0 2024-04-20T10:59:17.597 INFO:tasks.workunit.client.0.smithi027.stdout:2/365: dwrite d5/d44/f65 [0,4194304] 0 2024-04-20T10:59:17.597 INFO:tasks.workunit.client.0.smithi027.stdout:7/395: symlink d4/d1b/d58/l78 0 2024-04-20T10:59:17.597 INFO:tasks.workunit.client.0.smithi027.stdout:0/441: dread d0/d11/d2c/d2e/f96 [0,4194304] 0 2024-04-20T10:59:17.597 INFO:tasks.workunit.client.0.smithi027.stdout:0/442: fsync d0/d11/d2c/d2e/d2b/d3d/f4c 0 2024-04-20T10:59:17.598 INFO:tasks.workunit.client.0.smithi027.stdout:5/429: rename d19/d1b/d1c/d1e/d22/f32 to d19/d1b/f8d 0 2024-04-20T10:59:17.599 INFO:tasks.workunit.client.0.smithi027.stdout:2/366: creat d5/d26/d30/f79 x:0 0 0 2024-04-20T10:59:17.601 INFO:tasks.workunit.client.0.smithi027.stdout:1/530: dwrite d3/d7/d38/f65 [4194304,4194304] 0 2024-04-20T10:59:17.601 INFO:tasks.workunit.client.0.smithi027.stdout:1/531: chown d3/d7/d2d 4 1 2024-04-20T10:59:17.601 INFO:tasks.workunit.client.0.smithi027.stdout:1/532: write d3/d7/f57 [218533,89807] 0 2024-04-20T10:59:17.601 INFO:tasks.workunit.client.0.smithi027.stdout:7/396: mknod d4/d1b/d4a/d62/c79 0 2024-04-20T10:59:17.601 INFO:tasks.workunit.client.0.smithi027.stdout:7/397: write d4/d1b/d4a/d62/f6a [1353576,15164] 0 2024-04-20T10:59:17.602 INFO:tasks.workunit.client.0.smithi027.stdout:5/430: symlink d19/d1b/d1c/d1e/d25/d40/d82/l8e 0 2024-04-20T10:59:17.604 INFO:tasks.workunit.client.0.smithi027.stdout:1/533: dread d3/d7/f1f [0,4194304] 0 2024-04-20T10:59:17.605 INFO:tasks.workunit.client.0.smithi027.stdout:0/443: rename d0/d11/d2c/d2e/d2b/d7b/d34/l88 to d0/d11/d2c/d2e/d35/l97 0 2024-04-20T10:59:17.606 INFO:tasks.workunit.client.0.smithi027.stdout:0/444: dread - d0/d11/d2c/d2e/d2b/d3d/f84 zero size 2024-04-20T10:59:17.606 INFO:tasks.workunit.client.0.smithi027.stdout:5/431: creat d19/d1b/d1c/d1e/d25/d40/d82/f8f x:0 0 0 2024-04-20T10:59:17.608 INFO:tasks.workunit.client.0.smithi027.stdout:0/445: creat d0/d11/d2c/d2e/d35/d8c/f98 x:0 0 0 2024-04-20T10:59:17.609 INFO:tasks.workunit.client.0.smithi027.stdout:5/432: creat d19/d1b/d1c/d1e/d22/d68/f90 x:0 0 0 2024-04-20T10:59:17.609 INFO:tasks.workunit.client.0.smithi027.stdout:5/433: write d19/d1b/d1c/d1e/d22/d68/d75/f77 [144849,13910] 0 2024-04-20T10:59:17.610 INFO:tasks.workunit.client.0.smithi027.stdout:0/446: symlink d0/d11/d62/l99 0 2024-04-20T10:59:17.611 INFO:tasks.workunit.client.0.smithi027.stdout:0/447: readlink d0/d11/d2c/d2e/d2b/l91 0 2024-04-20T10:59:17.611 INFO:tasks.workunit.client.0.smithi027.stdout:0/448: chown d0/d11/d2c/d2e/d2b/d3d/f84 32939198 1 2024-04-20T10:59:17.612 INFO:tasks.workunit.client.0.smithi027.stdout:7/398: read d4/d5/f40 [1127526,100997] 0 2024-04-20T10:59:17.614 INFO:tasks.workunit.client.0.smithi027.stdout:7/399: mkdir d4/d1b/d4a/d7a 0 2024-04-20T10:59:17.615 INFO:tasks.workunit.client.0.smithi027.stdout:7/400: truncate d4/d1b/d4a/f44 1202102 0 2024-04-20T10:59:17.616 INFO:tasks.workunit.client.0.smithi027.stdout:0/449: dread d0/d11/d5c/f74 [0,4194304] 0 2024-04-20T10:59:17.623 INFO:tasks.workunit.client.0.smithi027.stdout:1/534: dwrite d3/f1c [0,4194304] 0 2024-04-20T10:59:17.623 INFO:tasks.workunit.client.0.smithi027.stdout:7/401: truncate d4/d5/d8/d21/f48 256911 0 2024-04-20T10:59:17.624 INFO:tasks.workunit.client.0.smithi027.stdout:5/434: getdents d19/d1b 0 2024-04-20T10:59:17.627 INFO:tasks.workunit.client.0.smithi027.stdout:0/450: write d0/d11/d2c/d2e/d55/f56 [2880370,63250] 0 2024-04-20T10:59:17.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/535: mknod d3/d45/d5d/cc7 0 2024-04-20T10:59:17.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/536: getdents d3/d45/dba 0 2024-04-20T10:59:17.628 INFO:tasks.workunit.client.0.smithi027.stdout:7/402: rename d4/d5/l5e to d4/d1b/d4a/d62/l7b 0 2024-04-20T10:59:17.630 INFO:tasks.workunit.client.0.smithi027.stdout:0/451: creat d0/d11/f9a x:0 0 0 2024-04-20T10:59:17.632 INFO:tasks.workunit.client.0.smithi027.stdout:1/537: rename d3/d7/d38/fa9 to d3/d45/fc8 0 2024-04-20T10:59:17.633 INFO:tasks.workunit.client.0.smithi027.stdout:0/452: dread d0/d11/d2c/f52 [0,4194304] 0 2024-04-20T10:59:17.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/538: rename d3/d41/c75 to d3/d7e/cc9 0 2024-04-20T10:59:17.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/539: readlink d3/lc3 0 2024-04-20T10:59:17.637 INFO:tasks.workunit.client.0.smithi027.stdout:0/453: truncate d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/f8a 617335 0 2024-04-20T10:59:17.637 INFO:tasks.workunit.client.0.smithi027.stdout:1/540: link d3/d7/d38/c7c d3/cca 0 2024-04-20T10:59:17.637 INFO:tasks.workunit.client.0.smithi027.stdout:1/541: mknod d3/d7/d28/ccb 0 2024-04-20T10:59:17.638 INFO:tasks.workunit.client.0.smithi027.stdout:0/454: symlink d0/d11/d2c/d2e/d2b/d7b/d12/d25/l9b 0 2024-04-20T10:59:17.638 INFO:tasks.workunit.client.0.smithi027.stdout:0/455: write d0/d11/d5c/f74 [5342265,22820] 0 2024-04-20T10:59:17.639 INFO:tasks.workunit.client.0.smithi027.stdout:0/456: dread d0/d11/d2c/f52 [0,4194304] 0 2024-04-20T10:59:17.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/435: dwrite d19/d1b/f8d [0,4194304] 0 2024-04-20T10:59:17.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/436: rmdir d19/d1b/d1c/d1e/d83 39 2024-04-20T10:59:17.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/437: fsync d19/d1b/d1c/f3b 0 2024-04-20T10:59:17.660 INFO:tasks.workunit.client.0.smithi027.stdout:5/438: mkdir d19/d1b/d1c/d1e/d25/d40/d78/d91 0 2024-04-20T10:59:17.661 INFO:tasks.workunit.client.0.smithi027.stdout:2/367: dwrite d5/d26/f2c [0,4194304] 0 2024-04-20T10:59:17.661 INFO:tasks.workunit.client.0.smithi027.stdout:7/403: dwrite d4/d5/f40 [0,4194304] 0 2024-04-20T10:59:17.661 INFO:tasks.workunit.client.0.smithi027.stdout:2/368: write d5/d1e/d21/f33 [892560,116901] 0 2024-04-20T10:59:17.661 INFO:tasks.workunit.client.0.smithi027.stdout:6/397: dwrite da/d21/d28/f34 [0,4194304] 0 2024-04-20T10:59:17.662 INFO:tasks.workunit.client.0.smithi027.stdout:1/542: dwrite d3/d7/d28/d92/fb5 [0,4194304] 0 2024-04-20T10:59:17.664 INFO:tasks.workunit.client.0.smithi027.stdout:0/457: dwrite d0/d11/d2c/d2e/d2b/d3d/f4c [0,4194304] 0 2024-04-20T10:59:17.666 INFO:tasks.workunit.client.0.smithi027.stdout:7/404: dread d4/d1b/ff [0,4194304] 0 2024-04-20T10:59:17.667 INFO:tasks.workunit.client.0.smithi027.stdout:7/405: chown d4/d5/d8/d21/f6e 511 1 2024-04-20T10:59:17.667 INFO:tasks.workunit.client.0.smithi027.stdout:2/369: symlink d5/d26/d30/l7a 0 2024-04-20T10:59:17.667 INFO:tasks.workunit.client.0.smithi027.stdout:6/398: symlink da/dd/d79/l82 0 2024-04-20T10:59:17.669 INFO:tasks.workunit.client.0.smithi027.stdout:2/370: dread d5/dc/d16/d2a/d4a/f67 [0,4194304] 0 2024-04-20T10:59:17.676 INFO:tasks.workunit.client.0.smithi027.stdout:1/543: rmdir d3/d7/d28/d92/dad 39 2024-04-20T10:59:17.677 INFO:tasks.workunit.client.0.smithi027.stdout:0/458: symlink d0/d11/d2c/d2e/d55/l9c 0 2024-04-20T10:59:17.678 INFO:tasks.workunit.client.0.smithi027.stdout:7/406: mkdir d4/d5/d8/d46/d77/d7c 0 2024-04-20T10:59:17.679 INFO:tasks.workunit.client.0.smithi027.stdout:2/371: rmdir d5/d26/d78 39 2024-04-20T10:59:17.680 INFO:tasks.workunit.client.0.smithi027.stdout:0/459: rename d0/d11/d2c/d2e/d61/f70 to d0/d11/d2c/f9d 0 2024-04-20T10:59:17.680 INFO:tasks.workunit.client.0.smithi027.stdout:0/460: read d0/d11/d2c/d2e/f1d [594203,3538] 0 2024-04-20T10:59:17.680 INFO:tasks.workunit.client.0.smithi027.stdout:0/461: read - d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f86 zero size 2024-04-20T10:59:17.680 INFO:tasks.workunit.client.0.smithi027.stdout:0/462: write d0/d59/f6a [442455,36034] 0 2024-04-20T10:59:17.681 INFO:tasks.workunit.client.0.smithi027.stdout:7/407: symlink d4/d5/d8/d45/l7d 0 2024-04-20T10:59:17.683 INFO:tasks.workunit.client.0.smithi027.stdout:6/399: dwrite da/d21/d28/f34 [4194304,4194304] 0 2024-04-20T10:59:17.685 INFO:tasks.workunit.client.0.smithi027.stdout:0/463: mkdir d0/d59/d9e 0 2024-04-20T10:59:17.685 INFO:tasks.workunit.client.0.smithi027.stdout:9/370: sync 2024-04-20T10:59:17.685 INFO:tasks.workunit.client.0.smithi027.stdout:9/371: dread - d4/d1b/d31/d23/d29/f70 zero size 2024-04-20T10:59:17.687 INFO:tasks.workunit.client.0.smithi027.stdout:0/464: symlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/l9f 0 2024-04-20T10:59:17.687 INFO:tasks.workunit.client.0.smithi027.stdout:0/465: fsync d0/d11/d2c/d2e/f2a 0 2024-04-20T10:59:17.687 INFO:tasks.workunit.client.0.smithi027.stdout:0/466: chown d0/d11/d2c/d2e 30 1 2024-04-20T10:59:17.687 INFO:tasks.workunit.client.0.smithi027.stdout:6/400: link c5 da/df/d19/d2e/c83 0 2024-04-20T10:59:17.688 INFO:tasks.workunit.client.0.smithi027.stdout:0/467: creat d0/d11/d4b/d5b/d6f/d82/fa0 x:0 0 0 2024-04-20T10:59:17.689 INFO:tasks.workunit.client.0.smithi027.stdout:9/372: rename d4/d1b/d50/d5d/f63 to d4/d2a/f77 0 2024-04-20T10:59:17.689 INFO:tasks.workunit.client.0.smithi027.stdout:9/373: chown d4/f7 743264 1 2024-04-20T10:59:17.689 INFO:tasks.workunit.client.0.smithi027.stdout:9/374: readlink d4/d1b/d1f/l65 0 2024-04-20T10:59:17.690 INFO:tasks.workunit.client.0.smithi027.stdout:6/401: getdents da/dd/d79 0 2024-04-20T10:59:17.690 INFO:tasks.workunit.client.0.smithi027.stdout:6/402: stat c5 0 2024-04-20T10:59:17.690 INFO:tasks.workunit.client.0.smithi027.stdout:2/372: dread d5/dc/d16/d2a/d4a/f50 [0,4194304] 0 2024-04-20T10:59:17.692 INFO:tasks.workunit.client.0.smithi027.stdout:2/373: link d5/d1e/d21/f3e d5/d26/d78/f7b 0 2024-04-20T10:59:17.692 INFO:tasks.workunit.client.0.smithi027.stdout:2/374: readlink d5/dc/l45 0 2024-04-20T10:59:17.693 INFO:tasks.workunit.client.0.smithi027.stdout:2/375: readlink d5/d26/d47/l6d 0 2024-04-20T10:59:17.693 INFO:tasks.workunit.client.0.smithi027.stdout:2/376: write d5/dc/f41 [393074,114019] 0 2024-04-20T10:59:17.693 INFO:tasks.workunit.client.0.smithi027.stdout:1/544: dwrite d3/d7/d38/da4/fc1 [0,4194304] 0 2024-04-20T10:59:17.693 INFO:tasks.workunit.client.0.smithi027.stdout:1/545: fdatasync d3/d7/f57 0 2024-04-20T10:59:17.702 INFO:tasks.workunit.client.0.smithi027.stdout:2/377: write d5/d26/f74 [1955738,124366] 0 2024-04-20T10:59:17.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/378: getdents d5/d1e 0 2024-04-20T10:59:17.704 INFO:tasks.workunit.client.0.smithi027.stdout:7/408: dwrite d4/fc [0,4194304] 0 2024-04-20T10:59:17.704 INFO:tasks.workunit.client.0.smithi027.stdout:7/409: readlink d4/d5/d8/d3b/l76 0 2024-04-20T10:59:17.704 INFO:tasks.workunit.client.0.smithi027.stdout:7/410: chown d4/d1b/d4a 694166 1 2024-04-20T10:59:17.704 INFO:tasks.workunit.client.0.smithi027.stdout:2/379: rmdir d5/dc/d16/d2a/d59 39 2024-04-20T10:59:17.706 INFO:tasks.workunit.client.0.smithi027.stdout:2/380: mkdir d5/d44/d7c 0 2024-04-20T10:59:17.706 INFO:tasks.workunit.client.0.smithi027.stdout:2/381: fdatasync d5/dc/f1c 0 2024-04-20T10:59:17.706 INFO:tasks.workunit.client.0.smithi027.stdout:2/382: readlink d5/d26/d47/l6d 0 2024-04-20T10:59:17.713 INFO:tasks.workunit.client.0.smithi027.stdout:2/383: write f1 [1551137,86994] 0 2024-04-20T10:59:17.714 INFO:tasks.workunit.client.0.smithi027.stdout:2/384: readlink d5/dc/d16/d2a/d4a/l63 0 2024-04-20T10:59:17.714 INFO:tasks.workunit.client.0.smithi027.stdout:2/385: chown d5/d26/d78/f7b 1115075 1 2024-04-20T10:59:17.720 INFO:tasks.workunit.client.0.smithi027.stdout:6/403: dwrite da/df/d19/d2e/d32/f75 [0,4194304] 0 2024-04-20T10:59:17.721 INFO:tasks.workunit.client.0.smithi027.stdout:6/404: creat da/f84 x:0 0 0 2024-04-20T10:59:17.722 INFO:tasks.workunit.client.0.smithi027.stdout:6/405: read - da/df/f29 zero size 2024-04-20T10:59:17.723 INFO:tasks.workunit.client.0.smithi027.stdout:6/406: creat da/df/d7e/f85 x:0 0 0 2024-04-20T10:59:17.723 INFO:tasks.workunit.client.0.smithi027.stdout:6/407: chown da/df/d19/d2e/l57 58333 1 2024-04-20T10:59:17.723 INFO:tasks.workunit.client.0.smithi027.stdout:6/408: chown da/df/d7e 95521 1 2024-04-20T10:59:17.724 INFO:tasks.workunit.client.0.smithi027.stdout:6/409: rename da/d21/d28/d2d/d51 to da/df/d7e/d86 0 2024-04-20T10:59:17.724 INFO:tasks.workunit.client.0.smithi027.stdout:0/468: dwrite d0/d11/d2c/d2e/f1d [0,4194304] 0 2024-04-20T10:59:17.724 INFO:tasks.workunit.client.0.smithi027.stdout:0/469: write d0/d11/d2c/d2e/d35/d8c/f98 [537099,14697] 0 2024-04-20T10:59:17.725 INFO:tasks.workunit.client.0.smithi027.stdout:0/470: mkdir d0/d11/d2c/da1 0 2024-04-20T10:59:17.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/471: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/fa2 x:0 0 0 2024-04-20T10:59:17.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/472: truncate d0/d11/d2c/d2e/f2a 1014934 0 2024-04-20T10:59:17.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/473: chown d0/d11/d5c/f74 25210 1 2024-04-20T10:59:17.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/474: write d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f86 [934184,22770] 0 2024-04-20T10:59:17.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/475: chown d0/d11/d2c/d2e/d2b/d7b/d12/c4f 17918 1 2024-04-20T10:59:17.728 INFO:tasks.workunit.client.0.smithi027.stdout:1/546: dwrite d3/d41/f4e [0,4194304] 0 2024-04-20T10:59:17.728 INFO:tasks.workunit.client.0.smithi027.stdout:0/476: readlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/l9f 0 2024-04-20T10:59:17.728 INFO:tasks.workunit.client.0.smithi027.stdout:0/477: creat d0/d11/d4b/fa3 x:0 0 0 2024-04-20T10:59:17.729 INFO:tasks.workunit.client.0.smithi027.stdout:6/410: write da/df/d19/d2e/d32/f44 [1990465,23173] 0 2024-04-20T10:59:17.729 INFO:tasks.workunit.client.0.smithi027.stdout:6/411: chown da/l80 14498403 1 2024-04-20T10:59:17.735 INFO:tasks.workunit.client.0.smithi027.stdout:1/547: rename d3/d7/d38/d9c to d3/d7/d2d/dcc 0 2024-04-20T10:59:17.735 INFO:tasks.workunit.client.0.smithi027.stdout:0/478: unlink d0/d11/d2c/d2e/d2b/f6e 0 2024-04-20T10:59:17.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/479: rmdir d0/d11/d5c 39 2024-04-20T10:59:17.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/480: dread - d0/d11/d5c/f63 zero size 2024-04-20T10:59:17.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/481: creat d0/d11/d2c/d2e/d2b/d7b/d34/fa4 x:0 0 0 2024-04-20T10:59:17.738 INFO:tasks.workunit.client.0.smithi027.stdout:0/482: link d0/d11/d2c/d2e/f1d d0/d11/d2c/d2e/d2b/d7b/d12/d1a/fa5 0 2024-04-20T10:59:17.738 INFO:tasks.workunit.client.0.smithi027.stdout:0/483: chown d0/d11/d2c/d2e/d61 25 1 2024-04-20T10:59:17.739 INFO:tasks.workunit.client.0.smithi027.stdout:6/412: dread da/df/d19/d2e/d6b/f62 [0,4194304] 0 2024-04-20T10:59:17.739 INFO:tasks.workunit.client.0.smithi027.stdout:6/413: chown da/dd/l50 977059287 1 2024-04-20T10:59:17.739 INFO:tasks.workunit.client.0.smithi027.stdout:1/548: dread d3/d7/d28/f8b [0,4194304] 0 2024-04-20T10:59:17.739 INFO:tasks.workunit.client.0.smithi027.stdout:0/484: truncate d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f2d 1647552 0 2024-04-20T10:59:17.740 INFO:tasks.workunit.client.0.smithi027.stdout:6/414: getdents da/d21/d28/d54 0 2024-04-20T10:59:17.740 INFO:tasks.workunit.client.0.smithi027.stdout:6/415: fdatasync da/f2a 0 2024-04-20T10:59:17.741 INFO:tasks.workunit.client.0.smithi027.stdout:6/416: truncate da/df/d19/d2e/d6b/f62 709914 0 2024-04-20T10:59:17.741 INFO:tasks.workunit.client.0.smithi027.stdout:0/485: dread d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f50 [0,4194304] 0 2024-04-20T10:59:17.742 INFO:tasks.workunit.client.0.smithi027.stdout:6/417: creat da/dd/d79/f87 x:0 0 0 2024-04-20T10:59:17.743 INFO:tasks.workunit.client.0.smithi027.stdout:0/486: truncate d0/f20 6523414 0 2024-04-20T10:59:17.743 INFO:tasks.workunit.client.0.smithi027.stdout:0/487: readlink d0/d11/d62/l99 0 2024-04-20T10:59:17.743 INFO:tasks.workunit.client.0.smithi027.stdout:0/488: mkdir d0/d11/d2c/da1/da6 0 2024-04-20T10:59:17.744 INFO:tasks.workunit.client.0.smithi027.stdout:6/418: mknod da/c88 0 2024-04-20T10:59:17.744 INFO:tasks.workunit.client.0.smithi027.stdout:0/489: rmdir d0/d11/d2c/d2e/d2b/d3d 39 2024-04-20T10:59:17.745 INFO:tasks.workunit.client.0.smithi027.stdout:6/419: creat da/df/d19/d2e/f89 x:0 0 0 2024-04-20T10:59:17.745 INFO:tasks.workunit.client.0.smithi027.stdout:6/420: fdatasync da/d21/f36 0 2024-04-20T10:59:17.749 INFO:tasks.workunit.client.0.smithi027.stdout:1/549: dread d3/d7/d38/db8/fbb [0,4194304] 0 2024-04-20T10:59:17.749 INFO:tasks.workunit.client.0.smithi027.stdout:2/386: dwrite f1 [0,4194304] 0 2024-04-20T10:59:17.750 INFO:tasks.workunit.client.0.smithi027.stdout:0/490: creat d0/d11/d2c/d2e/d35/d8c/fa7 x:0 0 0 2024-04-20T10:59:17.752 INFO:tasks.workunit.client.0.smithi027.stdout:6/421: link da/df/d7e/d86/c52 da/dd/d3c/c8a 0 2024-04-20T10:59:17.752 INFO:tasks.workunit.client.0.smithi027.stdout:6/422: chown da/l6c 5 1 2024-04-20T10:59:17.752 INFO:tasks.workunit.client.0.smithi027.stdout:6/423: chown da/d21/f5b 1802360541 1 2024-04-20T10:59:17.753 INFO:tasks.workunit.client.0.smithi027.stdout:1/550: link d3/cca d3/d7/d28/db0/ccd 0 2024-04-20T10:59:17.754 INFO:tasks.workunit.client.0.smithi027.stdout:2/387: mknod d5/d26/d78/c7d 0 2024-04-20T10:59:17.755 INFO:tasks.workunit.client.0.smithi027.stdout:2/388: stat d5/d26/l53 0 2024-04-20T10:59:17.755 INFO:tasks.workunit.client.0.smithi027.stdout:0/491: mknod d0/d11/d2c/d2e/d2b/d7b/d12/ca8 0 2024-04-20T10:59:17.757 INFO:tasks.workunit.client.0.smithi027.stdout:6/424: symlink da/df/d19/d2e/d6b/l8b 0 2024-04-20T10:59:17.757 INFO:tasks.workunit.client.0.smithi027.stdout:6/425: fsync da/d21/d28/d2d/d53/f5e 0 2024-04-20T10:59:17.759 INFO:tasks.workunit.client.0.smithi027.stdout:2/389: creat d5/d44/d7c/f7e x:0 0 0 2024-04-20T10:59:17.759 INFO:tasks.workunit.client.0.smithi027.stdout:2/390: chown d5/d26/d30 35 1 2024-04-20T10:59:17.759 INFO:tasks.workunit.client.0.smithi027.stdout:2/391: read - d5/dc/d16/d37/f4e zero size 2024-04-20T10:59:17.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/492: creat d0/d11/fa9 x:0 0 0 2024-04-20T10:59:17.761 INFO:tasks.workunit.client.0.smithi027.stdout:6/426: creat da/dd/d3d/f8c x:0 0 0 2024-04-20T10:59:17.762 INFO:tasks.workunit.client.0.smithi027.stdout:2/392: link d5/d26/d30/l7a d5/d26/d78/l7f 0 2024-04-20T10:59:17.762 INFO:tasks.workunit.client.0.smithi027.stdout:2/393: getdents d5/d1e 0 2024-04-20T10:59:17.763 INFO:tasks.workunit.client.0.smithi027.stdout:2/394: symlink d5/d26/d3d/l80 0 2024-04-20T10:59:17.763 INFO:tasks.workunit.client.0.smithi027.stdout:2/395: fdatasync d5/d1e/d21/f3e 0 2024-04-20T10:59:17.763 INFO:tasks.workunit.client.0.smithi027.stdout:2/396: chown d5/dc/f1c 21375859 1 2024-04-20T10:59:17.771 INFO:tasks.workunit.client.0.smithi027.stdout:2/397: dread d5/d44/f65 [0,4194304] 0 2024-04-20T10:59:17.771 INFO:tasks.workunit.client.0.smithi027.stdout:4/414: sync 2024-04-20T10:59:17.771 INFO:tasks.workunit.client.0.smithi027.stdout:3/465: sync 2024-04-20T10:59:17.771 INFO:tasks.workunit.client.0.smithi027.stdout:8/514: sync 2024-04-20T10:59:17.771 INFO:tasks.workunit.client.0.smithi027.stdout:8/515: getdents d9/de/d20/d4f/d97/da5 0 2024-04-20T10:59:17.772 INFO:tasks.workunit.client.0.smithi027.stdout:2/398: rename d5/dc/d16/d37/f6e to d5/dc/d16/d2a/d59/f81 0 2024-04-20T10:59:17.775 INFO:tasks.workunit.client.0.smithi027.stdout:0/493: write d0/d11/f54 [1673846,1600] 0 2024-04-20T10:59:17.778 INFO:tasks.workunit.client.0.smithi027.stdout:3/466: mkdir d2/dd/d57/d8d 0 2024-04-20T10:59:17.778 INFO:tasks.workunit.client.0.smithi027.stdout:8/516: link d9/de/d1e/d4a/f8f d9/fb5 0 2024-04-20T10:59:17.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/415: mknod d0/d14/d71/ca2 0 2024-04-20T10:59:17.781 INFO:tasks.workunit.client.0.smithi027.stdout:0/494: symlink d0/d11/d2c/d2e/d55/laa 0 2024-04-20T10:59:17.781 INFO:tasks.workunit.client.0.smithi027.stdout:3/467: mknod d2/d18/d2c/c8e 0 2024-04-20T10:59:17.781 INFO:tasks.workunit.client.0.smithi027.stdout:8/517: rename d9/de/d20/d4f to d9/de/d64/d6c/db6 0 2024-04-20T10:59:17.781 INFO:tasks.workunit.client.0.smithi027.stdout:8/518: fsync d9/f24 0 2024-04-20T10:59:17.781 INFO:tasks.workunit.client.0.smithi027.stdout:3/468: rmdir d2/d18/d2c 39 2024-04-20T10:59:17.782 INFO:tasks.workunit.client.0.smithi027.stdout:0/495: rmdir d0/d11/d2c/d2e/d55 39 2024-04-20T10:59:17.782 INFO:tasks.workunit.client.0.smithi027.stdout:0/496: readlink d0/d11/d2c/d2e/d2b/d7b/l80 0 2024-04-20T10:59:17.782 INFO:tasks.workunit.client.0.smithi027.stdout:0/497: write d0/d11/d2c/d2e/d35/d8c/f98 [1364322,98216] 0 2024-04-20T10:59:17.782 INFO:tasks.workunit.client.0.smithi027.stdout:6/427: dwrite da/df/f11 [0,4194304] 0 2024-04-20T10:59:17.783 INFO:tasks.workunit.client.0.smithi027.stdout:4/416: rename d0/d14/d17 to d0/d14/d6b/da3 0 2024-04-20T10:59:17.783 INFO:tasks.workunit.client.0.smithi027.stdout:8/519: read d9/de/f3a [106215,110818] 0 2024-04-20T10:59:17.784 INFO:tasks.workunit.client.0.smithi027.stdout:3/469: creat d2/dd/d57/d8c/f8f x:0 0 0 2024-04-20T10:59:17.784 INFO:tasks.workunit.client.0.smithi027.stdout:0/498: mknod d0/d11/d2c/cab 0 2024-04-20T10:59:17.785 INFO:tasks.workunit.client.0.smithi027.stdout:4/417: creat d0/d14/d6b/da3/d1a/fa4 x:0 0 0 2024-04-20T10:59:17.786 INFO:tasks.workunit.client.0.smithi027.stdout:4/418: stat d0/d14/d6b 0 2024-04-20T10:59:17.786 INFO:tasks.workunit.client.0.smithi027.stdout:3/470: creat d2/dd/d57/d8c/f90 x:0 0 0 2024-04-20T10:59:17.788 INFO:tasks.workunit.client.0.smithi027.stdout:0/499: rename d0/d11/d2c/d2e/d2b/d7b/fa to d0/d11/d2c/d2e/d55/fac 0 2024-04-20T10:59:17.789 INFO:tasks.workunit.client.0.smithi027.stdout:0/500: creat d0/d11/d2c/d2e/d2b/d7b/d12/fad x:0 0 0 2024-04-20T10:59:17.789 INFO:tasks.workunit.client.0.smithi027.stdout:0/501: truncate d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f86 1088075 0 2024-04-20T10:59:17.790 INFO:tasks.workunit.client.0.smithi027.stdout:0/502: unlink d0/d11/d2c/d2e/d2b/d7b/c60 0 2024-04-20T10:59:17.790 INFO:tasks.workunit.client.0.smithi027.stdout:8/520: dread d9/fb5 [0,4194304] 0 2024-04-20T10:59:17.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/503: write d0/d11/d2c/d2e/d35/d8c/fa7 [142171,16363] 0 2024-04-20T10:59:17.791 INFO:tasks.workunit.client.0.smithi027.stdout:8/521: write d9/de/d20/d48/d59/f60 [5011328,106152] 0 2024-04-20T10:59:17.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/504: symlink d0/d11/d2c/d2e/d2b/d3d/lae 0 2024-04-20T10:59:17.792 INFO:tasks.workunit.client.0.smithi027.stdout:0/505: write d0/d11/d2c/d2e/d2b/d7b/d64/f93 [928742,26365] 0 2024-04-20T10:59:17.792 INFO:tasks.workunit.client.0.smithi027.stdout:0/506: chown d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f 20376 1 2024-04-20T10:59:17.792 INFO:tasks.workunit.client.0.smithi027.stdout:0/507: fdatasync d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/f8a 0 2024-04-20T10:59:17.793 INFO:tasks.workunit.client.0.smithi027.stdout:6/428: write da/d21/d28/f34 [1440608,95787] 0 2024-04-20T10:59:17.793 INFO:tasks.workunit.client.0.smithi027.stdout:6/429: readlink da/df/d19/d2e/l57 0 2024-04-20T10:59:17.794 INFO:tasks.workunit.client.0.smithi027.stdout:6/430: rmdir da/d21 39 2024-04-20T10:59:17.794 INFO:tasks.workunit.client.0.smithi027.stdout:6/431: fdatasync da/d21/d28/d2d/d53/f6f 0 2024-04-20T10:59:17.794 INFO:tasks.workunit.client.0.smithi027.stdout:6/432: truncate da/df/d7e/f81 4924329 0 2024-04-20T10:59:17.794 INFO:tasks.workunit.client.0.smithi027.stdout:2/399: dwrite d5/dc/d16/d2a/d59/f81 [0,4194304] 0 2024-04-20T10:59:17.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/433: write da/fb [840351,16535] 0 2024-04-20T10:59:17.797 INFO:tasks.workunit.client.0.smithi027.stdout:4/419: dread d0/f13 [0,4194304] 0 2024-04-20T10:59:17.800 INFO:tasks.workunit.client.0.smithi027.stdout:6/434: mknod da/d21/d28/d54/c8d 0 2024-04-20T10:59:17.801 INFO:tasks.workunit.client.0.smithi027.stdout:3/471: dread d2/d18/d2c/d4c/f7e [0,4194304] 0 2024-04-20T10:59:17.805 INFO:tasks.workunit.client.0.smithi027.stdout:8/522: rename d9/de/d64/d6c/db6 to d9/de/d1e/d4a/d6d/db7 0 2024-04-20T10:59:17.806 INFO:tasks.workunit.client.0.smithi027.stdout:6/435: mknod da/df/d19/d2e/d6b/c8e 0 2024-04-20T10:59:17.806 INFO:tasks.workunit.client.0.smithi027.stdout:6/436: readlink da/dd/d3d/l68 0 2024-04-20T10:59:17.807 INFO:tasks.workunit.client.0.smithi027.stdout:3/472: mknod d2/dd/df/d23/c91 0 2024-04-20T10:59:17.809 INFO:tasks.workunit.client.0.smithi027.stdout:1/551: dwrite d3/d7/d38/f65 [0,4194304] 0 2024-04-20T10:59:17.809 INFO:tasks.workunit.client.0.smithi027.stdout:1/552: write d3/d45/d5d/d6c/f90 [3058860,63375] 0 2024-04-20T10:59:17.809 INFO:tasks.workunit.client.0.smithi027.stdout:6/437: dread da/df/d19/d2e/d32/f44 [0,4194304] 0 2024-04-20T10:59:17.809 INFO:tasks.workunit.client.0.smithi027.stdout:0/508: rename d0/d11/d2c/d2e/l37 to d0/d11/d5c/laf 0 2024-04-20T10:59:17.810 INFO:tasks.workunit.client.0.smithi027.stdout:8/523: chown d9/de/f71 187 1 2024-04-20T10:59:17.811 INFO:tasks.workunit.client.0.smithi027.stdout:3/473: symlink d2/dd/d57/l92 0 2024-04-20T10:59:17.813 INFO:tasks.workunit.client.0.smithi027.stdout:6/438: read da/df/f10 [1555812,23156] 0 2024-04-20T10:59:17.814 INFO:tasks.workunit.client.0.smithi027.stdout:6/439: dread da/d21/f66 [0,4194304] 0 2024-04-20T10:59:17.814 INFO:tasks.workunit.client.0.smithi027.stdout:6/440: fsync da/df/d19/d2e/f89 0 2024-04-20T10:59:17.816 INFO:tasks.workunit.client.0.smithi027.stdout:1/553: symlink d3/d7/d28/db0/lce 0 2024-04-20T10:59:17.817 INFO:tasks.workunit.client.0.smithi027.stdout:4/420: dwrite d0/d14/d6b/da3/f1e [0,4194304] 0 2024-04-20T10:59:17.817 INFO:tasks.workunit.client.0.smithi027.stdout:0/509: mknod d0/d11/d2c/d2e/d2b/d7b/d12/d1a/cb0 0 2024-04-20T10:59:17.817 INFO:tasks.workunit.client.0.smithi027.stdout:0/510: fdatasync d0/d11/d2c/d2e/d2b/d7b/f27 0 2024-04-20T10:59:17.818 INFO:tasks.workunit.client.0.smithi027.stdout:0/511: truncate d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f86 2122422 0 2024-04-20T10:59:17.818 INFO:tasks.workunit.client.0.smithi027.stdout:8/524: write d9/de/da3/d3c/f6f [1579712,109399] 0 2024-04-20T10:59:17.824 INFO:tasks.workunit.client.0.smithi027.stdout:3/474: mknod d2/d18/c93 0 2024-04-20T10:59:17.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/475: read d2/d18/d2e/f75 [211585,100358] 0 2024-04-20T10:59:17.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/476: chown d2/d80/l5c 7 1 2024-04-20T10:59:17.827 INFO:tasks.workunit.client.0.smithi027.stdout:6/441: dread da/f3b [0,4194304] 0 2024-04-20T10:59:17.827 INFO:tasks.workunit.client.0.smithi027.stdout:6/442: chown da/df/d19/d2e/d32/l7a 5 1 2024-04-20T10:59:17.827 INFO:tasks.workunit.client.0.smithi027.stdout:6/443: dread - da/dd/d79/f87 zero size 2024-04-20T10:59:17.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/444: read da/f2a [4609348,97493] 0 2024-04-20T10:59:17.837 INFO:tasks.workunit.client.0.smithi027.stdout:1/554: dwrite d3/d7/d38/da4/dac/fb3 [0,4194304] 0 2024-04-20T10:59:17.837 INFO:tasks.workunit.client.0.smithi027.stdout:1/555: chown d3/d45/d5d/d6c 24 1 2024-04-20T10:59:17.841 INFO:tasks.workunit.client.0.smithi027.stdout:6/445: dread da/df/d19/d2e/d6b/f4e [0,4194304] 0 2024-04-20T10:59:17.858 INFO:tasks.workunit.client.0.smithi027.stdout:0/512: symlink d0/d59/d9e/lb1 0 2024-04-20T10:59:17.858 INFO:tasks.workunit.client.0.smithi027.stdout:0/513: readlink d0/l13 0 2024-04-20T10:59:17.859 INFO:tasks.workunit.client.0.smithi027.stdout:1/556: dwrite d3/d7/d28/f48 [0,4194304] 0 2024-04-20T10:59:17.859 INFO:tasks.workunit.client.0.smithi027.stdout:1/557: fdatasync d3/d7/d38/da4/dac/fc0 0 2024-04-20T10:59:17.859 INFO:tasks.workunit.client.0.smithi027.stdout:1/558: fdatasync d3/d7/d2d/f4a 0 2024-04-20T10:59:17.865 INFO:tasks.workunit.client.0.smithi027.stdout:6/446: dwrite da/dd/d3d/f7f [0,4194304] 0 2024-04-20T10:59:17.867 INFO:tasks.workunit.client.0.smithi027.stdout:5/439: sync 2024-04-20T10:59:17.873 INFO:tasks.workunit.client.0.smithi027.stdout:6/447: getdents da/dd/d3d 0 2024-04-20T10:59:17.873 INFO:tasks.workunit.client.0.smithi027.stdout:6/448: dread - da/dd/d3d/f8c zero size 2024-04-20T10:59:17.882 INFO:tasks.workunit.client.0.smithi027.stdout:1/559: dwrite d3/d45/fb6 [0,4194304] 0 2024-04-20T10:59:17.882 INFO:tasks.workunit.client.0.smithi027.stdout:1/560: chown d3/c64 537 1 2024-04-20T10:59:17.883 INFO:tasks.workunit.client.0.smithi027.stdout:5/440: getdents d19/d1b/d1c 0 2024-04-20T10:59:17.885 INFO:tasks.workunit.client.0.smithi027.stdout:1/561: creat d3/d7/d38/da4/fcf x:0 0 0 2024-04-20T10:59:17.885 INFO:tasks.workunit.client.0.smithi027.stdout:6/449: write da/df/d19/d2e/d32/f75 [23483,55748] 0 2024-04-20T10:59:17.886 INFO:tasks.workunit.client.0.smithi027.stdout:5/441: symlink d19/d1b/d1c/d1e/d22/d68/d75/l92 0 2024-04-20T10:59:17.887 INFO:tasks.workunit.client.0.smithi027.stdout:1/562: creat d3/d7/d2d/dcc/fd0 x:0 0 0 2024-04-20T10:59:17.908 INFO:tasks.workunit.client.0.smithi027.stdout:4/421: creat d0/d14/d6b/da3/d1b/fa5 x:0 0 0 2024-04-20T10:59:17.908 INFO:tasks.workunit.client.0.smithi027.stdout:4/422: chown d0/d14/d6b/da3/d18/l66 3577139 1 2024-04-20T10:59:17.908 INFO:tasks.workunit.client.0.smithi027.stdout:8/525: mknod d9/de/d18/cb8 0 2024-04-20T10:59:17.908 INFO:tasks.workunit.client.0.smithi027.stdout:1/563: creat d3/d20/d85/fd1 x:0 0 0 2024-04-20T10:59:17.909 INFO:tasks.workunit.client.0.smithi027.stdout:4/423: symlink d0/d14/d6b/da3/d53/la6 0 2024-04-20T10:59:17.917 INFO:tasks.workunit.client.0.smithi027.stdout:1/564: dread d3/d7/d38/d62/f81 [0,4194304] 0 2024-04-20T10:59:17.921 INFO:tasks.workunit.client.0.smithi027.stdout:4/424: read d0/fb [7511508,99066] 0 2024-04-20T10:59:17.923 INFO:tasks.workunit.client.0.smithi027.stdout:1/565: dread d3/d7/d38/d62/f81 [0,4194304] 0 2024-04-20T10:59:17.923 INFO:tasks.workunit.client.0.smithi027.stdout:1/566: fdatasync d3/d7/f7a 0 2024-04-20T10:59:17.923 INFO:tasks.workunit.client.0.smithi027.stdout:1/567: write d3/d7/d28/d92/fb4 [1079613,35496] 0 2024-04-20T10:59:17.923 INFO:tasks.workunit.client.0.smithi027.stdout:1/568: chown d3/d7/d28/d35/f5e 4 1 2024-04-20T10:59:17.923 INFO:tasks.workunit.client.0.smithi027.stdout:8/526: rename d9/de/d20/d48/d59/d74/f80 to d9/de/d20/d48/d59/d74/fb9 0 2024-04-20T10:59:17.926 INFO:tasks.workunit.client.0.smithi027.stdout:1/569: dread d3/d7/d38/db8/fbb [0,4194304] 0 2024-04-20T10:59:17.926 INFO:tasks.workunit.client.0.smithi027.stdout:1/570: read - d3/d20/faf zero size 2024-04-20T10:59:17.927 INFO:tasks.workunit.client.0.smithi027.stdout:8/527: creat d9/de/d1e/d4a/d6d/d79/fba x:0 0 0 2024-04-20T10:59:17.927 INFO:tasks.workunit.client.0.smithi027.stdout:8/528: write d9/de/d64/d6a/f67 [4340050,109183] 0 2024-04-20T10:59:17.928 INFO:tasks.workunit.client.0.smithi027.stdout:4/425: rename d0/d14/d6b/da3/d1a/d3b/c7a to d0/d5e/ca7 0 2024-04-20T10:59:17.930 INFO:tasks.workunit.client.0.smithi027.stdout:1/571: creat d3/d7e/fd2 x:0 0 0 2024-04-20T10:59:17.932 INFO:tasks.workunit.client.0.smithi027.stdout:8/529: chown d9/de/l10 63751 1 2024-04-20T10:59:17.932 INFO:tasks.workunit.client.0.smithi027.stdout:8/530: stat d9/de/d64/d6a/c7e 0 2024-04-20T10:59:17.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/572: creat d3/d41/dc6/fd3 x:0 0 0 2024-04-20T10:59:17.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/573: truncate d3/d45/fc8 1061588 0 2024-04-20T10:59:17.936 INFO:tasks.workunit.client.0.smithi027.stdout:8/531: dread d9/de/f2b [0,4194304] 0 2024-04-20T10:59:17.937 INFO:tasks.workunit.client.0.smithi027.stdout:1/574: mkdir d3/d7/d38/da4/dac/dd4 0 2024-04-20T10:59:17.939 INFO:tasks.workunit.client.0.smithi027.stdout:8/532: creat d9/de/d1e/d4a/d6d/fbb x:0 0 0 2024-04-20T10:59:17.940 INFO:tasks.workunit.client.0.smithi027.stdout:1/575: creat d3/d45/d5d/fd5 x:0 0 0 2024-04-20T10:59:17.942 INFO:tasks.workunit.client.0.smithi027.stdout:8/533: rename d9/de/d20/f25 to d9/de/d1e/d4a/d6d/d75/d7c/fbc 0 2024-04-20T10:59:17.945 INFO:tasks.workunit.client.0.smithi027.stdout:1/576: mknod d3/d7/d38/db8/cd6 0 2024-04-20T10:59:17.950 INFO:tasks.workunit.client.0.smithi027.stdout:5/442: unlink l12 0 2024-04-20T10:59:17.951 INFO:tasks.workunit.client.0.smithi027.stdout:1/577: symlink d3/d7/d28/db0/ld7 0 2024-04-20T10:59:17.951 INFO:tasks.workunit.client.0.smithi027.stdout:1/578: stat d3/d45/c51 0 2024-04-20T10:59:17.952 INFO:tasks.workunit.client.0.smithi027.stdout:5/443: symlink d19/d1b/d1c/d1e/d25/l93 0 2024-04-20T10:59:17.954 INFO:tasks.workunit.client.0.smithi027.stdout:6/450: creat da/d21/f8f x:0 0 0 2024-04-20T10:59:17.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/451: write da/dd/f69 [1192553,102560] 0 2024-04-20T10:59:17.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/452: chown da/d21/f5b 6572329 1 2024-04-20T10:59:17.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/453: chown da/l5a 7967 1 2024-04-20T10:59:17.955 INFO:tasks.workunit.client.0.smithi027.stdout:1/579: creat d3/d7/d2d/fd8 x:0 0 0 2024-04-20T10:59:17.956 INFO:tasks.workunit.client.0.smithi027.stdout:0/514: mknod d0/d11/d2c/d2e/d2b/d7b/cb2 0 2024-04-20T10:59:17.957 INFO:tasks.workunit.client.0.smithi027.stdout:6/454: creat da/dd/d3d/f90 x:0 0 0 2024-04-20T10:59:17.960 INFO:tasks.workunit.client.0.smithi027.stdout:0/515: creat d0/d11/d2c/d2e/d2b/d3d/fb3 x:0 0 0 2024-04-20T10:59:17.960 INFO:tasks.workunit.client.0.smithi027.stdout:0/516: chown d0/d11/d2c/d2e/d2b/d3d/fb3 5374900 1 2024-04-20T10:59:17.960 INFO:tasks.workunit.client.0.smithi027.stdout:0/517: fdatasync d0/d11/d4b/fa3 0 2024-04-20T10:59:17.975 INFO:tasks.workunit.client.0.smithi027.stdout:0/518: dwrite d0/d11/d5c/f95 [0,4194304] 0 2024-04-20T10:59:17.975 INFO:tasks.workunit.client.0.smithi027.stdout:0/519: chown d0/d11/d2c/d2e/d55/f56 40812530 1 2024-04-20T10:59:17.975 INFO:tasks.workunit.client.0.smithi027.stdout:0/520: fsync d0/d11/fa9 0 2024-04-20T10:59:17.975 INFO:tasks.workunit.client.0.smithi027.stdout:0/521: fsync d0/d11/d2c/f9d 0 2024-04-20T10:59:17.975 INFO:tasks.workunit.client.0.smithi027.stdout:6/455: creat da/dd/f91 x:0 0 0 2024-04-20T10:59:17.976 INFO:tasks.workunit.client.0.smithi027.stdout:6/456: chown da/df/d19/d2e/d32/f44 39010 1 2024-04-20T10:59:17.977 INFO:tasks.workunit.client.0.smithi027.stdout:0/522: unlink d0/d59/d9e/lb1 0 2024-04-20T10:59:17.977 INFO:tasks.workunit.client.0.smithi027.stdout:0/523: write d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f90 [1067436,74302] 0 2024-04-20T10:59:17.977 INFO:tasks.workunit.client.0.smithi027.stdout:0/524: truncate d0/d11/d2c/d2e/d2b/d3d/f84 853088 0 2024-04-20T10:59:17.978 INFO:tasks.workunit.client.0.smithi027.stdout:1/580: dwrite d3/d45/d5d/fd5 [0,4194304] 0 2024-04-20T10:59:17.978 INFO:tasks.workunit.client.0.smithi027.stdout:1/581: dread - d3/d7/d38/d62/f6e zero size 2024-04-20T10:59:17.979 INFO:tasks.workunit.client.0.smithi027.stdout:0/525: rename d0/d11/d2c/d2e/d2b/d7b/d34/fa4 to d0/d11/d2c/d2e/d2b/d7b/d64/fb4 0 2024-04-20T10:59:17.979 INFO:tasks.workunit.client.0.smithi027.stdout:0/526: chown d0/d11/d2c/d2e/d2b/d7b/d34/d92 7 1 2024-04-20T10:59:17.979 INFO:tasks.workunit.client.0.smithi027.stdout:0/527: dread - d0/d11/d5c/f63 zero size 2024-04-20T10:59:17.980 INFO:tasks.workunit.client.0.smithi027.stdout:0/528: readlink d0/d11/d2c/d2e/d2b/d7b/d34/l76 0 2024-04-20T10:59:17.980 INFO:tasks.workunit.client.0.smithi027.stdout:0/529: fdatasync d0/d11/d2c/d2e/d2b/d7b/d34/f43 0 2024-04-20T10:59:17.980 INFO:tasks.workunit.client.0.smithi027.stdout:0/530: dread - d0/d11/d4b/fa3 zero size 2024-04-20T10:59:17.980 INFO:tasks.workunit.client.0.smithi027.stdout:0/531: stat d0/d11/d2c/d2e/d55/fac 0 2024-04-20T10:59:17.980 INFO:tasks.workunit.client.0.smithi027.stdout:1/582: mkdir d3/d7/d2d/dd9 0 2024-04-20T10:59:17.980 INFO:tasks.workunit.client.0.smithi027.stdout:1/583: read - d3/d7/d2d/d73/fc4 zero size 2024-04-20T10:59:17.981 INFO:tasks.workunit.client.0.smithi027.stdout:1/584: creat d3/d7/d38/da4/fda x:0 0 0 2024-04-20T10:59:17.982 INFO:tasks.workunit.client.0.smithi027.stdout:0/532: rename d0/f20 to d0/d11/d2c/d2e/d35/fb5 0 2024-04-20T10:59:17.987 INFO:tasks.workunit.client.0.smithi027.stdout:9/375: sync 2024-04-20T10:59:17.987 INFO:tasks.workunit.client.0.smithi027.stdout:7/411: sync 2024-04-20T10:59:17.987 INFO:tasks.workunit.client.0.smithi027.stdout:6/457: getdents da/dd/d3d 0 2024-04-20T10:59:17.988 INFO:tasks.workunit.client.0.smithi027.stdout:7/412: dread d4/d5/f33 [0,4194304] 0 2024-04-20T10:59:17.988 INFO:tasks.workunit.client.0.smithi027.stdout:1/585: symlink d3/d45/d5d/ldb 0 2024-04-20T10:59:17.990 INFO:tasks.workunit.client.0.smithi027.stdout:9/376: symlink d4/d74/l78 0 2024-04-20T10:59:17.991 INFO:tasks.workunit.client.0.smithi027.stdout:6/458: symlink da/df/d19/d2e/d6b/d65/l92 0 2024-04-20T10:59:17.992 INFO:tasks.workunit.client.0.smithi027.stdout:1/586: unlink d3/d7/d2d/f53 0 2024-04-20T10:59:17.992 INFO:tasks.workunit.client.0.smithi027.stdout:1/587: write d3/f1b [121807,52839] 0 2024-04-20T10:59:17.993 INFO:tasks.workunit.client.0.smithi027.stdout:9/377: rmdir d4/d1b/d31/d23/d29 39 2024-04-20T10:59:17.994 INFO:tasks.workunit.client.0.smithi027.stdout:6/459: creat da/df/d19/f93 x:0 0 0 2024-04-20T10:59:18.003 INFO:tasks.workunit.client.0.smithi027.stdout:0/533: dwrite d0/d11/d2c/d2e/d55/f56 [0,4194304] 0 2024-04-20T10:59:18.003 INFO:tasks.workunit.client.0.smithi027.stdout:0/534: stat d0/d11/d2c/d2e 0 2024-04-20T10:59:18.003 INFO:tasks.workunit.client.0.smithi027.stdout:7/413: dwrite d4/d5/d8/d3b/f74 [0,4194304] 0 2024-04-20T10:59:18.003 INFO:tasks.workunit.client.0.smithi027.stdout:7/414: write d4/d1b/d4a/f68 [118773,127795] 0 2024-04-20T10:59:18.004 INFO:tasks.workunit.client.0.smithi027.stdout:9/378: link d4/d1b/d31/f67 d4/d1b/d1f/d5a/f79 0 2024-04-20T10:59:18.004 INFO:tasks.workunit.client.0.smithi027.stdout:8/534: dread d9/de/d1e/d4a/d6d/d7d/f9f [0,4194304] 0 2024-04-20T10:59:18.006 INFO:tasks.workunit.client.0.smithi027.stdout:1/588: link d3/d45/d5d/fd5 d3/d7e/fdc 0 2024-04-20T10:59:18.006 INFO:tasks.workunit.client.0.smithi027.stdout:1/589: truncate d3/d7/d38/d62/f6e 462715 0 2024-04-20T10:59:18.006 INFO:tasks.workunit.client.0.smithi027.stdout:0/535: rename d0/d11/d2c/d2e/d2b/d7b/d12/d1a/c23 to d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/cb6 0 2024-04-20T10:59:18.008 INFO:tasks.workunit.client.0.smithi027.stdout:9/379: write d4/d1b/d31/d23/d29/f4c [3830337,13103] 0 2024-04-20T10:59:18.009 INFO:tasks.workunit.client.0.smithi027.stdout:9/380: write d4/d2a/f66 [638489,17421] 0 2024-04-20T10:59:18.009 INFO:tasks.workunit.client.0.smithi027.stdout:8/535: truncate f3 3208883 0 2024-04-20T10:59:18.010 INFO:tasks.workunit.client.0.smithi027.stdout:1/590: creat d3/d7/d28/fdd x:0 0 0 2024-04-20T10:59:18.011 INFO:tasks.workunit.client.0.smithi027.stdout:7/415: rename d4/d5/f19 to d4/d5b/f7e 0 2024-04-20T10:59:18.015 INFO:tasks.workunit.client.0.smithi027.stdout:9/381: creat d4/d1b/d31/d23/f7a x:0 0 0 2024-04-20T10:59:18.017 INFO:tasks.workunit.client.0.smithi027.stdout:1/591: creat d3/d7/d2d/fde x:0 0 0 2024-04-20T10:59:18.017 INFO:tasks.workunit.client.0.smithi027.stdout:1/592: dread - d3/d20/f96 zero size 2024-04-20T10:59:18.019 INFO:tasks.workunit.client.0.smithi027.stdout:7/416: symlink d4/d5/l7f 0 2024-04-20T10:59:18.020 INFO:tasks.workunit.client.0.smithi027.stdout:9/382: symlink d4/d14/l7b 0 2024-04-20T10:59:18.020 INFO:tasks.workunit.client.0.smithi027.stdout:9/383: truncate d4/d1b/d31/d23/d29/d4b/f6e 841166 0 2024-04-20T10:59:18.023 INFO:tasks.workunit.client.0.smithi027.stdout:0/536: dwrite d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/fa2 [0,4194304] 0 2024-04-20T10:59:18.026 INFO:tasks.workunit.client.0.smithi027.stdout:9/384: creat d4/d1b/d1f/f7c x:0 0 0 2024-04-20T10:59:18.029 INFO:tasks.workunit.client.0.smithi027.stdout:7/417: dread d4/d5/d8/d3b/f3e [0,4194304] 0 2024-04-20T10:59:18.029 INFO:tasks.workunit.client.0.smithi027.stdout:7/418: chown d4/d5/l2a 2 1 2024-04-20T10:59:18.031 INFO:tasks.workunit.client.0.smithi027.stdout:1/593: rename d3/d7/d2d to d3/d7/d38/ddf 0 2024-04-20T10:59:18.043 INFO:tasks.workunit.client.0.smithi027.stdout:9/385: dwrite d4/d1b/d31/f75 [0,4194304] 0 2024-04-20T10:59:18.044 INFO:tasks.workunit.client.0.smithi027.stdout:0/537: creat d0/d59/d9e/fb7 x:0 0 0 2024-04-20T10:59:18.045 INFO:tasks.workunit.client.0.smithi027.stdout:7/419: mkdir d4/d5/d8/d21/d80 0 2024-04-20T10:59:18.046 INFO:tasks.workunit.client.0.smithi027.stdout:1/594: creat d3/d7/d38/d62/fe0 x:0 0 0 2024-04-20T10:59:18.049 INFO:tasks.workunit.client.0.smithi027.stdout:9/386: dread d4/d14/f38 [0,4194304] 0 2024-04-20T10:59:18.073 INFO:tasks.workunit.client.0.smithi027.stdout:7/420: mkdir d4/d5/d81 0 2024-04-20T10:59:18.088 INFO:tasks.workunit.client.0.smithi027.stdout:7/421: dwrite d4/d5/d8/f4c [0,4194304] 0 2024-04-20T10:59:18.088 INFO:tasks.workunit.client.0.smithi027.stdout:1/595: unlink d3/f1e 0 2024-04-20T10:59:18.090 INFO:tasks.workunit.client.0.smithi027.stdout:9/387: creat d4/d1b/d1f/d45/f7d x:0 0 0 2024-04-20T10:59:18.104 INFO:tasks.workunit.client.0.smithi027.stdout:1/596: dwrite d3/d20/d85/fd1 [0,4194304] 0 2024-04-20T10:59:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:7/422: creat d4/d5/d8/f82 x:0 0 0 2024-04-20T10:59:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:9/388: symlink d4/d14/l7e 0 2024-04-20T10:59:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:9/389: fdatasync d4/d14/f49 0 2024-04-20T10:59:18.107 INFO:tasks.workunit.client.0.smithi027.stdout:9/390: chown d4/f6b 98648641 1 2024-04-20T10:59:18.452 INFO:tasks.workunit.client.0.smithi027.stdout:1/597: symlink d3/d20/d85/le1 0 2024-04-20T10:59:18.453 INFO:tasks.workunit.client.0.smithi027.stdout:7/423: rename d4/d5/d8/c10 to d4/d5/d8/d21/c83 0 2024-04-20T10:59:18.454 INFO:tasks.workunit.client.0.smithi027.stdout:1/598: rmdir d3/d7/d28/d92/dad 39 2024-04-20T10:59:18.655 INFO:tasks.workunit.client.0.smithi027.stdout:7/424: symlink d4/d5/d8/d45/l84 0 2024-04-20T10:59:18.655 INFO:tasks.workunit.client.0.smithi027.stdout:1/599: mknod d3/d7/d28/db0/ce2 0 2024-04-20T10:59:18.664 INFO:tasks.workunit.client.0.smithi027.stdout:3/477: sync 2024-04-20T10:59:18.665 INFO:tasks.workunit.client.0.smithi027.stdout:2/400: sync 2024-04-20T10:59:18.665 INFO:tasks.workunit.client.0.smithi027.stdout:4/426: sync 2024-04-20T10:59:18.665 INFO:tasks.workunit.client.0.smithi027.stdout:3/478: write d2/f25 [1261977,78647] 0 2024-04-20T10:59:18.666 INFO:tasks.workunit.client.0.smithi027.stdout:2/401: rmdir d5/dc/d16/d60 39 2024-04-20T10:59:18.669 INFO:tasks.workunit.client.0.smithi027.stdout:2/402: mknod d5/d44/d7c/c82 0 2024-04-20T10:59:18.671 INFO:tasks.workunit.client.0.smithi027.stdout:7/425: dwrite d4/d1b/d58/f5f [0,4194304] 0 2024-04-20T10:59:18.671 INFO:tasks.workunit.client.0.smithi027.stdout:7/426: stat d4/d1b/d4a/d62/f6a 0 2024-04-20T10:59:18.671 INFO:tasks.workunit.client.0.smithi027.stdout:7/427: truncate d4/d5/d8/d21/f31 2582354 0 2024-04-20T10:59:18.672 INFO:tasks.workunit.client.0.smithi027.stdout:2/403: mknod d5/dc/d16/d2a/c83 0 2024-04-20T10:59:18.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/428: link d4/d4d/f4f d4/d5/d8/f85 0 2024-04-20T10:59:18.676 INFO:tasks.workunit.client.0.smithi027.stdout:7/429: rename d4/d1b/d4a/c69 to d4/d5/d8/d21/d80/c86 0 2024-04-20T10:59:18.677 INFO:tasks.workunit.client.0.smithi027.stdout:2/404: dread d5/d26/d47/f5f [0,4194304] 0 2024-04-20T10:59:18.677 INFO:tasks.workunit.client.0.smithi027.stdout:2/405: truncate d5/dc/d16/d37/f46 189815 0 2024-04-20T10:59:18.679 INFO:tasks.workunit.client.0.smithi027.stdout:2/406: creat d5/dc/d16/d2a/f84 x:0 0 0 2024-04-20T10:59:18.680 INFO:tasks.workunit.client.0.smithi027.stdout:2/407: mknod d5/d1e/d21/c85 0 2024-04-20T10:59:18.681 INFO:tasks.workunit.client.0.smithi027.stdout:2/408: creat d5/f86 x:0 0 0 2024-04-20T10:59:18.681 INFO:tasks.workunit.client.0.smithi027.stdout:2/409: readlink d5/d26/d47/l6d 0 2024-04-20T10:59:18.682 INFO:tasks.workunit.client.0.smithi027.stdout:2/410: fsync d5/d26/d47/f5f 0 2024-04-20T10:59:18.686 INFO:tasks.workunit.client.0.smithi027.stdout:4/427: dwrite d0/d14/d6b/da3/d1b/f21 [4194304,4194304] 0 2024-04-20T10:59:18.687 INFO:tasks.workunit.client.0.smithi027.stdout:3/479: dwrite d2/d18/f29 [0,4194304] 0 2024-04-20T10:59:18.687 INFO:tasks.workunit.client.0.smithi027.stdout:3/480: readlink d2/d7f/l8a 0 2024-04-20T10:59:18.688 INFO:tasks.workunit.client.0.smithi027.stdout:4/428: creat d0/d14/d6b/da3/d18/fa8 x:0 0 0 2024-04-20T10:59:18.693 INFO:tasks.workunit.client.0.smithi027.stdout:3/481: mkdir d2/d94 0 2024-04-20T10:59:18.694 INFO:tasks.workunit.client.0.smithi027.stdout:4/429: creat d0/d14/d6b/da3/d1b/d33/d5c/d6f/fa9 x:0 0 0 2024-04-20T10:59:18.695 INFO:tasks.workunit.client.0.smithi027.stdout:3/482: creat d2/d94/f95 x:0 0 0 2024-04-20T10:59:18.696 INFO:tasks.workunit.client.0.smithi027.stdout:4/430: creat d0/d14/d6b/da3/d53/d6a/faa x:0 0 0 2024-04-20T10:59:18.697 INFO:tasks.workunit.client.0.smithi027.stdout:3/483: mknod d2/dd/c96 0 2024-04-20T10:59:18.698 INFO:tasks.workunit.client.0.smithi027.stdout:4/431: symlink d0/d14/d6b/da3/d53/d6a/da1/lab 0 2024-04-20T10:59:18.700 INFO:tasks.workunit.client.0.smithi027.stdout:4/432: mknod d0/d48/cac 0 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:8/536: sync 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:9/391: sync 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/444: sync 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:6/460: sync 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/445: dread - d19/d1b/d1c/f6a zero size 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/446: write d19/d1b/d1c/d1e/d58/f7b [1047077,19501] 0 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/447: stat d19/d1b/d1c/d1e/d58/f61 0 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/448: chown d19/d1b/d1c/d1e/d22/c7f 22 1 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/449: chown f3 155 1 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:2/411: dwrite d5/d26/d3d/f48 [0,4194304] 0 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:2/412: truncate d5/dc/f55 745384 0 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:2/413: write d5/d1e/f70 [133830,41599] 0 2024-04-20T10:59:18.701 INFO:tasks.workunit.client.0.smithi027.stdout:2/414: chown d5/d26/d30 181 1 2024-04-20T10:59:18.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/433: link d0/d14/d6b/da3/d1b/d2f/l50 d0/d48/lad 0 2024-04-20T10:59:18.703 INFO:tasks.workunit.client.0.smithi027.stdout:0/538: sync 2024-04-20T10:59:18.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/434: write d0/d14/f4a [1918311,92324] 0 2024-04-20T10:59:18.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/435: chown d0/d14/d6b/da3/d53/d79 21 1 2024-04-20T10:59:18.703 INFO:tasks.workunit.client.0.smithi027.stdout:8/537: mkdir d9/de/d20/d48/d59/da0/dbd 0 2024-04-20T10:59:18.703 INFO:tasks.workunit.client.0.smithi027.stdout:8/538: write d9/de/d64/d6c/f9e [364691,7927] 0 2024-04-20T10:59:18.703 INFO:tasks.workunit.client.0.smithi027.stdout:9/392: unlink d4/d1b/d50/f56 0 2024-04-20T10:59:18.703 INFO:tasks.workunit.client.0.smithi027.stdout:9/393: write d4/d1b/d1f/d45/f7d [683225,127491] 0 2024-04-20T10:59:18.704 INFO:tasks.workunit.client.0.smithi027.stdout:6/461: mknod da/df/d7e/c94 0 2024-04-20T10:59:18.704 INFO:tasks.workunit.client.0.smithi027.stdout:6/462: read - da/df/d19/f74 zero size 2024-04-20T10:59:18.704 INFO:tasks.workunit.client.0.smithi027.stdout:6/463: fdatasync da/dd/f91 0 2024-04-20T10:59:18.704 INFO:tasks.workunit.client.0.smithi027.stdout:5/450: rename d19/d1b/d1c/f3b to d19/d1b/d1c/d1e/d58/f94 0 2024-04-20T10:59:18.706 INFO:tasks.workunit.client.0.smithi027.stdout:8/539: symlink d9/de/da3/dab/lbe 0 2024-04-20T10:59:18.706 INFO:tasks.workunit.client.0.smithi027.stdout:8/540: chown d9/de/da3/d3c/f44 32501 1 2024-04-20T10:59:18.706 INFO:tasks.workunit.client.0.smithi027.stdout:8/541: read - d9/de/d20/f9d zero size 2024-04-20T10:59:18.707 INFO:tasks.workunit.client.0.smithi027.stdout:8/542: readlink d9/de/d1e/d4a/d6d/db7/l9a 0 2024-04-20T10:59:18.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/394: mkdir d4/d1b/d31/d23/d29/d7f 0 2024-04-20T10:59:18.707 INFO:tasks.workunit.client.0.smithi027.stdout:9/395: write d4/d14/f38 [2613354,89576] 0 2024-04-20T10:59:18.707 INFO:tasks.workunit.client.0.smithi027.stdout:6/464: creat da/d21/f95 x:0 0 0 2024-04-20T10:59:18.708 INFO:tasks.workunit.client.0.smithi027.stdout:5/451: mknod d19/d1b/d1c/d1e/d22/c95 0 2024-04-20T10:59:18.711 INFO:tasks.workunit.client.0.smithi027.stdout:8/543: mkdir d9/de/d20/dbf 0 2024-04-20T10:59:18.714 INFO:tasks.workunit.client.0.smithi027.stdout:8/544: unlink d9/de/da3/d3c/f44 0 2024-04-20T10:59:18.717 INFO:tasks.workunit.client.0.smithi027.stdout:5/452: dread d19/d1b/d1c/d1e/d22/d47/f6b [0,4194304] 0 2024-04-20T10:59:18.717 INFO:tasks.workunit.client.0.smithi027.stdout:1/600: dread d3/d7/f57 [0,4194304] 0 2024-04-20T10:59:18.718 INFO:tasks.workunit.client.0.smithi027.stdout:8/545: symlink d9/de/d64/d6a/lc0 0 2024-04-20T10:59:18.719 INFO:tasks.workunit.client.0.smithi027.stdout:1/601: mkdir d3/d7/d38/da4/dac/de3 0 2024-04-20T10:59:18.723 INFO:tasks.workunit.client.0.smithi027.stdout:0/539: dwrite d0/d11/d2c/d2e/d2b/d7b/f1c [0,4194304] 0 2024-04-20T10:59:18.723 INFO:tasks.workunit.client.0.smithi027.stdout:0/540: chown d0/d11/d2c/d2e/d2b/d7b/d34/f43 5948 1 2024-04-20T10:59:18.723 INFO:tasks.workunit.client.0.smithi027.stdout:0/541: truncate d0/d11/d5c/f63 289359 0 2024-04-20T10:59:18.723 INFO:tasks.workunit.client.0.smithi027.stdout:0/542: chown d0/d11/d2c/d2e/d2b/d7b/f1c 3606204 1 2024-04-20T10:59:18.723 INFO:tasks.workunit.client.0.smithi027.stdout:0/543: chown d0/d11/f54 6 1 2024-04-20T10:59:18.726 INFO:tasks.workunit.client.0.smithi027.stdout:0/544: unlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/cb0 0 2024-04-20T10:59:18.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/545: getdents d0/d11/d2c/d2e/d55 0 2024-04-20T10:59:18.728 INFO:tasks.workunit.client.0.smithi027.stdout:8/546: read d9/de/f1d [270805,98681] 0 2024-04-20T10:59:18.730 INFO:tasks.workunit.client.0.smithi027.stdout:0/546: rename d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/fa2 to d0/d11/d62/fb8 0 2024-04-20T10:59:18.732 INFO:tasks.workunit.client.0.smithi027.stdout:8/547: mkdir d9/de/d1e/dc1 0 2024-04-20T10:59:18.733 INFO:tasks.workunit.client.0.smithi027.stdout:2/415: truncate d5/d26/d3d/f48 4034888 0 2024-04-20T10:59:18.734 INFO:tasks.workunit.client.0.smithi027.stdout:0/547: getdents d0/d11/d2c/d2e/d2b/d7b/d34 0 2024-04-20T10:59:18.735 INFO:tasks.workunit.client.0.smithi027.stdout:4/436: truncate d0/d14/d6b/da3/f16 1769987 0 2024-04-20T10:59:18.736 INFO:tasks.workunit.client.0.smithi027.stdout:4/437: write d0/d48/f9d [55965,59184] 0 2024-04-20T10:59:18.736 INFO:tasks.workunit.client.0.smithi027.stdout:4/438: truncate d0/d14/d6b/da3/d18/fa8 1039535 0 2024-04-20T10:59:18.736 INFO:tasks.workunit.client.0.smithi027.stdout:4/439: write d0/d14/d6b/da3/f1c [647559,122340] 0 2024-04-20T10:59:18.737 INFO:tasks.workunit.client.0.smithi027.stdout:8/548: rename d9/de/d20/d53/l8b to d9/de/d1e/d4a/d6d/d75/lc2 0 2024-04-20T10:59:18.739 INFO:tasks.workunit.client.0.smithi027.stdout:2/416: rename d5/d26/l53 to d5/dc/d16/d60/l87 0 2024-04-20T10:59:18.740 INFO:tasks.workunit.client.0.smithi027.stdout:8/549: creat d9/de/d20/dbf/fc3 x:0 0 0 2024-04-20T10:59:18.740 INFO:tasks.workunit.client.0.smithi027.stdout:8/550: write d9/de/d20/d48/d59/f99 [29097,51310] 0 2024-04-20T10:59:18.741 INFO:tasks.workunit.client.0.smithi027.stdout:4/440: mkdir d0/d14/d6b/da3/d1a/dae 0 2024-04-20T10:59:18.741 INFO:tasks.workunit.client.0.smithi027.stdout:4/441: chown d0/d14/d6b/da3/d53/la6 19020664 1 2024-04-20T10:59:18.741 INFO:tasks.workunit.client.0.smithi027.stdout:4/442: write d0/d14/d6b/da3/d1a/fa4 [30848,85185] 0 2024-04-20T10:59:18.741 INFO:tasks.workunit.client.0.smithi027.stdout:6/465: dwrite da/d21/f70 [0,4194304] 0 2024-04-20T10:59:18.742 INFO:tasks.workunit.client.0.smithi027.stdout:2/417: truncate d5/d26/f74 665886 0 2024-04-20T10:59:18.743 INFO:tasks.workunit.client.0.smithi027.stdout:1/602: dwrite d3/d7e/fd2 [0,4194304] 0 2024-04-20T10:59:18.744 INFO:tasks.workunit.client.0.smithi027.stdout:8/551: dread d9/de/d64/d6c/f8c [0,4194304] 0 2024-04-20T10:59:18.744 INFO:tasks.workunit.client.0.smithi027.stdout:6/466: mkdir da/d21/d96 0 2024-04-20T10:59:18.747 INFO:tasks.workunit.client.0.smithi027.stdout:1/603: read d3/d7/d38/db8/fbb [2974966,69388] 0 2024-04-20T10:59:18.747 INFO:tasks.workunit.client.0.smithi027.stdout:1/604: readlink d3/d7/d28/l31 0 2024-04-20T10:59:18.749 INFO:tasks.workunit.client.0.smithi027.stdout:6/467: mknod da/df/d1f/c97 0 2024-04-20T10:59:18.749 INFO:tasks.workunit.client.0.smithi027.stdout:0/548: dread d0/d11/d2c/d2e/d55/fac [0,4194304] 0 2024-04-20T10:59:18.750 INFO:tasks.workunit.client.0.smithi027.stdout:8/552: mkdir d9/de/d64/d6c/dc4 0 2024-04-20T10:59:18.751 INFO:tasks.workunit.client.0.smithi027.stdout:1/605: stat d3/d7/d38/c7c 0 2024-04-20T10:59:18.752 INFO:tasks.workunit.client.0.smithi027.stdout:8/553: rename d9/de/d18/f2d to d9/fc5 0 2024-04-20T10:59:18.752 INFO:tasks.workunit.client.0.smithi027.stdout:8/554: fsync d9/de/d1e/d4a/d6d/d79/f92 0 2024-04-20T10:59:18.756 INFO:tasks.workunit.client.0.smithi027.stdout:8/555: rename d9/lc to d9/de/d1e/d4a/d6d/d75/d7c/lc6 0 2024-04-20T10:59:18.756 INFO:tasks.workunit.client.0.smithi027.stdout:8/556: truncate d9/de/d1e/d4a/d6d/d75/f83 4544496 0 2024-04-20T10:59:18.756 INFO:tasks.workunit.client.0.smithi027.stdout:8/557: write d9/de/d20/d48/f85 [630547,30180] 0 2024-04-20T10:59:18.758 INFO:tasks.workunit.client.0.smithi027.stdout:6/468: write da/dd/d3d/f7f [527654,81967] 0 2024-04-20T10:59:18.758 INFO:tasks.workunit.client.0.smithi027.stdout:6/469: write da/f84 [605620,99525] 0 2024-04-20T10:59:18.758 INFO:tasks.workunit.client.0.smithi027.stdout:6/470: chown da/df/d19/f93 33323 1 2024-04-20T10:59:18.760 INFO:tasks.workunit.client.0.smithi027.stdout:8/558: rmdir d9/de/d20/d48/d59/dad 0 2024-04-20T10:59:18.761 INFO:tasks.workunit.client.0.smithi027.stdout:6/471: rmdir da/d21/d28/d2d 39 2024-04-20T10:59:18.762 INFO:tasks.workunit.client.0.smithi027.stdout:0/549: dread d0/d11/d2c/d2e/d2b/d7b/d64/f7c [0,4194304] 0 2024-04-20T10:59:18.762 INFO:tasks.workunit.client.0.smithi027.stdout:0/550: chown d0/d11/d2c/d2e/d2b/d7b/d12 70345 1 2024-04-20T10:59:18.764 INFO:tasks.workunit.client.0.smithi027.stdout:5/453: dwrite d19/d1b/d1c/d1e/f57 [4194304,4194304] 0 2024-04-20T10:59:18.764 INFO:tasks.workunit.client.0.smithi027.stdout:5/454: truncate d19/d1b/d1c/d1e/d22/f89 629440 0 2024-04-20T10:59:18.764 INFO:tasks.workunit.client.0.smithi027.stdout:5/455: chown d19/d1b/d1c/d1e/d22/f30 2329143 1 2024-04-20T10:59:18.764 INFO:tasks.workunit.client.0.smithi027.stdout:5/456: chown d19/d1b/d1c/d1e/d58/f60 13900 1 2024-04-20T10:59:18.764 INFO:tasks.workunit.client.0.smithi027.stdout:2/418: dread f1 [0,4194304] 0 2024-04-20T10:59:18.764 INFO:tasks.workunit.client.0.smithi027.stdout:2/419: readlink d5/d26/l2d 0 2024-04-20T10:59:18.766 INFO:tasks.workunit.client.0.smithi027.stdout:8/559: creat d9/de/d20/d48/d59/da0/dbd/fc7 x:0 0 0 2024-04-20T10:59:18.767 INFO:tasks.workunit.client.0.smithi027.stdout:0/551: getdents d0/d11/d2c/d2e/d2b/d7b/d12/d1a 0 2024-04-20T10:59:18.772 INFO:tasks.workunit.client.0.smithi027.stdout:5/457: creat d19/d1b/d1c/d1e/d3c/f96 x:0 0 0 2024-04-20T10:59:18.772 INFO:tasks.workunit.client.0.smithi027.stdout:2/420: mknod d5/d44/c88 0 2024-04-20T10:59:18.779 INFO:tasks.workunit.client.0.smithi027.stdout:1/606: dread d3/d7/ff [0,4194304] 0 2024-04-20T10:59:18.781 INFO:tasks.workunit.client.0.smithi027.stdout:0/552: creat d0/d59/d9e/fb9 x:0 0 0 2024-04-20T10:59:18.781 INFO:tasks.workunit.client.0.smithi027.stdout:0/553: write d0/d59/f68 [315714,45788] 0 2024-04-20T10:59:18.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/430: sync 2024-04-20T10:59:18.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/421: dread d5/d26/d47/f5f [0,4194304] 0 2024-04-20T10:59:18.787 INFO:tasks.workunit.client.0.smithi027.stdout:1/607: truncate d3/d41/f8c 115140 0 2024-04-20T10:59:18.787 INFO:tasks.workunit.client.0.smithi027.stdout:1/608: chown d3/d7/f23 159365232 1 2024-04-20T10:59:18.788 INFO:tasks.workunit.client.0.smithi027.stdout:7/431: rmdir d4/d5/d8/d21/d80 39 2024-04-20T10:59:18.797 INFO:tasks.workunit.client.0.smithi027.stdout:1/609: dread d3/d45/fb6 [0,4194304] 0 2024-04-20T10:59:18.801 INFO:tasks.workunit.client.0.smithi027.stdout:6/472: dwrite da/d21/f5b [0,4194304] 0 2024-04-20T10:59:18.802 INFO:tasks.workunit.client.0.smithi027.stdout:0/554: dwrite d0/d11/d2c/d2e/d2b/d3d/fb3 [0,4194304] 0 2024-04-20T10:59:18.803 INFO:tasks.workunit.client.0.smithi027.stdout:8/560: dwrite d9/de/d1e/d4a/f45 [0,4194304] 0 2024-04-20T10:59:18.809 INFO:tasks.workunit.client.0.smithi027.stdout:0/555: write d0/f1f [2050675,41623] 0 2024-04-20T10:59:19.044 INFO:tasks.workunit.client.0.smithi027.stdout:5/458: symlink d19/l97 0 2024-04-20T10:59:19.044 INFO:tasks.workunit.client.0.smithi027.stdout:2/422: mkdir d5/dc/d16/d2a/d89 0 2024-04-20T10:59:19.044 INFO:tasks.workunit.client.0.smithi027.stdout:5/459: chown d19/d1b/d1c/d1e/d3c/c8a 218 1 2024-04-20T10:59:19.044 INFO:tasks.workunit.client.0.smithi027.stdout:8/561: mknod d9/de/d20/d53/cc8 0 2024-04-20T10:59:19.045 INFO:tasks.workunit.client.0.smithi027.stdout:0/556: symlink d0/d11/d2c/da1/lba 0 2024-04-20T10:59:19.048 INFO:tasks.workunit.client.0.smithi027.stdout:7/432: rename d4/d5/d8/d21/d80 to d4/d1b/d4a/d62/d87 0 2024-04-20T10:59:19.049 INFO:tasks.workunit.client.0.smithi027.stdout:2/423: mkdir d5/d26/d47/d8a 0 2024-04-20T10:59:19.051 INFO:tasks.workunit.client.0.smithi027.stdout:0/557: symlink d0/d11/d4b/d5b/d6f/d82/lbb 0 2024-04-20T10:59:19.053 INFO:tasks.workunit.client.0.smithi027.stdout:6/473: rename da/df/d19/d2e/c83 to da/df/c98 0 2024-04-20T10:59:19.053 INFO:tasks.workunit.client.0.smithi027.stdout:6/474: readlink da/df/d19/l27 0 2024-04-20T10:59:19.054 INFO:tasks.workunit.client.0.smithi027.stdout:2/424: link d5/d44/c88 d5/d26/c8b 0 2024-04-20T10:59:19.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/475: creat da/f99 x:0 0 0 2024-04-20T10:59:19.057 INFO:tasks.workunit.client.0.smithi027.stdout:2/425: rename d5/dc/d16/d2a/d4a/c72 to d5/dc/d16/d60/c8c 0 2024-04-20T10:59:19.059 INFO:tasks.workunit.client.0.smithi027.stdout:6/476: mkdir da/d21/d28/d2d/d53/d9a 0 2024-04-20T10:59:19.059 INFO:tasks.workunit.client.0.smithi027.stdout:6/477: dread - da/dd/d3d/f8c zero size 2024-04-20T10:59:19.060 INFO:tasks.workunit.client.0.smithi027.stdout:0/558: write d0/d11/d2c/d2e/d2b/d7b/d12/d1a/fa5 [2091553,112051] 0 2024-04-20T10:59:19.062 INFO:tasks.workunit.client.0.smithi027.stdout:8/562: dwrite d9/fd [0,4194304] 0 2024-04-20T10:59:19.062 INFO:tasks.workunit.client.0.smithi027.stdout:4/443: sync 2024-04-20T10:59:19.062 INFO:tasks.workunit.client.0.smithi027.stdout:9/396: sync 2024-04-20T10:59:19.062 INFO:tasks.workunit.client.0.smithi027.stdout:3/484: sync 2024-04-20T10:59:19.063 INFO:tasks.workunit.client.0.smithi027.stdout:4/444: readlink d0/d14/l2b 0 2024-04-20T10:59:19.063 INFO:tasks.workunit.client.0.smithi027.stdout:9/397: truncate d4/d14/f27 1742182 0 2024-04-20T10:59:19.063 INFO:tasks.workunit.client.0.smithi027.stdout:1/610: sync 2024-04-20T10:59:19.064 INFO:tasks.workunit.client.0.smithi027.stdout:8/563: getdents d9/de/d20/d48 0 2024-04-20T10:59:19.064 INFO:tasks.workunit.client.0.smithi027.stdout:5/460: write f16 [446000,25791] 0 2024-04-20T10:59:19.064 INFO:tasks.workunit.client.0.smithi027.stdout:3/485: rename d2/f1d to d2/dd/d57/f97 0 2024-04-20T10:59:19.064 INFO:tasks.workunit.client.0.smithi027.stdout:8/564: creat d9/de/da3/dab/fc9 x:0 0 0 2024-04-20T10:59:19.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/461: creat d19/d1b/d1c/f98 x:0 0 0 2024-04-20T10:59:19.065 INFO:tasks.workunit.client.0.smithi027.stdout:7/433: dwrite d4/d1b/d4a/f68 [0,4194304] 0 2024-04-20T10:59:19.065 INFO:tasks.workunit.client.0.smithi027.stdout:7/434: chown d4/d5/d8/c61 30 1 2024-04-20T10:59:19.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/398: rename d4/d74/l78 to d4/d1b/l80 0 2024-04-20T10:59:19.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/399: chown d4/d1b/d31/d51/l62 1029723 1 2024-04-20T10:59:19.066 INFO:tasks.workunit.client.0.smithi027.stdout:3/486: getdents d2/d18/d2c/d4c 0 2024-04-20T10:59:19.067 INFO:tasks.workunit.client.0.smithi027.stdout:3/487: truncate d2/dd/f77 831183 0 2024-04-20T10:59:19.067 INFO:tasks.workunit.client.0.smithi027.stdout:8/565: truncate d9/de/d20/d48/f85 94513 0 2024-04-20T10:59:19.075 INFO:tasks.workunit.client.0.smithi027.stdout:7/435: creat d4/d1b/f88 x:0 0 0 2024-04-20T10:59:19.076 INFO:tasks.workunit.client.0.smithi027.stdout:1/611: rename d3/d41/f4e to d3/d7/d28/d92/dad/fe4 0 2024-04-20T10:59:19.079 INFO:tasks.workunit.client.0.smithi027.stdout:8/566: mknod d9/de/da3/d3c/d4b/cca 0 2024-04-20T10:59:19.092 INFO:tasks.workunit.client.0.smithi027.stdout:6/478: write da/d21/d28/f34 [7232134,21323] 0 2024-04-20T10:59:19.094 INFO:tasks.workunit.client.0.smithi027.stdout:6/479: creat da/d21/f9b x:0 0 0 2024-04-20T10:59:19.094 INFO:tasks.workunit.client.0.smithi027.stdout:6/480: read da/f1b [223170,70439] 0 2024-04-20T10:59:19.097 INFO:tasks.workunit.client.0.smithi027.stdout:2/426: dwrite d5/dc/d16/d2a/d59/f81 [4194304,4194304] 0 2024-04-20T10:59:19.097 INFO:tasks.workunit.client.0.smithi027.stdout:0/559: dread d0/d11/f21 [0,4194304] 0 2024-04-20T10:59:19.097 INFO:tasks.workunit.client.0.smithi027.stdout:2/427: chown d5/dc/l12 111 1 2024-04-20T10:59:19.098 INFO:tasks.workunit.client.0.smithi027.stdout:0/560: symlink d0/d11/d2c/d2e/d35/d8c/lbc 0 2024-04-20T10:59:19.099 INFO:tasks.workunit.client.0.smithi027.stdout:2/428: creat d5/dc/d16/d2a/d59/f8d x:0 0 0 2024-04-20T10:59:19.101 INFO:tasks.workunit.client.0.smithi027.stdout:0/561: creat d0/d11/d4b/d5b/fbd x:0 0 0 2024-04-20T10:59:19.101 INFO:tasks.workunit.client.0.smithi027.stdout:2/429: mkdir d5/dc/d16/d60/d8e 0 2024-04-20T10:59:19.106 INFO:tasks.workunit.client.0.smithi027.stdout:2/430: truncate d5/d26/d3d/f48 2363018 0 2024-04-20T10:59:19.106 INFO:tasks.workunit.client.0.smithi027.stdout:2/431: fsync d5/d1e/f42 0 2024-04-20T10:59:19.106 INFO:tasks.workunit.client.0.smithi027.stdout:2/432: readlink d5/d26/d78/l62 0 2024-04-20T10:59:19.107 INFO:tasks.workunit.client.0.smithi027.stdout:2/433: mkdir d5/dc/d16/d2a/d4a/d8f 0 2024-04-20T10:59:19.107 INFO:tasks.workunit.client.0.smithi027.stdout:2/434: chown d5/d44/c88 25 1 2024-04-20T10:59:19.107 INFO:tasks.workunit.client.0.smithi027.stdout:2/435: truncate d5/dc/d16/d2a/d4a/f50 4540468 0 2024-04-20T10:59:19.108 INFO:tasks.workunit.client.0.smithi027.stdout:2/436: chown d5/d1e 7 1 2024-04-20T10:59:19.108 INFO:tasks.workunit.client.0.smithi027.stdout:2/437: fdatasync d5/dc/f41 0 2024-04-20T10:59:19.108 INFO:tasks.workunit.client.0.smithi027.stdout:2/438: dread - d5/dc/d16/d2a/f84 zero size 2024-04-20T10:59:19.109 INFO:tasks.workunit.client.0.smithi027.stdout:2/439: write d5/dc/f41 [1256401,79210] 0 2024-04-20T10:59:19.109 INFO:tasks.workunit.client.0.smithi027.stdout:3/488: dwrite d2/d18/f29 [0,4194304] 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:3/489: creat d2/dd/d57/f98 x:0 0 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:3/490: mkdir d2/d18/d2c/d4c/d67/d99 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:9/400: dwrite d4/d2a/f66 [0,4194304] 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:5/462: dwrite d19/d1b/f43 [0,4194304] 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:1/612: dwrite d3/d7/d38/f56 [0,4194304] 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:1/613: truncate d3/d45/d5d/d6c/f72 603473 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:3/491: write d2/dd/df/d33/f3c [4252961,78042] 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:3/492: write d2/d80/f5a [885944,61922] 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:4/445: dwrite d0/d14/d6b/da3/d1b/d33/f9a [0,4194304] 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:8/567: dwrite d9/de/d1e/d4a/d6d/fbb [0,4194304] 0 2024-04-20T10:59:19.116 INFO:tasks.workunit.client.0.smithi027.stdout:6/481: dwrite da/df/d19/f93 [0,4194304] 0 2024-04-20T10:59:19.117 INFO:tasks.workunit.client.0.smithi027.stdout:3/493: creat d2/d18/d2e/f9a x:0 0 0 2024-04-20T10:59:19.119 INFO:tasks.workunit.client.0.smithi027.stdout:4/446: symlink d0/d14/d6b/da3/d1b/d33/laf 0 2024-04-20T10:59:19.119 INFO:tasks.workunit.client.0.smithi027.stdout:4/447: readlink d0/d14/d6b/da3/d53/d6a/l8f 0 2024-04-20T10:59:19.119 INFO:tasks.workunit.client.0.smithi027.stdout:4/448: stat d0/d14/d6b/da3/d18/c45 0 2024-04-20T10:59:19.119 INFO:tasks.workunit.client.0.smithi027.stdout:4/449: chown d0/d14/d6b/da3/d18 8 1 2024-04-20T10:59:19.119 INFO:tasks.workunit.client.0.smithi027.stdout:4/450: write d0/f27 [4848069,94032] 0 2024-04-20T10:59:19.121 INFO:tasks.workunit.client.0.smithi027.stdout:8/568: creat d9/de/d1e/fcb x:0 0 0 2024-04-20T10:59:19.122 INFO:tasks.workunit.client.0.smithi027.stdout:9/401: link d4/d2a/c6c d4/d14/d43/c81 0 2024-04-20T10:59:19.125 INFO:tasks.workunit.client.0.smithi027.stdout:4/451: mkdir d0/d14/d6b/da3/d53/d6a/da1/db0 0 2024-04-20T10:59:19.126 INFO:tasks.workunit.client.0.smithi027.stdout:8/569: write d9/de/f2f [4797322,47938] 0 2024-04-20T10:59:19.127 INFO:tasks.workunit.client.0.smithi027.stdout:9/402: rmdir d4/d1b/d50 39 2024-04-20T10:59:19.131 INFO:tasks.workunit.client.0.smithi027.stdout:3/494: dread d2/dd/d57/f97 [0,4194304] 0 2024-04-20T10:59:19.134 INFO:tasks.workunit.client.0.smithi027.stdout:3/495: dread d2/dd/f77 [0,4194304] 0 2024-04-20T10:59:19.134 INFO:tasks.workunit.client.0.smithi027.stdout:3/496: chown d2/d18/d2c/c8e 235 1 2024-04-20T10:59:19.136 INFO:tasks.workunit.client.0.smithi027.stdout:3/497: truncate d2/d18/f29 391701 0 2024-04-20T10:59:19.139 INFO:tasks.workunit.client.0.smithi027.stdout:1/614: dwrite d3/d7/d28/d92/fb5 [0,4194304] 0 2024-04-20T10:59:19.139 INFO:tasks.workunit.client.0.smithi027.stdout:1/615: getdents d3/d7/d38/da4/dbc 0 2024-04-20T10:59:19.139 INFO:tasks.workunit.client.0.smithi027.stdout:2/440: dwrite d5/f58 [0,4194304] 0 2024-04-20T10:59:19.139 INFO:tasks.workunit.client.0.smithi027.stdout:0/562: dwrite d0/d11/d2c/d2e/d2b/d7b/d12/f44 [0,4194304] 0 2024-04-20T10:59:19.141 INFO:tasks.workunit.client.0.smithi027.stdout:3/498: rmdir d2/d18/d2c/d4c/d67/d99 0 2024-04-20T10:59:19.141 INFO:tasks.workunit.client.0.smithi027.stdout:5/463: dwrite d19/d1b/d1c/f98 [0,4194304] 0 2024-04-20T10:59:19.142 INFO:tasks.workunit.client.0.smithi027.stdout:2/441: link d5/dc/d16/d37/f6b d5/d44/f90 0 2024-04-20T10:59:19.142 INFO:tasks.workunit.client.0.smithi027.stdout:2/442: write d5/d44/d7c/f7e [28731,92586] 0 2024-04-20T10:59:19.143 INFO:tasks.workunit.client.0.smithi027.stdout:6/482: dwrite da/d21/f95 [0,4194304] 0 2024-04-20T10:59:19.145 INFO:tasks.workunit.client.0.smithi027.stdout:0/563: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/fbe x:0 0 0 2024-04-20T10:59:19.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/499: truncate d2/d18/d2c/d4c/f7e 1167762 0 2024-04-20T10:59:19.147 INFO:tasks.workunit.client.0.smithi027.stdout:1/616: write d3/d7e/fdc [1542380,86116] 0 2024-04-20T10:59:19.148 INFO:tasks.workunit.client.0.smithi027.stdout:5/464: mknod d19/d1b/d1c/c99 0 2024-04-20T10:59:19.150 INFO:tasks.workunit.client.0.smithi027.stdout:6/483: stat da/dd/c73 0 2024-04-20T10:59:19.151 INFO:tasks.workunit.client.0.smithi027.stdout:6/484: write da/d21/f9b [251415,118500] 0 2024-04-20T10:59:19.152 INFO:tasks.workunit.client.0.smithi027.stdout:0/564: mkdir d0/d59/dbf 0 2024-04-20T10:59:19.152 INFO:tasks.workunit.client.0.smithi027.stdout:3/500: getdents d2/d94 0 2024-04-20T10:59:19.152 INFO:tasks.workunit.client.0.smithi027.stdout:3/501: write d2/d18/d2e/f75 [1070656,104863] 0 2024-04-20T10:59:19.153 INFO:tasks.workunit.client.0.smithi027.stdout:1/617: mkdir d3/d7/d38/da4/dbc/de5 0 2024-04-20T10:59:19.154 INFO:tasks.workunit.client.0.smithi027.stdout:3/502: symlink d2/dd/df/d23/d68/l9b 0 2024-04-20T10:59:19.155 INFO:tasks.workunit.client.0.smithi027.stdout:8/570: dwrite d9/de/d20/d48/d59/da0/dbd/fc7 [0,4194304] 0 2024-04-20T10:59:19.156 INFO:tasks.workunit.client.0.smithi027.stdout:4/452: dwrite d0/d14/d6b/da3/d1b/d2f/f67 [0,4194304] 0 2024-04-20T10:59:19.156 INFO:tasks.workunit.client.0.smithi027.stdout:1/618: mkdir d3/d41/dc6/de6 0 2024-04-20T10:59:19.156 INFO:tasks.workunit.client.0.smithi027.stdout:3/503: symlink d2/dd/d22/l9c 0 2024-04-20T10:59:19.156 INFO:tasks.workunit.client.0.smithi027.stdout:3/504: chown d2/d18/f3f 6834428 1 2024-04-20T10:59:19.157 INFO:tasks.workunit.client.0.smithi027.stdout:9/403: dwrite d4/d14/f44 [4194304,4194304] 0 2024-04-20T10:59:19.158 INFO:tasks.workunit.client.0.smithi027.stdout:3/505: dread d2/dd/f77 [0,4194304] 0 2024-04-20T10:59:19.160 INFO:tasks.workunit.client.0.smithi027.stdout:8/571: creat d9/de/d1e/dc1/fcc x:0 0 0 2024-04-20T10:59:19.160 INFO:tasks.workunit.client.0.smithi027.stdout:8/572: chown d9/de/d64/d6a/f67 1 1 2024-04-20T10:59:19.162 INFO:tasks.workunit.client.0.smithi027.stdout:1/619: rename d3/d7/d38/f65 to d3/d45/fe7 0 2024-04-20T10:59:19.166 INFO:tasks.workunit.client.0.smithi027.stdout:4/453: dread d0/f13 [0,4194304] 0 2024-04-20T10:59:19.166 INFO:tasks.workunit.client.0.smithi027.stdout:4/454: getdents d0/d48/d68/d76 0 2024-04-20T10:59:19.166 INFO:tasks.workunit.client.0.smithi027.stdout:4/455: dread - d0/d14/d6b/da3/d1a/f8a zero size 2024-04-20T10:59:19.166 INFO:tasks.workunit.client.0.smithi027.stdout:4/456: stat d0/d14/d6b/da3 0 2024-04-20T10:59:19.168 INFO:tasks.workunit.client.0.smithi027.stdout:6/485: dwrite da/df/d19/d2e/d6b/f62 [0,4194304] 0 2024-04-20T10:59:19.168 INFO:tasks.workunit.client.0.smithi027.stdout:6/486: dread - da/df/f29 zero size 2024-04-20T10:59:19.176 INFO:tasks.workunit.client.0.smithi027.stdout:9/404: creat d4/d1b/d31/d23/d29/f82 x:0 0 0 2024-04-20T10:59:19.177 INFO:tasks.workunit.client.0.smithi027.stdout:3/506: mkdir d2/dd/d57/d8d/d9d 0 2024-04-20T10:59:19.180 INFO:tasks.workunit.client.0.smithi027.stdout:4/457: mknod d0/d14/d6b/da3/cb1 0 2024-04-20T10:59:19.180 INFO:tasks.workunit.client.0.smithi027.stdout:4/458: stat d0/d48/d68 0 2024-04-20T10:59:19.182 INFO:tasks.workunit.client.0.smithi027.stdout:3/507: mknod d2/d18/c9e 0 2024-04-20T10:59:19.184 INFO:tasks.workunit.client.0.smithi027.stdout:3/508: symlink d2/l9f 0 2024-04-20T10:59:19.186 INFO:tasks.workunit.client.0.smithi027.stdout:3/509: creat d2/dd/d22/fa0 x:0 0 0 2024-04-20T10:59:19.186 INFO:tasks.workunit.client.0.smithi027.stdout:3/510: write d2/dd/d57/d8c/f8f [598443,80813] 0 2024-04-20T10:59:19.187 INFO:tasks.workunit.client.0.smithi027.stdout:9/405: dread d4/d1b/f22 [0,4194304] 0 2024-04-20T10:59:19.187 INFO:tasks.workunit.client.0.smithi027.stdout:9/406: fsync d4/d2a/f77 0 2024-04-20T10:59:19.187 INFO:tasks.workunit.client.0.smithi027.stdout:3/511: symlink d2/d18/d2c/d4c/d67/la1 0 2024-04-20T10:59:19.188 INFO:tasks.workunit.client.0.smithi027.stdout:4/459: dread d0/d14/d6b/da3/d1b/f3d [0,4194304] 0 2024-04-20T10:59:19.189 INFO:tasks.workunit.client.0.smithi027.stdout:9/407: creat d4/d1b/d1f/d45/f83 x:0 0 0 2024-04-20T10:59:19.189 INFO:tasks.workunit.client.0.smithi027.stdout:1/620: dwrite d3/d7/d38/d62/fb9 [0,4194304] 0 2024-04-20T10:59:19.189 INFO:tasks.workunit.client.0.smithi027.stdout:3/512: creat d2/dd/d22/fa2 x:0 0 0 2024-04-20T10:59:19.189 INFO:tasks.workunit.client.0.smithi027.stdout:3/513: fdatasync d2/dd/df/f3a 0 2024-04-20T10:59:19.193 INFO:tasks.workunit.client.0.smithi027.stdout:6/487: dwrite da/d21/f36 [0,4194304] 0 2024-04-20T10:59:19.194 INFO:tasks.workunit.client.0.smithi027.stdout:6/488: write da/df/d7e/f85 [90613,70836] 0 2024-04-20T10:59:19.194 INFO:tasks.workunit.client.0.smithi027.stdout:6/489: read da/df/d19/d2e/d32/f44 [3166342,53330] 0 2024-04-20T10:59:19.194 INFO:tasks.workunit.client.0.smithi027.stdout:4/460: creat d0/d5e/d6c/fb2 x:0 0 0 2024-04-20T10:59:19.194 INFO:tasks.workunit.client.0.smithi027.stdout:4/461: readlink d0/d14/d6b/da3/d53/d79/l99 0 2024-04-20T10:59:19.194 INFO:tasks.workunit.client.0.smithi027.stdout:4/462: chown d0/d5e/d6c/l80 19284 1 2024-04-20T10:59:19.194 INFO:tasks.workunit.client.0.smithi027.stdout:9/408: unlink d4/d1b/d31/d23/d29/f4c 0 2024-04-20T10:59:19.195 INFO:tasks.workunit.client.0.smithi027.stdout:1/621: rename d3/d45/d5d/l77 to d3/d7/d38/ddf/d73/le8 0 2024-04-20T10:59:19.195 INFO:tasks.workunit.client.0.smithi027.stdout:7/436: sync 2024-04-20T10:59:19.195 INFO:tasks.workunit.client.0.smithi027.stdout:7/437: readlink d4/d5/d8/d21/l2c 0 2024-04-20T10:59:19.195 INFO:tasks.workunit.client.0.smithi027.stdout:7/438: write d4/d1b/d4a/f44 [1438443,65587] 0 2024-04-20T10:59:19.195 INFO:tasks.workunit.client.0.smithi027.stdout:3/514: truncate d2/dd/df/d23/f50 4530586 0 2024-04-20T10:59:19.195 INFO:tasks.workunit.client.0.smithi027.stdout:3/515: write d2/dd/df/d23/f86 [709353,64321] 0 2024-04-20T10:59:19.196 INFO:tasks.workunit.client.0.smithi027.stdout:3/516: dread d2/d18/d2e/f75 [0,4194304] 0 2024-04-20T10:59:19.199 INFO:tasks.workunit.client.0.smithi027.stdout:4/463: symlink d0/d14/d6b/da3/d18/lb3 0 2024-04-20T10:59:19.199 INFO:tasks.workunit.client.0.smithi027.stdout:4/464: chown d0/d48 2069369 1 2024-04-20T10:59:19.199 INFO:tasks.workunit.client.0.smithi027.stdout:9/409: symlink d4/d1b/d31/d23/d29/d7f/l84 0 2024-04-20T10:59:19.199 INFO:tasks.workunit.client.0.smithi027.stdout:6/490: dread da/d21/f5b [0,4194304] 0 2024-04-20T10:59:19.199 INFO:tasks.workunit.client.0.smithi027.stdout:6/491: rename da to da/dd/d79/d9c 22 2024-04-20T10:59:19.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/622: mkdir d3/d7/d38/ddf/de9 0 2024-04-20T10:59:19.200 INFO:tasks.workunit.client.0.smithi027.stdout:7/439: rmdir d4/d5/d8/d21 39 2024-04-20T10:59:19.201 INFO:tasks.workunit.client.0.smithi027.stdout:7/440: readlink d4/d1b/d58/l78 0 2024-04-20T10:59:19.202 INFO:tasks.workunit.client.0.smithi027.stdout:3/517: creat d2/dd/d57/d8d/fa3 x:0 0 0 2024-04-20T10:59:19.203 INFO:tasks.workunit.client.0.smithi027.stdout:4/465: rmdir d0/d14/d6b/da3/d1b/d33 39 2024-04-20T10:59:19.205 INFO:tasks.workunit.client.0.smithi027.stdout:1/623: write d3/d7/d38/d62/f81 [3436899,88453] 0 2024-04-20T10:59:19.206 INFO:tasks.workunit.client.0.smithi027.stdout:1/624: chown d3/d7/d38/da4/cb2 331 1 2024-04-20T10:59:19.206 INFO:tasks.workunit.client.0.smithi027.stdout:6/492: mknod da/d21/d28/c9d 0 2024-04-20T10:59:19.206 INFO:tasks.workunit.client.0.smithi027.stdout:6/493: readlink da/dd/d3d/l40 0 2024-04-20T10:59:19.208 INFO:tasks.workunit.client.0.smithi027.stdout:3/518: creat d2/d7f/fa4 x:0 0 0 2024-04-20T10:59:19.209 INFO:tasks.workunit.client.0.smithi027.stdout:4/466: creat d0/d14/d6b/da3/d53/fb4 x:0 0 0 2024-04-20T10:59:19.210 INFO:tasks.workunit.client.0.smithi027.stdout:9/410: symlink d4/d14/d43/l85 0 2024-04-20T10:59:19.211 INFO:tasks.workunit.client.0.smithi027.stdout:1/625: creat d3/d41/dc6/de6/fea x:0 0 0 2024-04-20T10:59:19.211 INFO:tasks.workunit.client.0.smithi027.stdout:6/494: truncate da/df/f1a 3210664 0 2024-04-20T10:59:19.213 INFO:tasks.workunit.client.0.smithi027.stdout:3/519: creat d2/dd/d57/fa5 x:0 0 0 2024-04-20T10:59:19.215 INFO:tasks.workunit.client.0.smithi027.stdout:1/626: write d3/d45/fb6 [3419562,101904] 0 2024-04-20T10:59:19.215 INFO:tasks.workunit.client.0.smithi027.stdout:4/467: creat d0/d14/d6b/da3/d1a/dae/fb5 x:0 0 0 2024-04-20T10:59:19.215 INFO:tasks.workunit.client.0.smithi027.stdout:4/468: write d0/d14/d6b/da3/d53/fb4 [1040411,10437] 0 2024-04-20T10:59:19.215 INFO:tasks.workunit.client.0.smithi027.stdout:4/469: write d0/d14/d6b/da3/d53/d6a/faa [976432,57430] 0 2024-04-20T10:59:19.215 INFO:tasks.workunit.client.0.smithi027.stdout:4/470: chown d0/d14/f4a 938623456 1 2024-04-20T10:59:19.216 INFO:tasks.workunit.client.0.smithi027.stdout:6/495: mkdir da/dd/d3c/d9e 0 2024-04-20T10:59:19.216 INFO:tasks.workunit.client.0.smithi027.stdout:6/496: fdatasync da/df/d19/d2e/d32/f75 0 2024-04-20T10:59:19.217 INFO:tasks.workunit.client.0.smithi027.stdout:3/520: link d2/d18/d2c/c8e d2/d80/ca6 0 2024-04-20T10:59:19.217 INFO:tasks.workunit.client.0.smithi027.stdout:3/521: read - d2/d94/f95 zero size 2024-04-20T10:59:19.217 INFO:tasks.workunit.client.0.smithi027.stdout:3/522: dread - d2/d18/d2c/d4c/d67/f6d zero size 2024-04-20T10:59:19.218 INFO:tasks.workunit.client.0.smithi027.stdout:7/441: dwrite d4/d5/d8/d21/f31 [0,4194304] 0 2024-04-20T10:59:19.223 INFO:tasks.workunit.client.0.smithi027.stdout:1/627: rename d3/d41/c68 to d3/d7/d38/da4/dbc/ceb 0 2024-04-20T10:59:19.223 INFO:tasks.workunit.client.0.smithi027.stdout:9/411: unlink d4/d2a/f46 0 2024-04-20T10:59:19.224 INFO:tasks.workunit.client.0.smithi027.stdout:3/523: creat d2/d18/d2e/fa7 x:0 0 0 2024-04-20T10:59:19.224 INFO:tasks.workunit.client.0.smithi027.stdout:1/628: rename d3/d45/d5d/ldb to d3/d7/d38/db8/lec 0 2024-04-20T10:59:19.225 INFO:tasks.workunit.client.0.smithi027.stdout:9/412: creat d4/d14/d43/f86 x:0 0 0 2024-04-20T10:59:19.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/442: dread d4/d1b/ff [0,4194304] 0 2024-04-20T10:59:19.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/443: readlink d4/d5/d8/d21/l3f 0 2024-04-20T10:59:19.232 INFO:tasks.workunit.client.0.smithi027.stdout:7/444: dread d4/d5/d8/f38 [0,4194304] 0 2024-04-20T10:59:19.240 INFO:tasks.workunit.client.0.smithi027.stdout:6/497: dwrite da/df/d19/f74 [0,4194304] 0 2024-04-20T10:59:19.241 INFO:tasks.workunit.client.0.smithi027.stdout:6/498: creat da/d21/d28/d54/f9f x:0 0 0 2024-04-20T10:59:19.243 INFO:tasks.workunit.client.0.smithi027.stdout:6/499: mknod da/df/d19/d2e/d6b/ca0 0 2024-04-20T10:59:19.244 INFO:tasks.workunit.client.0.smithi027.stdout:6/500: mknod da/dd/d3c/d9e/ca1 0 2024-04-20T10:59:19.244 INFO:tasks.workunit.client.0.smithi027.stdout:6/501: creat da/df/fa2 x:0 0 0 2024-04-20T10:59:19.245 INFO:tasks.workunit.client.0.smithi027.stdout:6/502: mkdir da/df/d7e/d86/da3 0 2024-04-20T10:59:19.245 INFO:tasks.workunit.client.0.smithi027.stdout:6/503: stat da/df/d19/d2e 0 2024-04-20T10:59:19.247 INFO:tasks.workunit.client.0.smithi027.stdout:4/471: dwrite d0/d14/d6b/da3/d1b/d2f/f73 [0,4194304] 0 2024-04-20T10:59:19.248 INFO:tasks.workunit.client.0.smithi027.stdout:1/629: dwrite d3/d7/d38/da4/fc1 [4194304,4194304] 0 2024-04-20T10:59:19.248 INFO:tasks.workunit.client.0.smithi027.stdout:1/630: dread - d3/d7/d38/da4/fcf zero size 2024-04-20T10:59:19.249 INFO:tasks.workunit.client.0.smithi027.stdout:4/472: creat d0/fb6 x:0 0 0 2024-04-20T10:59:19.251 INFO:tasks.workunit.client.0.smithi027.stdout:3/524: dwrite d2/dd/d57/f97 [0,4194304] 0 2024-04-20T10:59:19.251 INFO:tasks.workunit.client.0.smithi027.stdout:3/525: write d2/d7f/fa4 [309428,56727] 0 2024-04-20T10:59:19.251 INFO:tasks.workunit.client.0.smithi027.stdout:3/526: chown d2/dd/f58 212319 1 2024-04-20T10:59:19.258 INFO:tasks.workunit.client.0.smithi027.stdout:5/465: sync 2024-04-20T10:59:19.258 INFO:tasks.workunit.client.0.smithi027.stdout:0/565: sync 2024-04-20T10:59:19.259 INFO:tasks.workunit.client.0.smithi027.stdout:2/443: sync 2024-04-20T10:59:19.259 INFO:tasks.workunit.client.0.smithi027.stdout:0/566: dread - d0/d11/d2c/d2e/d55/f6b zero size 2024-04-20T10:59:19.259 INFO:tasks.workunit.client.0.smithi027.stdout:7/445: dwrite d4/f5a [0,4194304] 0 2024-04-20T10:59:19.260 INFO:tasks.workunit.client.0.smithi027.stdout:5/466: rename d19/d1b/d1c/d1e/d22/f67 to d19/d1b/d1c/d1e/d22/d47/f9a 0 2024-04-20T10:59:19.260 INFO:tasks.workunit.client.0.smithi027.stdout:2/444: symlink d5/d1e/d21/l91 0 2024-04-20T10:59:19.262 INFO:tasks.workunit.client.0.smithi027.stdout:0/567: symlink d0/d11/d2c/d2e/d2b/d3d/lc0 0 2024-04-20T10:59:19.266 INFO:tasks.workunit.client.0.smithi027.stdout:7/446: creat d4/d5/d8/d46/f89 x:0 0 0 2024-04-20T10:59:19.267 INFO:tasks.workunit.client.0.smithi027.stdout:5/467: creat d19/f9b x:0 0 0 2024-04-20T10:59:19.268 INFO:tasks.workunit.client.0.smithi027.stdout:2/445: symlink d5/dc/d16/d32/l92 0 2024-04-20T10:59:19.269 INFO:tasks.workunit.client.0.smithi027.stdout:0/568: link d0/d11/d2c/d2e/d2b/d7b/d64/f93 d0/d11/d2c/d2e/d2b/d7b/d12/fc1 0 2024-04-20T10:59:19.271 INFO:tasks.workunit.client.0.smithi027.stdout:7/447: creat d4/d1b/d4a/d62/d87/f8a x:0 0 0 2024-04-20T10:59:19.271 INFO:tasks.workunit.client.0.smithi027.stdout:7/448: truncate d4/f70 341982 0 2024-04-20T10:59:19.272 INFO:tasks.workunit.client.0.smithi027.stdout:2/446: creat d5/dc/d16/d2a/f93 x:0 0 0 2024-04-20T10:59:19.272 INFO:tasks.workunit.client.0.smithi027.stdout:2/447: write d5/dc/f1c [4549217,89845] 0 2024-04-20T10:59:19.274 INFO:tasks.workunit.client.0.smithi027.stdout:0/569: mknod d0/d11/d2c/d2e/d2b/d3d/cc2 0 2024-04-20T10:59:19.275 INFO:tasks.workunit.client.0.smithi027.stdout:2/448: creat d5/dc/d16/d2a/d4a/f94 x:0 0 0 2024-04-20T10:59:19.276 INFO:tasks.workunit.client.0.smithi027.stdout:0/570: rename d0/d11/d2c/d2e/d2b/d7b/d12/d1a/c4a to d0/d11/d4b/d5b/d6f/cc3 0 2024-04-20T10:59:19.276 INFO:tasks.workunit.client.0.smithi027.stdout:8/573: sync 2024-04-20T10:59:19.276 INFO:tasks.workunit.client.0.smithi027.stdout:8/574: stat d9/de/d1e/d4a/d6d/d75/d7c/fbc 0 2024-04-20T10:59:19.277 INFO:tasks.workunit.client.0.smithi027.stdout:7/449: write d4/f5a [2806999,114212] 0 2024-04-20T10:59:19.277 INFO:tasks.workunit.client.0.smithi027.stdout:2/449: creat d5/dc/d16/d2a/d59/f95 x:0 0 0 2024-04-20T10:59:19.279 INFO:tasks.workunit.client.0.smithi027.stdout:0/571: rename d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/cb6 to d0/d11/d2c/d2e/d61/cc4 0 2024-04-20T10:59:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:0/572: fdatasync d0/d11/d2c/d2e/d2b/d7b/d34/f43 0 2024-04-20T10:59:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:0/573: fsync d0/d11/d2c/d2e/d2b/d7b/d12/f3e 0 2024-04-20T10:59:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:0/574: dread - d0/d11/d62/f81 zero size 2024-04-20T10:59:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:0/575: chown d0/d11/d2c/d2e/d2b/d7b/d34/l76 0 1 2024-04-20T10:59:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:0/576: fsync d0/d11/d2c/d2e/d35/fb5 0 2024-04-20T10:59:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:0/577: chown d0/d11/d2c/d2e/d2b/d7b/f27 1209714219 1 2024-04-20T10:59:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:8/575: mkdir d9/de/da3/dcd 0 2024-04-20T10:59:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:8/576: chown d9/de/d20/d48/d59/da0/dbd 100371 1 2024-04-20T10:59:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:7/450: symlink d4/d1b/d4a/d7a/l8b 0 2024-04-20T10:59:19.281 INFO:tasks.workunit.client.0.smithi027.stdout:0/578: truncate d0/d11/d2c/d2e/f1d 106533 0 2024-04-20T10:59:19.281 INFO:tasks.workunit.client.0.smithi027.stdout:0/579: write d0/d11/d2c/d2e/d2b/d7b/d12/fad [388219,124863] 0 2024-04-20T10:59:19.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/577: mknod d9/de/da3/cce 0 2024-04-20T10:59:19.281 INFO:tasks.workunit.client.0.smithi027.stdout:7/451: rmdir d4/d1b/d4a/d62 39 2024-04-20T10:59:19.282 INFO:tasks.workunit.client.0.smithi027.stdout:7/452: creat d4/d5/d8/d46/d77/d7c/f8c x:0 0 0 2024-04-20T10:59:19.286 INFO:tasks.workunit.client.0.smithi027.stdout:7/453: write d4/d5/d8/d3b/f3e [2466753,32369] 0 2024-04-20T10:59:19.290 INFO:tasks.workunit.client.0.smithi027.stdout:4/473: dwrite d0/d14/d6b/da3/d1b/d33/f35 [0,4194304] 0 2024-04-20T10:59:19.290 INFO:tasks.workunit.client.0.smithi027.stdout:1/631: dwrite d3/d7/d38/ddf/dcc/fd0 [0,4194304] 0 2024-04-20T10:59:19.292 INFO:tasks.workunit.client.0.smithi027.stdout:5/468: dwrite f17 [0,4194304] 0 2024-04-20T10:59:19.292 INFO:tasks.workunit.client.0.smithi027.stdout:8/578: read d9/de/da3/f30 [3417191,39163] 0 2024-04-20T10:59:19.292 INFO:tasks.workunit.client.0.smithi027.stdout:8/579: chown d9/de/d1e/d4a/d6d/d75/c94 207705148 1 2024-04-20T10:59:19.293 INFO:tasks.workunit.client.0.smithi027.stdout:3/527: dwrite d2/d80/f45 [0,4194304] 0 2024-04-20T10:59:19.293 INFO:tasks.workunit.client.0.smithi027.stdout:3/528: chown d2/d18/d2e/c71 140485140 1 2024-04-20T10:59:19.293 INFO:tasks.workunit.client.0.smithi027.stdout:3/529: readlink d2/dd/df/d23/d68/l9b 0 2024-04-20T10:59:19.294 INFO:tasks.workunit.client.0.smithi027.stdout:4/474: link d0/d48/l4f d0/d14/d6b/da3/d53/d79/lb7 0 2024-04-20T10:59:19.295 INFO:tasks.workunit.client.0.smithi027.stdout:7/454: dread d4/d1b/f35 [0,4194304] 0 2024-04-20T10:59:19.296 INFO:tasks.workunit.client.0.smithi027.stdout:5/469: dread d19/d1b/d1c/d1e/d58/f61 [0,4194304] 0 2024-04-20T10:59:19.297 INFO:tasks.workunit.client.0.smithi027.stdout:1/632: creat d3/d45/fed x:0 0 0 2024-04-20T10:59:19.300 INFO:tasks.workunit.client.0.smithi027.stdout:0/580: getdents d0/d11/d4b/d5b/d6f 0 2024-04-20T10:59:19.300 INFO:tasks.workunit.client.0.smithi027.stdout:0/581: fsync d0/d59/d9e/fb7 0 2024-04-20T10:59:19.301 INFO:tasks.workunit.client.0.smithi027.stdout:2/450: getdents d5/dc/d16/d2a/d59 0 2024-04-20T10:59:19.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/530: mknod d2/dd/d22/ca8 0 2024-04-20T10:59:19.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/531: readlink d2/d18/d2c/d4c/l4d 0 2024-04-20T10:59:19.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/532: stat d2/dd/f10 0 2024-04-20T10:59:19.307 INFO:tasks.workunit.client.0.smithi027.stdout:4/475: mknod d0/d14/d6b/da3/d1b/d33/cb8 0 2024-04-20T10:59:19.307 INFO:tasks.workunit.client.0.smithi027.stdout:4/476: chown d0/d48/d68/l75 31 1 2024-04-20T10:59:19.310 INFO:tasks.workunit.client.0.smithi027.stdout:7/455: rename d4/c4e to d4/d5/d81/c8d 0 2024-04-20T10:59:19.312 INFO:tasks.workunit.client.0.smithi027.stdout:1/633: rename d3/d7/d38/da4/fc1 to d3/d45/fee 0 2024-04-20T10:59:19.314 INFO:tasks.workunit.client.0.smithi027.stdout:0/582: rename d0/d11/d5c/laf to d0/d11/d4b/d5b/d6f/d82/lc5 0 2024-04-20T10:59:19.315 INFO:tasks.workunit.client.0.smithi027.stdout:0/583: mknod d0/d11/d2c/d2e/d35/d8c/cc6 0 2024-04-20T10:59:19.315 INFO:tasks.workunit.client.0.smithi027.stdout:5/470: dwrite d19/d1b/f8d [4194304,4194304] 0 2024-04-20T10:59:19.317 INFO:tasks.workunit.client.0.smithi027.stdout:0/584: symlink d0/d59/dbf/lc7 0 2024-04-20T10:59:19.317 INFO:tasks.workunit.client.0.smithi027.stdout:3/533: truncate d2/d80/f45 938210 0 2024-04-20T10:59:19.318 INFO:tasks.workunit.client.0.smithi027.stdout:5/471: mkdir d19/d1b/d1c/d9c 0 2024-04-20T10:59:19.319 INFO:tasks.workunit.client.0.smithi027.stdout:0/585: symlink d0/d59/lc8 0 2024-04-20T10:59:19.320 INFO:tasks.workunit.client.0.smithi027.stdout:5/472: mkdir d19/d1b/d1c/d1e/d58/d9d 0 2024-04-20T10:59:19.320 INFO:tasks.workunit.client.0.smithi027.stdout:5/473: chown d19/d1b/d1c/f6a 1365457 1 2024-04-20T10:59:19.321 INFO:tasks.workunit.client.0.smithi027.stdout:0/586: truncate d0/d11/d2c/d2e/d2b/d7b/d64/f93 689805 0 2024-04-20T10:59:19.321 INFO:tasks.workunit.client.0.smithi027.stdout:9/413: sync 2024-04-20T10:59:19.321 INFO:tasks.workunit.client.0.smithi027.stdout:6/504: sync 2024-04-20T10:59:19.321 INFO:tasks.workunit.client.0.smithi027.stdout:9/414: dread - d4/d1b/d31/d23/d29/f70 zero size 2024-04-20T10:59:19.322 INFO:tasks.workunit.client.0.smithi027.stdout:0/587: creat d0/d11/d4b/d5b/d6f/d82/fc9 x:0 0 0 2024-04-20T10:59:19.322 INFO:tasks.workunit.client.0.smithi027.stdout:5/474: dread d19/d1b/f86 [0,4194304] 0 2024-04-20T10:59:19.325 INFO:tasks.workunit.client.0.smithi027.stdout:0/588: unlink d0/d11/f9a 0 2024-04-20T10:59:19.325 INFO:tasks.workunit.client.0.smithi027.stdout:0/589: chown d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f 364 1 2024-04-20T10:59:19.325 INFO:tasks.workunit.client.0.smithi027.stdout:0/590: chown d0/d11/d2c 12548 1 2024-04-20T10:59:19.325 INFO:tasks.workunit.client.0.smithi027.stdout:0/591: write d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/f8a [1590119,115515] 0 2024-04-20T10:59:19.325 INFO:tasks.workunit.client.0.smithi027.stdout:9/415: symlink d4/d2a/l87 0 2024-04-20T10:59:19.325 INFO:tasks.workunit.client.0.smithi027.stdout:9/416: stat d4/d1b/f22 0 2024-04-20T10:59:19.330 INFO:tasks.workunit.client.0.smithi027.stdout:8/580: dwrite d9/de/d1e/f66 [4194304,4194304] 0 2024-04-20T10:59:19.330 INFO:tasks.workunit.client.0.smithi027.stdout:8/581: dread - d9/de/d1e/dc1/fcc zero size 2024-04-20T10:59:19.330 INFO:tasks.workunit.client.0.smithi027.stdout:8/582: chown d9/de/d1e/d4a/d6d/d79/fba 973327706 1 2024-04-20T10:59:19.331 INFO:tasks.workunit.client.0.smithi027.stdout:8/583: fdatasync d9/de/d1e/dc1/fcc 0 2024-04-20T10:59:19.331 INFO:tasks.workunit.client.0.smithi027.stdout:6/505: dread da/dd/d3d/f7f [0,4194304] 0 2024-04-20T10:59:19.331 INFO:tasks.workunit.client.0.smithi027.stdout:6/506: chown da/df/d7e 12 1 2024-04-20T10:59:19.331 INFO:tasks.workunit.client.0.smithi027.stdout:6/507: stat da/df/f17 0 2024-04-20T10:59:19.332 INFO:tasks.workunit.client.0.smithi027.stdout:2/451: dwrite d5/d44/f65 [0,4194304] 0 2024-04-20T10:59:19.335 INFO:tasks.workunit.client.0.smithi027.stdout:8/584: symlink d9/de/d20/d48/d62/lcf 0 2024-04-20T10:59:19.335 INFO:tasks.workunit.client.0.smithi027.stdout:8/585: truncate d9/de/da3/d3c/f6f 5129702 0 2024-04-20T10:59:19.335 INFO:tasks.workunit.client.0.smithi027.stdout:8/586: chown d9/de/d64/d6a 65659 1 2024-04-20T10:59:19.335 INFO:tasks.workunit.client.0.smithi027.stdout:6/508: rmdir da/d21/d28/d2d/d53 39 2024-04-20T10:59:19.335 INFO:tasks.workunit.client.0.smithi027.stdout:9/417: read d4/d1b/f22 [3518289,79394] 0 2024-04-20T10:59:19.335 INFO:tasks.workunit.client.0.smithi027.stdout:9/418: write d4/f1e [4423042,65855] 0 2024-04-20T10:59:19.335 INFO:tasks.workunit.client.0.smithi027.stdout:2/452: symlink d5/dc/d16/d2a/d4a/l96 0 2024-04-20T10:59:19.337 INFO:tasks.workunit.client.0.smithi027.stdout:8/587: symlink d9/de/d1e/d4a/d6d/ld0 0 2024-04-20T10:59:19.338 INFO:tasks.workunit.client.0.smithi027.stdout:6/509: creat da/d21/d96/fa4 x:0 0 0 2024-04-20T10:59:19.338 INFO:tasks.workunit.client.0.smithi027.stdout:6/510: dread da/df/f4d [0,4194304] 0 2024-04-20T10:59:19.338 INFO:tasks.workunit.client.0.smithi027.stdout:6/511: chown da/df/d19/d2e/f89 78842 1 2024-04-20T10:59:19.338 INFO:tasks.workunit.client.0.smithi027.stdout:6/512: truncate da/df/f29 726973 0 2024-04-20T10:59:19.339 INFO:tasks.workunit.client.0.smithi027.stdout:5/475: write d19/d1b/d1c/d1e/d58/f5e [1989623,62362] 0 2024-04-20T10:59:19.339 INFO:tasks.workunit.client.0.smithi027.stdout:6/513: creat da/dd/d79/fa5 x:0 0 0 2024-04-20T10:59:19.339 INFO:tasks.workunit.client.0.smithi027.stdout:5/476: stat d19/c48 0 2024-04-20T10:59:19.341 INFO:tasks.workunit.client.0.smithi027.stdout:9/419: dread d4/d14/f21 [0,4194304] 0 2024-04-20T10:59:19.342 INFO:tasks.workunit.client.0.smithi027.stdout:6/514: link da/d21/d28/d2d/f43 da/d21/d28/d2d/fa6 0 2024-04-20T10:59:19.342 INFO:tasks.workunit.client.0.smithi027.stdout:5/477: creat d19/d1b/d1c/d9c/f9e x:0 0 0 2024-04-20T10:59:19.343 INFO:tasks.workunit.client.0.smithi027.stdout:1/634: dwrite d3/d7/f55 [4194304,4194304] 0 2024-04-20T10:59:19.343 INFO:tasks.workunit.client.0.smithi027.stdout:1/635: chown d3/d7/d28/db0 45158889 1 2024-04-20T10:59:19.343 INFO:tasks.workunit.client.0.smithi027.stdout:9/420: creat d4/f88 x:0 0 0 2024-04-20T10:59:19.345 INFO:tasks.workunit.client.0.smithi027.stdout:3/534: dwrite d2/d18/d2e/fa7 [0,4194304] 0 2024-04-20T10:59:19.346 INFO:tasks.workunit.client.0.smithi027.stdout:5/478: unlink d19/d1b/d1c/d1e/d25/d40/d82/f8f 0 2024-04-20T10:59:19.346 INFO:tasks.workunit.client.0.smithi027.stdout:0/592: write d0/d11/d2c/d2e/d2b/d7b/d12/f44 [1569809,16254] 0 2024-04-20T10:59:19.346 INFO:tasks.workunit.client.0.smithi027.stdout:0/593: dread d0/d11/d2c/d2e/d2b/d7b/d12/d1a/fa5 [0,4194304] 0 2024-04-20T10:59:19.348 INFO:tasks.workunit.client.0.smithi027.stdout:1/636: truncate d3/d45/fe7 2443660 0 2024-04-20T10:59:19.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/421: truncate d4/d2a/f69 2261974 0 2024-04-20T10:59:19.352 INFO:tasks.workunit.client.0.smithi027.stdout:3/535: mknod d2/dd/d57/d8d/d9d/ca9 0 2024-04-20T10:59:19.353 INFO:tasks.workunit.client.0.smithi027.stdout:5/479: truncate f0 4442132 0 2024-04-20T10:59:19.354 INFO:tasks.workunit.client.0.smithi027.stdout:2/453: dwrite d5/f4b [0,4194304] 0 2024-04-20T10:59:19.354 INFO:tasks.workunit.client.0.smithi027.stdout:0/594: link d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f2d d0/d11/d2c/d2e/d55/fca 0 2024-04-20T10:59:19.355 INFO:tasks.workunit.client.0.smithi027.stdout:3/536: dread d2/d18/d2c/d4c/f7e [0,4194304] 0 2024-04-20T10:59:19.356 INFO:tasks.workunit.client.0.smithi027.stdout:9/422: rename d4/d1b/d1f/d5a/f55 to d4/d1b/f89 0 2024-04-20T10:59:19.358 INFO:tasks.workunit.client.0.smithi027.stdout:2/454: symlink d5/dc/d16/d32/l97 0 2024-04-20T10:59:19.359 INFO:tasks.workunit.client.0.smithi027.stdout:0/595: creat d0/d11/d4b/fcb x:0 0 0 2024-04-20T10:59:19.360 INFO:tasks.workunit.client.0.smithi027.stdout:3/537: link d2/dd/f58 d2/dd/df/faa 0 2024-04-20T10:59:19.362 INFO:tasks.workunit.client.0.smithi027.stdout:9/423: read d4/d14/d43/f52 [2786129,99203] 0 2024-04-20T10:59:19.362 INFO:tasks.workunit.client.0.smithi027.stdout:9/424: fdatasync d4/d1b/d1f/f7c 0 2024-04-20T10:59:19.363 INFO:tasks.workunit.client.0.smithi027.stdout:0/596: creat d0/d11/d2c/d2e/d2b/d7b/d34/d5a/fcc x:0 0 0 2024-04-20T10:59:19.363 INFO:tasks.workunit.client.0.smithi027.stdout:0/597: write d0/d11/d2c/d2e/d55/fca [2177640,109809] 0 2024-04-20T10:59:19.363 INFO:tasks.workunit.client.0.smithi027.stdout:0/598: dread - d0/d11/fa9 zero size 2024-04-20T10:59:19.364 INFO:tasks.workunit.client.0.smithi027.stdout:6/515: dwrite da/df/d19/d2e/f89 [0,4194304] 0 2024-04-20T10:59:19.366 INFO:tasks.workunit.client.0.smithi027.stdout:5/480: rename d19/f4f to d19/d1b/d1c/d1e/d22/d47/f9f 0 2024-04-20T10:59:19.366 INFO:tasks.workunit.client.0.smithi027.stdout:5/481: write d19/d1b/d1c/d1e/f56 [4675573,87212] 0 2024-04-20T10:59:19.367 INFO:tasks.workunit.client.0.smithi027.stdout:1/637: dwrite d3/d7/f8 [4194304,4194304] 0 2024-04-20T10:59:19.368 INFO:tasks.workunit.client.0.smithi027.stdout:2/455: dread d5/d26/f2c [0,4194304] 0 2024-04-20T10:59:19.368 INFO:tasks.workunit.client.0.smithi027.stdout:0/599: creat d0/d11/d2c/fcd x:0 0 0 2024-04-20T10:59:19.372 INFO:tasks.workunit.client.0.smithi027.stdout:3/538: dread d2/f21 [0,4194304] 0 2024-04-20T10:59:19.372 INFO:tasks.workunit.client.0.smithi027.stdout:6/516: creat da/df/d7e/d86/da3/fa7 x:0 0 0 2024-04-20T10:59:19.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/425: rename d4/d1b/d31/d23/d29/d4b to d4/d1b/d1f/d45/d8a 0 2024-04-20T10:59:19.379 INFO:tasks.workunit.client.0.smithi027.stdout:2/456: mknod d5/c98 0 2024-04-20T10:59:19.379 INFO:tasks.workunit.client.0.smithi027.stdout:0/600: creat d0/d11/d2c/d2e/d2b/d7b/fce x:0 0 0 2024-04-20T10:59:19.380 INFO:tasks.workunit.client.0.smithi027.stdout:5/482: dread d19/d1b/d1c/d1e/f56 [0,4194304] 0 2024-04-20T10:59:19.380 INFO:tasks.workunit.client.0.smithi027.stdout:5/483: fdatasync f18 0 2024-04-20T10:59:19.387 INFO:tasks.workunit.client.0.smithi027.stdout:6/517: rmdir da/df/d19/d2e/d6b/d65 39 2024-04-20T10:59:19.387 INFO:tasks.workunit.client.0.smithi027.stdout:6/518: stat c9 0 2024-04-20T10:59:19.389 INFO:tasks.workunit.client.0.smithi027.stdout:2/457: rename d5/d44/d7c/c82 to d5/d26/d3d/c99 0 2024-04-20T10:59:19.393 INFO:tasks.workunit.client.0.smithi027.stdout:0/601: mkdir d0/d11/d2c/d2e/d2b/d7b/dcf 0 2024-04-20T10:59:19.394 INFO:tasks.workunit.client.0.smithi027.stdout:6/519: unlink da/d21/f70 0 2024-04-20T10:59:19.395 INFO:tasks.workunit.client.0.smithi027.stdout:6/520: fsync da/df/d19/f93 0 2024-04-20T10:59:19.396 INFO:tasks.workunit.client.0.smithi027.stdout:0/602: symlink d0/ld0 0 2024-04-20T10:59:19.397 INFO:tasks.workunit.client.0.smithi027.stdout:6/521: creat da/dd/d3c/fa8 x:0 0 0 2024-04-20T10:59:19.402 INFO:tasks.workunit.client.0.smithi027.stdout:0/603: mkdir d0/d11/d4b/d5b/d6f/d82/dd1 0 2024-04-20T10:59:19.402 INFO:tasks.workunit.client.0.smithi027.stdout:0/604: stat d0/d11/d5c/f74 0 2024-04-20T10:59:19.402 INFO:tasks.workunit.client.0.smithi027.stdout:0/605: readlink d0/d11/d2c/d2e/d2b/d7b/d12/l29 0 2024-04-20T10:59:19.402 INFO:tasks.workunit.client.0.smithi027.stdout:0/606: truncate d0/d11/d4b/fcb 459430 0 2024-04-20T10:59:19.403 INFO:tasks.workunit.client.0.smithi027.stdout:6/522: truncate da/dd/d3d/f7f 1814408 0 2024-04-20T10:59:19.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/458: dread d5/d26/f3a [0,4194304] 0 2024-04-20T10:59:19.404 INFO:tasks.workunit.client.0.smithi027.stdout:0/607: creat d0/d59/dbf/fd2 x:0 0 0 2024-04-20T10:59:19.406 INFO:tasks.workunit.client.0.smithi027.stdout:3/539: dwrite d2/d18/f26 [0,4194304] 0 2024-04-20T10:59:19.406 INFO:tasks.workunit.client.0.smithi027.stdout:1/638: dwrite d3/d7/f8 [4194304,4194304] 0 2024-04-20T10:59:19.406 INFO:tasks.workunit.client.0.smithi027.stdout:1/639: chown d3/d20/c3e 244441 1 2024-04-20T10:59:19.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/523: rmdir da/df 39 2024-04-20T10:59:19.407 INFO:tasks.workunit.client.0.smithi027.stdout:9/426: dwrite d4/d14/f21 [0,4194304] 0 2024-04-20T10:59:19.407 INFO:tasks.workunit.client.0.smithi027.stdout:5/484: dwrite d19/d1b/d1c/d1e/d22/d47/f9f [0,4194304] 0 2024-04-20T10:59:19.410 INFO:tasks.workunit.client.0.smithi027.stdout:0/608: mkdir d0/d11/d4b/d5b/d6f/d82/dd3 0 2024-04-20T10:59:19.413 INFO:tasks.workunit.client.0.smithi027.stdout:9/427: link d4/l59 d4/d1b/d31/l8b 0 2024-04-20T10:59:19.418 INFO:tasks.workunit.client.0.smithi027.stdout:0/609: mknod d0/d11/d2c/d2e/d2b/d7b/d12/d25/cd4 0 2024-04-20T10:59:19.423 INFO:tasks.workunit.client.0.smithi027.stdout:6/524: creat da/df/d19/fa9 x:0 0 0 2024-04-20T10:59:19.424 INFO:tasks.workunit.client.0.smithi027.stdout:5/485: dread d19/d1b/d1c/d1e/d22/d47/f9f [0,4194304] 0 2024-04-20T10:59:19.424 INFO:tasks.workunit.client.0.smithi027.stdout:9/428: creat d4/d1b/d50/d5d/f8c x:0 0 0 2024-04-20T10:59:19.424 INFO:tasks.workunit.client.0.smithi027.stdout:1/640: dread d3/d7/d28/f59 [0,4194304] 0 2024-04-20T10:59:19.424 INFO:tasks.workunit.client.0.smithi027.stdout:2/459: dwrite d5/d26/d47/f5f [0,4194304] 0 2024-04-20T10:59:19.424 INFO:tasks.workunit.client.0.smithi027.stdout:3/540: dwrite d2/dd/d57/f97 [0,4194304] 0 2024-04-20T10:59:19.424 INFO:tasks.workunit.client.0.smithi027.stdout:6/525: link da/df/l72 da/d21/d28/d2d/d53/laa 0 2024-04-20T10:59:19.424 INFO:tasks.workunit.client.0.smithi027.stdout:5/486: creat d19/d1b/d1c/d1e/d25/fa0 x:0 0 0 2024-04-20T10:59:19.424 INFO:tasks.workunit.client.0.smithi027.stdout:5/487: write d19/d1b/d1c/d1e/d58/f60 [38476,124478] 0 2024-04-20T10:59:19.426 INFO:tasks.workunit.client.0.smithi027.stdout:9/429: dread d4/d14/f44 [0,4194304] 0 2024-04-20T10:59:19.426 INFO:tasks.workunit.client.0.smithi027.stdout:9/430: stat d4/d14/l7e 0 2024-04-20T10:59:19.427 INFO:tasks.workunit.client.0.smithi027.stdout:2/460: rmdir d5/dc/d16/d2a/d89 0 2024-04-20T10:59:19.429 INFO:tasks.workunit.client.0.smithi027.stdout:1/641: mknod d3/d45/cef 0 2024-04-20T10:59:19.429 INFO:tasks.workunit.client.0.smithi027.stdout:1/642: chown d3/d45/d5d/la8 20 1 2024-04-20T10:59:19.429 INFO:tasks.workunit.client.0.smithi027.stdout:1/643: dread - d3/d7/d38/ddf/f84 zero size 2024-04-20T10:59:19.429 INFO:tasks.workunit.client.0.smithi027.stdout:1/644: fdatasync d3/d7/d38/ddf/d73/f98 0 2024-04-20T10:59:19.429 INFO:tasks.workunit.client.0.smithi027.stdout:3/541: write d2/dd/d57/f97 [1551149,109812] 0 2024-04-20T10:59:19.430 INFO:tasks.workunit.client.0.smithi027.stdout:5/488: mknod d19/d1b/d1c/d1e/d22/d47/ca1 0 2024-04-20T10:59:19.431 INFO:tasks.workunit.client.0.smithi027.stdout:9/431: write d4/d14/f44 [1593634,51714] 0 2024-04-20T10:59:19.432 INFO:tasks.workunit.client.0.smithi027.stdout:1/645: link d3/d20/d85/l9e d3/d7/d38/da4/lf0 0 2024-04-20T10:59:19.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/477: sync 2024-04-20T10:59:19.433 INFO:tasks.workunit.client.0.smithi027.stdout:8/588: sync 2024-04-20T10:59:19.433 INFO:tasks.workunit.client.0.smithi027.stdout:7/456: sync 2024-04-20T10:59:19.433 INFO:tasks.workunit.client.0.smithi027.stdout:7/457: dread - d4/d5/d8/d21/f6e zero size 2024-04-20T10:59:19.433 INFO:tasks.workunit.client.0.smithi027.stdout:3/542: creat d2/d18/d2c/d4c/d67/fab x:0 0 0 2024-04-20T10:59:19.435 INFO:tasks.workunit.client.0.smithi027.stdout:6/526: write da/d21/d28/f3a [1143181,104791] 0 2024-04-20T10:59:19.435 INFO:tasks.workunit.client.0.smithi027.stdout:6/527: stat da/dd 0 2024-04-20T10:59:19.437 INFO:tasks.workunit.client.0.smithi027.stdout:9/432: unlink d4/d14/f19 0 2024-04-20T10:59:19.438 INFO:tasks.workunit.client.0.smithi027.stdout:1/646: creat d3/d7/d28/ff1 x:0 0 0 2024-04-20T10:59:19.439 INFO:tasks.workunit.client.0.smithi027.stdout:1/647: dread d3/d7/d28/f70 [0,4194304] 0 2024-04-20T10:59:19.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/478: rmdir d0/d14/d6b/da3/d1a/dae 39 2024-04-20T10:59:19.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/479: fdatasync d0/d14/d6b/da3/d1b/d2f/f3f 0 2024-04-20T10:59:19.441 INFO:tasks.workunit.client.0.smithi027.stdout:8/589: mkdir d9/de/d1e/dc1/dd1 0 2024-04-20T10:59:19.443 INFO:tasks.workunit.client.0.smithi027.stdout:6/528: rename da/df to da/d21/dab 0 2024-04-20T10:59:19.444 INFO:tasks.workunit.client.0.smithi027.stdout:9/433: symlink d4/d1b/d1f/l8d 0 2024-04-20T10:59:19.446 INFO:tasks.workunit.client.0.smithi027.stdout:4/480: mkdir d0/d14/d6b/da3/d18/db9 0 2024-04-20T10:59:19.446 INFO:tasks.workunit.client.0.smithi027.stdout:4/481: chown d0/d14/d6b/da3/d18/f4b 2119 1 2024-04-20T10:59:19.447 INFO:tasks.workunit.client.0.smithi027.stdout:8/590: creat d9/de/fd2 x:0 0 0 2024-04-20T10:59:19.449 INFO:tasks.workunit.client.0.smithi027.stdout:1/648: rename d3/d20/fbd to d3/d7e/ff2 0 2024-04-20T10:59:19.450 INFO:tasks.workunit.client.0.smithi027.stdout:6/529: creat da/d21/dab/d7e/d86/da3/fac x:0 0 0 2024-04-20T10:59:19.450 INFO:tasks.workunit.client.0.smithi027.stdout:4/482: getdents d0/d14/d6b/da3/d53/d6a 0 2024-04-20T10:59:19.451 INFO:tasks.workunit.client.0.smithi027.stdout:1/649: link d3/d20/lb7 d3/d45/d5d/lf3 0 2024-04-20T10:59:19.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/483: creat d0/d5e/fba x:0 0 0 2024-04-20T10:59:19.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/484: write d0/d14/d6b/da3/d1b/f30 [4226932,13394] 0 2024-04-20T10:59:19.459 INFO:tasks.workunit.client.0.smithi027.stdout:0/610: dwrite d0/d11/d2c/d2e/d2b/d3d/f84 [0,4194304] 0 2024-04-20T10:59:19.461 INFO:tasks.workunit.client.0.smithi027.stdout:1/650: getdents d3/d7/d38/ddf/d73 0 2024-04-20T10:59:19.462 INFO:tasks.workunit.client.0.smithi027.stdout:4/485: write d0/d14/d6b/da3/d1b/d33/f57 [3055207,32152] 0 2024-04-20T10:59:19.463 INFO:tasks.workunit.client.0.smithi027.stdout:8/591: dread d9/de/d1e/d4a/f45 [0,4194304] 0 2024-04-20T10:59:19.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/651: rmdir d3/d20 39 2024-04-20T10:59:19.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/652: fsync d3/d7/d28/f3d 0 2024-04-20T10:59:19.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/653: chown d3/d7/d28/db0/ce2 98 1 2024-04-20T10:59:19.464 INFO:tasks.workunit.client.0.smithi027.stdout:1/654: chown d3/d7/d38/c93 1 1 2024-04-20T10:59:19.465 INFO:tasks.workunit.client.0.smithi027.stdout:0/611: dread d0/d59/f6a [0,4194304] 0 2024-04-20T10:59:19.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/489: dwrite d19/d1b/d1c/d1e/d25/d40/f81 [0,4194304] 0 2024-04-20T10:59:19.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/490: fsync d19/d1b/d1c/d1e/d3c/f3d 0 2024-04-20T10:59:19.465 INFO:tasks.workunit.client.0.smithi027.stdout:4/486: rename d0/d14/d6b/da3/f1d to d0/d14/d6b/da3/d53/d6a/da1/db0/fbb 0 2024-04-20T10:59:19.465 INFO:tasks.workunit.client.0.smithi027.stdout:4/487: fdatasync d0/f37 0 2024-04-20T10:59:19.466 INFO:tasks.workunit.client.0.smithi027.stdout:8/592: creat d9/de/da3/d3c/fd3 x:0 0 0 2024-04-20T10:59:19.466 INFO:tasks.workunit.client.0.smithi027.stdout:8/593: write d9/de/d20/d48/d59/f99 [776315,29357] 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:1/655: rmdir d3/d7/d38/da4/dac/dd4 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:1/656: write d3/d7/d38/d62/fb9 [4393186,90207] 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:7/458: dwrite d4/f3d [0,4194304] 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:6/530: dread da/d21/f5b [0,4194304] 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:8/594: dread d9/de/d20/d48/d59/d74/fb9 [0,4194304] 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:8/595: stat d9/fc5 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:6/531: fsync da/d21/dab/d19/d2e/f89 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:8/596: write d9/de/da3/f7f [3754159,31493] 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:2/461: dwrite d5/d1e/d21/f33 [0,4194304] 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:6/532: write da/d21/dab/d19/d2e/d6b/f62 [4696621,118073] 0 2024-04-20T10:59:19.468 INFO:tasks.workunit.client.0.smithi027.stdout:6/533: write da/d21/d96/fa4 [239769,54708] 0 2024-04-20T10:59:19.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/488: rmdir d0/d14/d6b/da3/d1b/d33/d97 39 2024-04-20T10:59:19.469 INFO:tasks.workunit.client.0.smithi027.stdout:9/434: dwrite d4/d2a/f61 [0,4194304] 0 2024-04-20T10:59:19.470 INFO:tasks.workunit.client.0.smithi027.stdout:1/657: rename d3/d45/c6f to d3/d7/d38/da4/dac/cf4 0 2024-04-20T10:59:19.471 INFO:tasks.workunit.client.0.smithi027.stdout:1/658: chown d3/d7/d28/c97 16297 1 2024-04-20T10:59:19.475 INFO:tasks.workunit.client.0.smithi027.stdout:5/491: getdents d19/d1b/d1c/d1e/d22/d47 0 2024-04-20T10:59:19.478 INFO:tasks.workunit.client.0.smithi027.stdout:5/492: dread d19/d1b/d1c/d1e/d22/d47/f6b [0,4194304] 0 2024-04-20T10:59:19.478 INFO:tasks.workunit.client.0.smithi027.stdout:5/493: chown d19/d1b/d1c/d1e/d3c/f24 712747636 1 2024-04-20T10:59:19.481 INFO:tasks.workunit.client.0.smithi027.stdout:8/597: write d9/de/d20/d53/f77 [585291,83434] 0 2024-04-20T10:59:19.481 INFO:tasks.workunit.client.0.smithi027.stdout:4/489: rename d0/d14/d6b/da3/d1a/f70 to d0/d5e/d6c/fbc 0 2024-04-20T10:59:19.483 INFO:tasks.workunit.client.0.smithi027.stdout:1/659: rmdir d3/d7/d38 39 2024-04-20T10:59:19.484 INFO:tasks.workunit.client.0.smithi027.stdout:7/459: write d4/d1b/f72 [519575,54084] 0 2024-04-20T10:59:19.484 INFO:tasks.workunit.client.0.smithi027.stdout:7/460: dread - d4/d5/f6c zero size 2024-04-20T10:59:19.486 INFO:tasks.workunit.client.0.smithi027.stdout:5/494: creat d19/d1b/d1c/d1e/d25/d40/d78/fa2 x:0 0 0 2024-04-20T10:59:19.490 INFO:tasks.workunit.client.0.smithi027.stdout:4/490: dread d0/d14/d6b/da3/d1b/f21 [4194304,4194304] 0 2024-04-20T10:59:19.490 INFO:tasks.workunit.client.0.smithi027.stdout:4/491: write d0/d14/d6b/da3/d1b/f61 [5020121,39955] 0 2024-04-20T10:59:19.491 INFO:tasks.workunit.client.0.smithi027.stdout:8/598: symlink d9/de/d18/ld4 0 2024-04-20T10:59:19.493 INFO:tasks.workunit.client.0.smithi027.stdout:4/492: dread d0/d14/d6b/da3/d1b/f3d [0,4194304] 0 2024-04-20T10:59:19.495 INFO:tasks.workunit.client.0.smithi027.stdout:6/534: dwrite da/d21/d28/d2d/f43 [0,4194304] 0 2024-04-20T10:59:19.499 INFO:tasks.workunit.client.0.smithi027.stdout:2/462: dwrite d5/f58 [4194304,4194304] 0 2024-04-20T10:59:19.501 INFO:tasks.workunit.client.0.smithi027.stdout:0/612: dwrite d0/d11/d2c/d2e/d61/f78 [0,4194304] 0 2024-04-20T10:59:19.501 INFO:tasks.workunit.client.0.smithi027.stdout:8/599: dread d9/de/d20/d53/f77 [0,4194304] 0 2024-04-20T10:59:19.502 INFO:tasks.workunit.client.0.smithi027.stdout:0/613: chown d0/d11/d2c/d2e/d35/fb5 1786516599 1 2024-04-20T10:59:19.504 INFO:tasks.workunit.client.0.smithi027.stdout:1/660: stat d3/d45/l60 0 2024-04-20T10:59:19.505 INFO:tasks.workunit.client.0.smithi027.stdout:3/543: write d2/d18/d2c/f69 [2456448,39426] 0 2024-04-20T10:59:19.507 INFO:tasks.workunit.client.0.smithi027.stdout:8/600: creat d9/de/fd5 x:0 0 0 2024-04-20T10:59:19.508 INFO:tasks.workunit.client.0.smithi027.stdout:0/614: symlink d0/d59/d9e/ld5 0 2024-04-20T10:59:19.508 INFO:tasks.workunit.client.0.smithi027.stdout:0/615: chown d0/d11/d62/f81 555349 1 2024-04-20T10:59:19.510 INFO:tasks.workunit.client.0.smithi027.stdout:1/661: rename d3/d7/la6 to d3/d41/dc6/lf5 0 2024-04-20T10:59:19.510 INFO:tasks.workunit.client.0.smithi027.stdout:3/544: getdents d2/dd/d57/d8d/d9d 0 2024-04-20T10:59:19.510 INFO:tasks.workunit.client.0.smithi027.stdout:2/463: dread d5/dc/d16/d2a/d59/f81 [0,4194304] 0 2024-04-20T10:59:19.512 INFO:tasks.workunit.client.0.smithi027.stdout:1/662: creat d3/d7/ff6 x:0 0 0 2024-04-20T10:59:19.513 INFO:tasks.workunit.client.0.smithi027.stdout:0/616: rmdir d0/d11/d4b/d5b/d6f 39 2024-04-20T10:59:19.513 INFO:tasks.workunit.client.0.smithi027.stdout:0/617: chown d0/d11/d2c/d2e/d2b/d7b/d12/d25/l9b 407497798 1 2024-04-20T10:59:19.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/464: creat d5/d26/d47/d8a/f9a x:0 0 0 2024-04-20T10:59:19.517 INFO:tasks.workunit.client.0.smithi027.stdout:5/495: dread d19/d1b/d1c/d1e/d58/f61 [0,4194304] 0 2024-04-20T10:59:19.520 INFO:tasks.workunit.client.0.smithi027.stdout:9/435: dwrite d4/d2a/f69 [0,4194304] 0 2024-04-20T10:59:19.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/663: getdents d3/d7/d38 0 2024-04-20T10:59:19.522 INFO:tasks.workunit.client.0.smithi027.stdout:7/461: dwrite d4/d1b/f35 [4194304,4194304] 0 2024-04-20T10:59:19.523 INFO:tasks.workunit.client.0.smithi027.stdout:7/462: chown d4/d5/d8/f4c 0 1 2024-04-20T10:59:19.523 INFO:tasks.workunit.client.0.smithi027.stdout:4/493: dwrite d0/d14/d6b/da3/f2e [0,4194304] 0 2024-04-20T10:59:19.523 INFO:tasks.workunit.client.0.smithi027.stdout:0/618: rename d0/d11/d2c/d2e/d2b/d7b/l87 to d0/d11/d4b/ld6 0 2024-04-20T10:59:19.524 INFO:tasks.workunit.client.0.smithi027.stdout:6/535: dwrite da/d21/dab/d19/f74 [0,4194304] 0 2024-04-20T10:59:19.524 INFO:tasks.workunit.client.0.smithi027.stdout:6/536: truncate da/dd/d79/fa5 389268 0 2024-04-20T10:59:19.527 INFO:tasks.workunit.client.0.smithi027.stdout:7/463: link d4/d5/d8/d21/f31 d4/d5/d81/f8e 0 2024-04-20T10:59:19.531 INFO:tasks.workunit.client.0.smithi027.stdout:0/619: mkdir d0/d11/d2c/d2e/d2b/d7b/d12/dd7 0 2024-04-20T10:59:19.532 INFO:tasks.workunit.client.0.smithi027.stdout:0/620: chown d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f 19754 1 2024-04-20T10:59:19.532 INFO:tasks.workunit.client.0.smithi027.stdout:0/621: fdatasync d0/d11/d2c/d2e/d55/f6b 0 2024-04-20T10:59:19.534 INFO:tasks.workunit.client.0.smithi027.stdout:9/436: rename d4/d1b/c48 to d4/d1b/d31/d23/d29/c8e 0 2024-04-20T10:59:19.536 INFO:tasks.workunit.client.0.smithi027.stdout:7/464: creat d4/d5/d8/d46/f8f x:0 0 0 2024-04-20T10:59:19.536 INFO:tasks.workunit.client.0.smithi027.stdout:7/465: readlink d4/d5/d8/l36 0 2024-04-20T10:59:19.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/545: dwrite d2/d80/f5a [0,4194304] 0 2024-04-20T10:59:19.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/546: truncate d2/dd/d22/fa0 66570 0 2024-04-20T10:59:19.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/547: chown d2/d18/d2c/f78 2125545686 1 2024-04-20T10:59:19.549 INFO:tasks.workunit.client.0.smithi027.stdout:5/496: dwrite d19/d1b/d1c/d1e/d3c/f31 [4194304,4194304] 0 2024-04-20T10:59:19.549 INFO:tasks.workunit.client.0.smithi027.stdout:5/497: fdatasync d19/d1b/d1c/d1e/d3c/f3d 0 2024-04-20T10:59:19.549 INFO:tasks.workunit.client.0.smithi027.stdout:1/664: dwrite d3/d7/d28/f48 [0,4194304] 0 2024-04-20T10:59:19.550 INFO:tasks.workunit.client.0.smithi027.stdout:1/665: chown d3/d7/d28/db0/ce2 18 1 2024-04-20T10:59:19.550 INFO:tasks.workunit.client.0.smithi027.stdout:6/537: rename da/dd/d3d/f8c to da/d21/d28/fad 0 2024-04-20T10:59:19.550 INFO:tasks.workunit.client.0.smithi027.stdout:6/538: dread - da/f99 zero size 2024-04-20T10:59:19.551 INFO:tasks.workunit.client.0.smithi027.stdout:7/466: getdents d4/d1b/d4a/d7a 0 2024-04-20T10:59:19.555 INFO:tasks.workunit.client.0.smithi027.stdout:3/548: rmdir d2/dd/df/d33 39 2024-04-20T10:59:19.556 INFO:tasks.workunit.client.0.smithi027.stdout:5/498: mkdir d19/d1b/d1c/d1e/da3 0 2024-04-20T10:59:19.556 INFO:tasks.workunit.client.0.smithi027.stdout:8/601: truncate d9/de/d20/d48/d59/f99 619599 0 2024-04-20T10:59:19.557 INFO:tasks.workunit.client.0.smithi027.stdout:1/666: getdents d3/d45/d5d 0 2024-04-20T10:59:19.557 INFO:tasks.workunit.client.0.smithi027.stdout:5/499: dread d19/d1b/d1c/d1e/d3c/f39 [0,4194304] 0 2024-04-20T10:59:19.557 INFO:tasks.workunit.client.0.smithi027.stdout:7/467: write d4/d5/d8/d3b/f74 [1025322,116682] 0 2024-04-20T10:59:19.558 INFO:tasks.workunit.client.0.smithi027.stdout:9/437: dwrite d4/f88 [0,4194304] 0 2024-04-20T10:59:19.559 INFO:tasks.workunit.client.0.smithi027.stdout:0/622: rename d0/d11/d2c/d2e/d2b/d7b/d34/c3c to d0/d11/d4b/d5b/cd8 0 2024-04-20T10:59:19.559 INFO:tasks.workunit.client.0.smithi027.stdout:6/539: mkdir da/d21/d28/d2d/d53/dae 0 2024-04-20T10:59:19.560 INFO:tasks.workunit.client.0.smithi027.stdout:6/540: stat da/d21/dab/d19/d2e/f38 0 2024-04-20T10:59:19.561 INFO:tasks.workunit.client.0.smithi027.stdout:8/602: unlink d9/de/f47 0 2024-04-20T10:59:19.561 INFO:tasks.workunit.client.0.smithi027.stdout:8/603: getdents d9/de/d64/d6c/dc4 0 2024-04-20T10:59:19.561 INFO:tasks.workunit.client.0.smithi027.stdout:8/604: chown d9/de/d18 952490366 1 2024-04-20T10:59:19.564 INFO:tasks.workunit.client.0.smithi027.stdout:9/438: mkdir d4/d1b/d8f 0 2024-04-20T10:59:19.564 INFO:tasks.workunit.client.0.smithi027.stdout:9/439: fdatasync d4/d1b/d1f/d45/f71 0 2024-04-20T10:59:19.565 INFO:tasks.workunit.client.0.smithi027.stdout:9/440: truncate d4/d1b/d31/f3d 110109 0 2024-04-20T10:59:19.565 INFO:tasks.workunit.client.0.smithi027.stdout:9/441: readlink d4/d1b/d31/d23/d29/d7f/l84 0 2024-04-20T10:59:19.566 INFO:tasks.workunit.client.0.smithi027.stdout:0/623: rename d0/d11/d2c/d2e/d2b/d7b/d12/d25/c49 to d0/d11/d2c/d2e/d2b/d3d/cd9 0 2024-04-20T10:59:19.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/442: dread d4/d1b/d1f/d45/f3f [0,4194304] 0 2024-04-20T10:59:19.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/443: write d4/d1b/d31/d23/d29/f82 [1031973,73243] 0 2024-04-20T10:59:19.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/444: dread - d4/d1b/d31/f67 zero size 2024-04-20T10:59:19.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/445: stat d4/d1b/d1f/d45/d8a/l5e 0 2024-04-20T10:59:19.566 INFO:tasks.workunit.client.0.smithi027.stdout:3/549: dread d2/d18/f53 [0,4194304] 0 2024-04-20T10:59:19.567 INFO:tasks.workunit.client.0.smithi027.stdout:6/541: mkdir da/d21/dab/d7e/daf 0 2024-04-20T10:59:19.568 INFO:tasks.workunit.client.0.smithi027.stdout:8/605: mkdir d9/de/d1e/d4a/d6d/d7d/dd6 0 2024-04-20T10:59:19.568 INFO:tasks.workunit.client.0.smithi027.stdout:0/624: creat d0/d11/d2c/fda x:0 0 0 2024-04-20T10:59:19.569 INFO:tasks.workunit.client.0.smithi027.stdout:0/625: write d0/d11/d2c/d2e/d2b/d3d/f84 [4465091,10729] 0 2024-04-20T10:59:19.569 INFO:tasks.workunit.client.0.smithi027.stdout:9/446: chown d4/cd 1660362594 1 2024-04-20T10:59:19.569 INFO:tasks.workunit.client.0.smithi027.stdout:5/500: read d19/d1b/d1c/d1e/d3c/f24 [2919602,47496] 0 2024-04-20T10:59:19.569 INFO:tasks.workunit.client.0.smithi027.stdout:7/468: write d4/d5/f40 [1402935,20791] 0 2024-04-20T10:59:19.569 INFO:tasks.workunit.client.0.smithi027.stdout:7/469: truncate d4/d1b/d58/f75 459771 0 2024-04-20T10:59:19.569 INFO:tasks.workunit.client.0.smithi027.stdout:3/550: creat d2/d18/fac x:0 0 0 2024-04-20T10:59:19.571 INFO:tasks.workunit.client.0.smithi027.stdout:6/542: rename da/fb to da/d21/dab/d7e/d86/fb0 0 2024-04-20T10:59:19.572 INFO:tasks.workunit.client.0.smithi027.stdout:8/606: creat d9/de/da3/d3c/fd7 x:0 0 0 2024-04-20T10:59:19.573 INFO:tasks.workunit.client.0.smithi027.stdout:0/626: symlink d0/d11/d2c/d2e/d2b/d7b/d12/dd7/ldb 0 2024-04-20T10:59:19.575 INFO:tasks.workunit.client.0.smithi027.stdout:9/447: unlink d4/d1b/d31/d23/d29/f70 0 2024-04-20T10:59:19.576 INFO:tasks.workunit.client.0.smithi027.stdout:5/501: symlink d19/d1b/d1c/d1e/d25/d40/d78/d91/la4 0 2024-04-20T10:59:19.576 INFO:tasks.workunit.client.0.smithi027.stdout:5/502: truncate d19/d1b/d1c/d9c/f9e 945584 0 2024-04-20T10:59:19.577 INFO:tasks.workunit.client.0.smithi027.stdout:7/470: mknod d4/d5/d8/d45/c90 0 2024-04-20T10:59:19.578 INFO:tasks.workunit.client.0.smithi027.stdout:6/543: rmdir da/d21/d28/d54 39 2024-04-20T10:59:19.580 INFO:tasks.workunit.client.0.smithi027.stdout:8/607: rmdir d9 39 2024-04-20T10:59:19.581 INFO:tasks.workunit.client.0.smithi027.stdout:0/627: symlink d0/d11/d4b/d5b/d6f/ldc 0 2024-04-20T10:59:19.581 INFO:tasks.workunit.client.0.smithi027.stdout:0/628: truncate d0/d11/d2c/d2e/d2b/d7b/f27 1195395 0 2024-04-20T10:59:19.585 INFO:tasks.workunit.client.0.smithi027.stdout:7/471: creat d4/d4d/f91 x:0 0 0 2024-04-20T10:59:19.591 INFO:tasks.workunit.client.0.smithi027.stdout:3/551: dwrite d2/d18/d2c/f78 [0,4194304] 0 2024-04-20T10:59:19.591 INFO:tasks.workunit.client.0.smithi027.stdout:5/503: dread d19/d1b/d1c/d1e/d25/d40/f81 [0,4194304] 0 2024-04-20T10:59:19.591 INFO:tasks.workunit.client.0.smithi027.stdout:6/544: truncate da/d21/dab/d19/d2e/f41 1184430 0 2024-04-20T10:59:19.591 INFO:tasks.workunit.client.0.smithi027.stdout:6/545: chown da/d21/dab/c23 32 1 2024-04-20T10:59:19.592 INFO:tasks.workunit.client.0.smithi027.stdout:8/608: mknod d9/de/d1e/d4a/d6d/db7/cd8 0 2024-04-20T10:59:19.597 INFO:tasks.workunit.client.0.smithi027.stdout:5/504: dread d19/d1b/d1c/d1e/d22/f30 [0,4194304] 0 2024-04-20T10:59:19.603 INFO:tasks.workunit.client.0.smithi027.stdout:3/552: getdents d2/d18/d2c/d4c/d67 0 2024-04-20T10:59:19.604 INFO:tasks.workunit.client.0.smithi027.stdout:7/472: dwrite d4/d5b/f7e [0,4194304] 0 2024-04-20T10:59:19.604 INFO:tasks.workunit.client.0.smithi027.stdout:6/546: link da/dd/d3d/c47 da/d21/dab/d7e/daf/cb1 0 2024-04-20T10:59:19.605 INFO:tasks.workunit.client.0.smithi027.stdout:5/505: creat d19/d1b/d1c/d1e/d25/fa5 x:0 0 0 2024-04-20T10:59:19.606 INFO:tasks.workunit.client.0.smithi027.stdout:3/553: creat d2/fad x:0 0 0 2024-04-20T10:59:19.609 INFO:tasks.workunit.client.0.smithi027.stdout:5/506: rename d19/d1b/d1c/d1e/d3c/l21 to d19/d1b/d1c/d1e/d25/la6 0 2024-04-20T10:59:19.609 INFO:tasks.workunit.client.0.smithi027.stdout:5/507: chown d19/d1b/d1c/c66 10 1 2024-04-20T10:59:19.610 INFO:tasks.workunit.client.0.smithi027.stdout:5/508: write d19/d1b/f86 [192908,50946] 0 2024-04-20T10:59:19.611 INFO:tasks.workunit.client.0.smithi027.stdout:3/554: link d2/d18/d2c/d4c/d67/f6d d2/dd/d22/fae 0 2024-04-20T10:59:19.623 INFO:tasks.workunit.client.0.smithi027.stdout:6/547: dwrite da/d21/dab/d19/f6a [0,4194304] 0 2024-04-20T10:59:19.624 INFO:tasks.workunit.client.0.smithi027.stdout:7/473: mknod d4/c92 0 2024-04-20T10:59:19.625 INFO:tasks.workunit.client.0.smithi027.stdout:5/509: mknod d19/d1b/d1c/d9c/ca7 0 2024-04-20T10:59:19.625 INFO:tasks.workunit.client.0.smithi027.stdout:5/510: chown d19/d1b/d1c/d1e/d22/d47/f55 7187859 1 2024-04-20T10:59:19.625 INFO:tasks.workunit.client.0.smithi027.stdout:3/555: dread d2/dd/f10 [4194304,4194304] 0 2024-04-20T10:59:19.625 INFO:tasks.workunit.client.0.smithi027.stdout:6/548: creat da/d21/fb2 x:0 0 0 2024-04-20T10:59:19.625 INFO:tasks.workunit.client.0.smithi027.stdout:7/474: getdents d4/d5/d8/d46/d77/d7c 0 2024-04-20T10:59:19.628 INFO:tasks.workunit.client.0.smithi027.stdout:6/549: rename da/d21/d28/d2d/f43 to da/d21/dab/d19/d2e/d32/fb3 0 2024-04-20T10:59:19.628 INFO:tasks.workunit.client.0.smithi027.stdout:6/550: write da/dd/d3c/fa8 [223153,106706] 0 2024-04-20T10:59:19.628 INFO:tasks.workunit.client.0.smithi027.stdout:7/475: truncate d4/d5/f1f 1452217 0 2024-04-20T10:59:19.630 INFO:tasks.workunit.client.0.smithi027.stdout:5/511: unlink d19/c51 0 2024-04-20T10:59:19.634 INFO:tasks.workunit.client.0.smithi027.stdout:7/476: mkdir d4/d1b/d4a/d7a/d93 0 2024-04-20T10:59:19.637 INFO:tasks.workunit.client.0.smithi027.stdout:6/551: rename da/d21/dab/d19/d2e/f89 to da/d21/dab/d19/d2e/fb4 0 2024-04-20T10:59:19.637 INFO:tasks.workunit.client.0.smithi027.stdout:6/552: fdatasync da/d21/fb2 0 2024-04-20T10:59:19.638 INFO:tasks.workunit.client.0.smithi027.stdout:7/477: mkdir d4/d5/d8/d21/d94 0 2024-04-20T10:59:19.645 INFO:tasks.workunit.client.0.smithi027.stdout:3/556: dwrite d2/d18/f76 [0,4194304] 0 2024-04-20T10:59:19.647 INFO:tasks.workunit.client.0.smithi027.stdout:6/553: unlink da/d21/dab/l37 0 2024-04-20T10:59:19.647 INFO:tasks.workunit.client.0.smithi027.stdout:6/554: dread - da/d21/d28/d54/f9f zero size 2024-04-20T10:59:19.647 INFO:tasks.workunit.client.0.smithi027.stdout:6/555: truncate da/d21/dab/d7e/f85 189376 0 2024-04-20T10:59:19.648 INFO:tasks.workunit.client.0.smithi027.stdout:7/478: rmdir d4/d5/d81 39 2024-04-20T10:59:19.656 INFO:tasks.workunit.client.0.smithi027.stdout:0/629: dwrite d0/d11/d2c/d2e/d2b/d7b/fce [0,4194304] 0 2024-04-20T10:59:19.656 INFO:tasks.workunit.client.0.smithi027.stdout:6/556: creat da/d21/dab/fb5 x:0 0 0 2024-04-20T10:59:19.657 INFO:tasks.workunit.client.0.smithi027.stdout:7/479: unlink d4/c34 0 2024-04-20T10:59:19.657 INFO:tasks.workunit.client.0.smithi027.stdout:7/480: dread - d4/d1b/d4a/d62/f56 zero size 2024-04-20T10:59:19.658 INFO:tasks.workunit.client.0.smithi027.stdout:0/630: rmdir d0/d59 39 2024-04-20T10:59:19.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/557: link da/dd/l50 da/d21/d28/d54/lb6 0 2024-04-20T10:59:19.660 INFO:tasks.workunit.client.0.smithi027.stdout:7/481: truncate d4/fc 2451443 0 2024-04-20T10:59:19.661 INFO:tasks.workunit.client.0.smithi027.stdout:0/631: rename d0/d11/d2c/d2e/d2b/d7b/d12/c39 to d0/d11/d2c/d2e/d2b/d7b/cdd 0 2024-04-20T10:59:19.663 INFO:tasks.workunit.client.0.smithi027.stdout:6/558: creat da/dd/d79/fb7 x:0 0 0 2024-04-20T10:59:19.663 INFO:tasks.workunit.client.0.smithi027.stdout:6/559: chown da/dd/d3c/d9e 112 1 2024-04-20T10:59:19.663 INFO:tasks.workunit.client.0.smithi027.stdout:6/560: truncate da/dd/d79/fb7 948864 0 2024-04-20T10:59:19.663 INFO:tasks.workunit.client.0.smithi027.stdout:6/561: chown da/d21/d28/d2d/d53/dae 849 1 2024-04-20T10:59:19.664 INFO:tasks.workunit.client.0.smithi027.stdout:7/482: mknod d4/d1b/d4a/d7a/c95 0 2024-04-20T10:59:19.666 INFO:tasks.workunit.client.0.smithi027.stdout:7/483: dread d4/d1b/ff [0,4194304] 0 2024-04-20T10:59:19.667 INFO:tasks.workunit.client.0.smithi027.stdout:0/632: rmdir d0/d11/d2c/d2e/d35/d8c 39 2024-04-20T10:59:19.667 INFO:tasks.workunit.client.0.smithi027.stdout:0/633: chown d0/d11/d2c/d2e/d2b/d7b/d12/d25/c33 33714 1 2024-04-20T10:59:19.672 INFO:tasks.workunit.client.0.smithi027.stdout:7/484: creat d4/d5/d8/d21/f96 x:0 0 0 2024-04-20T10:59:19.681 INFO:tasks.workunit.client.0.smithi027.stdout:0/634: dwrite d0/d11/d4b/d5b/d6f/d82/fc9 [0,4194304] 0 2024-04-20T10:59:19.682 INFO:tasks.workunit.client.0.smithi027.stdout:6/562: mkdir da/db8 0 2024-04-20T10:59:19.682 INFO:tasks.workunit.client.0.smithi027.stdout:7/485: rmdir d4/d5 39 2024-04-20T10:59:19.685 INFO:tasks.workunit.client.0.smithi027.stdout:9/448: sync 2024-04-20T10:59:19.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/465: sync 2024-04-20T10:59:19.685 INFO:tasks.workunit.client.0.smithi027.stdout:1/667: sync 2024-04-20T10:59:19.685 INFO:tasks.workunit.client.0.smithi027.stdout:4/494: sync 2024-04-20T10:59:19.685 INFO:tasks.workunit.client.0.smithi027.stdout:4/495: chown d0/d5e/d6c/fb2 23 1 2024-04-20T10:59:19.685 INFO:tasks.workunit.client.0.smithi027.stdout:4/496: fsync d0/d14/d6b/da3/d53/d6a/da1/db0/fbb 0 2024-04-20T10:59:19.686 INFO:tasks.workunit.client.0.smithi027.stdout:0/635: rename d0/d11/d2c/d2e/d2b/l91 to d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/lde 0 2024-04-20T10:59:19.686 INFO:tasks.workunit.client.0.smithi027.stdout:0/636: fsync d0/d59/d9e/fb7 0 2024-04-20T10:59:19.686 INFO:tasks.workunit.client.0.smithi027.stdout:7/486: mknod d4/d5/d8/c97 0 2024-04-20T10:59:19.687 INFO:tasks.workunit.client.0.smithi027.stdout:1/668: unlink d3/d41/c63 0 2024-04-20T10:59:19.688 INFO:tasks.workunit.client.0.smithi027.stdout:2/466: unlink d5/dc/l45 0 2024-04-20T10:59:19.688 INFO:tasks.workunit.client.0.smithi027.stdout:7/487: dread d4/d5/f1f [0,4194304] 0 2024-04-20T10:59:19.688 INFO:tasks.workunit.client.0.smithi027.stdout:7/488: write d4/d4d/f60 [546787,114471] 0 2024-04-20T10:59:19.688 INFO:tasks.workunit.client.0.smithi027.stdout:7/489: write d4/d5/d8/d3b/f42 [5133165,23711] 0 2024-04-20T10:59:19.689 INFO:tasks.workunit.client.0.smithi027.stdout:4/497: rename d0/lf to d0/d14/d6b/da3/d53/d6a/lbd 0 2024-04-20T10:59:19.690 INFO:tasks.workunit.client.0.smithi027.stdout:0/637: truncate d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f2d 99928 0 2024-04-20T10:59:19.690 INFO:tasks.workunit.client.0.smithi027.stdout:1/669: rmdir d3/d20/d85 39 2024-04-20T10:59:19.691 INFO:tasks.workunit.client.0.smithi027.stdout:7/490: creat d4/f98 x:0 0 0 2024-04-20T10:59:19.693 INFO:tasks.workunit.client.0.smithi027.stdout:2/467: rename d5/dc/d16/d2a/d4a/l96 to d5/d1e/l9b 0 2024-04-20T10:59:19.694 INFO:tasks.workunit.client.0.smithi027.stdout:4/498: truncate d0/f27 3746581 0 2024-04-20T10:59:19.694 INFO:tasks.workunit.client.0.smithi027.stdout:4/499: fdatasync d0/d14/d6b/da3/f16 0 2024-04-20T10:59:19.695 INFO:tasks.workunit.client.0.smithi027.stdout:1/670: creat d3/d7/d38/db8/ff7 x:0 0 0 2024-04-20T10:59:19.695 INFO:tasks.workunit.client.0.smithi027.stdout:1/671: chown d3/d7/c2a 3 1 2024-04-20T10:59:19.695 INFO:tasks.workunit.client.0.smithi027.stdout:1/672: stat d3/d7e/f83 0 2024-04-20T10:59:19.695 INFO:tasks.workunit.client.0.smithi027.stdout:1/673: readlink d3/d7/d38/ddf/l8f 0 2024-04-20T10:59:19.696 INFO:tasks.workunit.client.0.smithi027.stdout:7/491: creat d4/d1b/d4a/d7a/d93/f99 x:0 0 0 2024-04-20T10:59:19.696 INFO:tasks.workunit.client.0.smithi027.stdout:7/492: stat d4/d1b/c47 0 2024-04-20T10:59:19.699 INFO:tasks.workunit.client.0.smithi027.stdout:2/468: rmdir d5/dc/d16/d2a/d59 39 2024-04-20T10:59:19.699 INFO:tasks.workunit.client.0.smithi027.stdout:2/469: truncate d5/dc/d16/d37/f6b 416216 0 2024-04-20T10:59:19.699 INFO:tasks.workunit.client.0.smithi027.stdout:2/470: fdatasync d5/dc/d16/d2a/d4a/f94 0 2024-04-20T10:59:19.699 INFO:tasks.workunit.client.0.smithi027.stdout:2/471: chown d5/dc/d16/d2a/d4a/f50 216451037 1 2024-04-20T10:59:19.700 INFO:tasks.workunit.client.0.smithi027.stdout:4/500: truncate d0/d14/d6b/da3/f16 883502 0 2024-04-20T10:59:19.700 INFO:tasks.workunit.client.0.smithi027.stdout:4/501: write d0/d14/d6b/da3/d18/f72 [975185,118882] 0 2024-04-20T10:59:19.701 INFO:tasks.workunit.client.0.smithi027.stdout:9/449: dwrite d4/d1b/d31/d23/f7a [0,4194304] 0 2024-04-20T10:59:19.701 INFO:tasks.workunit.client.0.smithi027.stdout:9/450: fdatasync d4/d1b/d1f/d45/f71 0 2024-04-20T10:59:19.701 INFO:tasks.workunit.client.0.smithi027.stdout:9/451: write d4/d1b/d1f/d45/f83 [589971,95832] 0 2024-04-20T10:59:19.701 INFO:tasks.workunit.client.0.smithi027.stdout:9/452: write d4/d1b/d1f/f7c [1012221,106219] 0 2024-04-20T10:59:19.702 INFO:tasks.workunit.client.0.smithi027.stdout:3/557: write d2/d80/f45 [851926,16549] 0 2024-04-20T10:59:19.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/674: rename d3/d45/c7b to d3/d20/cf8 0 2024-04-20T10:59:19.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/675: read d3/d45/d5d/fd5 [1244691,120476] 0 2024-04-20T10:59:19.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/676: stat d3/c2c 0 2024-04-20T10:59:19.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/677: stat d3/d7/d38/ddf/l4f 0 2024-04-20T10:59:19.703 INFO:tasks.workunit.client.0.smithi027.stdout:7/493: mknod d4/d5/d8/c9a 0 2024-04-20T10:59:19.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/502: truncate d0/fb 6230858 0 2024-04-20T10:59:19.704 INFO:tasks.workunit.client.0.smithi027.stdout:4/503: dread - d0/d14/d6b/da3/d1b/d33/d5c/d6f/f74 zero size 2024-04-20T10:59:19.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/678: dread d3/d7/d38/d62/f6e [0,4194304] 0 2024-04-20T10:59:19.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/679: fdatasync d3/d7/d28/f3d 0 2024-04-20T10:59:19.707 INFO:tasks.workunit.client.0.smithi027.stdout:7/494: creat d4/d1b/d4a/d62/d87/f9b x:0 0 0 2024-04-20T10:59:19.710 INFO:tasks.workunit.client.0.smithi027.stdout:7/495: dread d4/d1b/f63 [0,4194304] 0 2024-04-20T10:59:19.710 INFO:tasks.workunit.client.0.smithi027.stdout:7/496: chown d4/d5/d8/c61 111 1 2024-04-20T10:59:19.713 INFO:tasks.workunit.client.0.smithi027.stdout:7/497: write d4/d5/d8/f38 [139783,81237] 0 2024-04-20T10:59:19.713 INFO:tasks.workunit.client.0.smithi027.stdout:7/498: write d4/d4d/f91 [805344,114528] 0 2024-04-20T10:59:19.714 INFO:tasks.workunit.client.0.smithi027.stdout:4/504: rmdir d0/d14/d6b/da3 39 2024-04-20T10:59:19.715 INFO:tasks.workunit.client.0.smithi027.stdout:9/453: unlink d4/d14/d43/c81 0 2024-04-20T10:59:19.716 INFO:tasks.workunit.client.0.smithi027.stdout:4/505: creat d0/d5e/d6c/fbe x:0 0 0 2024-04-20T10:59:19.719 INFO:tasks.workunit.client.0.smithi027.stdout:4/506: dread d0/d14/f4a [0,4194304] 0 2024-04-20T10:59:19.719 INFO:tasks.workunit.client.0.smithi027.stdout:4/507: fsync d0/d14/d6b/da3/d53/d79/f89 0 2024-04-20T10:59:19.722 INFO:tasks.workunit.client.0.smithi027.stdout:9/454: creat d4/d1b/d8f/f90 x:0 0 0 2024-04-20T10:59:19.724 INFO:tasks.workunit.client.0.smithi027.stdout:3/558: dwrite d2/dd/f10 [4194304,4194304] 0 2024-04-20T10:59:19.728 INFO:tasks.workunit.client.0.smithi027.stdout:1/680: dwrite d3/d7/d28/d92/dad/fbf [0,4194304] 0 2024-04-20T10:59:19.728 INFO:tasks.workunit.client.0.smithi027.stdout:1/681: write d3/d7/d38/f56 [4727915,42368] 0 2024-04-20T10:59:19.729 INFO:tasks.workunit.client.0.smithi027.stdout:3/559: symlink d2/laf 0 2024-04-20T10:59:19.732 INFO:tasks.workunit.client.0.smithi027.stdout:2/472: dread d5/f58 [0,4194304] 0 2024-04-20T10:59:19.732 INFO:tasks.workunit.client.0.smithi027.stdout:2/473: fdatasync d5/dc/f41 0 2024-04-20T10:59:19.733 INFO:tasks.workunit.client.0.smithi027.stdout:1/682: creat d3/d7/d38/da4/dac/ff9 x:0 0 0 2024-04-20T10:59:19.733 INFO:tasks.workunit.client.0.smithi027.stdout:1/683: write d3/d7/d28/f59 [4267864,69914] 0 2024-04-20T10:59:19.734 INFO:tasks.workunit.client.0.smithi027.stdout:1/684: write d3/d7/d38/da4/dac/fc0 [514521,130461] 0 2024-04-20T10:59:19.734 INFO:tasks.workunit.client.0.smithi027.stdout:9/455: mkdir d4/d14/d43/d91 0 2024-04-20T10:59:19.735 INFO:tasks.workunit.client.0.smithi027.stdout:3/560: link d2/dd/df/l2b d2/d94/lb0 0 2024-04-20T10:59:19.736 INFO:tasks.workunit.client.0.smithi027.stdout:3/561: read - d2/d18/d2c/d4c/f6e zero size 2024-04-20T10:59:19.736 INFO:tasks.workunit.client.0.smithi027.stdout:3/562: write d2/d94/f95 [1020902,65147] 0 2024-04-20T10:59:19.736 INFO:tasks.workunit.client.0.smithi027.stdout:3/563: write d2/dd/d57/f83 [4311986,77881] 0 2024-04-20T10:59:19.736 INFO:tasks.workunit.client.0.smithi027.stdout:2/474: creat d5/d1e/f9c x:0 0 0 2024-04-20T10:59:19.738 INFO:tasks.workunit.client.0.smithi027.stdout:1/685: rename d3/cca to d3/d45/cfa 0 2024-04-20T10:59:19.740 INFO:tasks.workunit.client.0.smithi027.stdout:3/564: creat d2/dd/d57/d8c/fb1 x:0 0 0 2024-04-20T10:59:19.740 INFO:tasks.workunit.client.0.smithi027.stdout:3/565: chown d2/d80/c49 30773 1 2024-04-20T10:59:19.741 INFO:tasks.workunit.client.0.smithi027.stdout:4/508: dwrite d0/d14/d6b/da3/d1b/d33/f57 [0,4194304] 0 2024-04-20T10:59:19.741 INFO:tasks.workunit.client.0.smithi027.stdout:4/509: chown d0/d14/d6b/da3/d53/d6a 3977040 1 2024-04-20T10:59:19.741 INFO:tasks.workunit.client.0.smithi027.stdout:2/475: mkdir d5/d1e/d9d 0 2024-04-20T10:59:19.742 INFO:tasks.workunit.client.0.smithi027.stdout:1/686: mkdir d3/d20/dfb 0 2024-04-20T10:59:19.744 INFO:tasks.workunit.client.0.smithi027.stdout:9/456: rename l3 to d4/d74/d72/l92 0 2024-04-20T10:59:19.744 INFO:tasks.workunit.client.0.smithi027.stdout:3/566: symlink d2/d18/d2c/lb2 0 2024-04-20T10:59:19.746 INFO:tasks.workunit.client.0.smithi027.stdout:4/510: unlink d0/fb6 0 2024-04-20T10:59:19.746 INFO:tasks.workunit.client.0.smithi027.stdout:4/511: write d0/fd [2707385,123320] 0 2024-04-20T10:59:19.746 INFO:tasks.workunit.client.0.smithi027.stdout:4/512: chown d0/d14/d6b/da3/d1b/d33/c40 7263 1 2024-04-20T10:59:19.746 INFO:tasks.workunit.client.0.smithi027.stdout:1/687: creat d3/d20/ffc x:0 0 0 2024-04-20T10:59:19.747 INFO:tasks.workunit.client.0.smithi027.stdout:8/609: sync 2024-04-20T10:59:19.747 INFO:tasks.workunit.client.0.smithi027.stdout:8/610: fdatasync d9/de/d20/d48/d59/f60 0 2024-04-20T10:59:19.749 INFO:tasks.workunit.client.0.smithi027.stdout:3/567: rename d2/d18/d2e/f9a to d2/d18/d2e/fb3 0 2024-04-20T10:59:19.749 INFO:tasks.workunit.client.0.smithi027.stdout:4/513: mkdir d0/d14/d6b/da3/d18/dbf 0 2024-04-20T10:59:19.751 INFO:tasks.workunit.client.0.smithi027.stdout:1/688: creat d3/d7/ffd x:0 0 0 2024-04-20T10:59:19.751 INFO:tasks.workunit.client.0.smithi027.stdout:1/689: write d3/d20/faf [476642,15166] 0 2024-04-20T10:59:19.755 INFO:tasks.workunit.client.0.smithi027.stdout:8/611: mknod d9/de/d1e/d4a/d6d/d7d/dd6/cd9 0 2024-04-20T10:59:19.755 INFO:tasks.workunit.client.0.smithi027.stdout:8/612: write d9/de/da3/d3c/f9c [8276274,101787] 0 2024-04-20T10:59:19.755 INFO:tasks.workunit.client.0.smithi027.stdout:3/568: unlink d2/dd/df/d23/f86 0 2024-04-20T10:59:19.755 INFO:tasks.workunit.client.0.smithi027.stdout:4/514: unlink d0/d14/d6b/da3/d1a/d3b/l83 0 2024-04-20T10:59:19.756 INFO:tasks.workunit.client.0.smithi027.stdout:4/515: truncate d0/d5e/d6c/fbe 630340 0 2024-04-20T10:59:19.756 INFO:tasks.workunit.client.0.smithi027.stdout:9/457: read d4/f7 [2444544,117765] 0 2024-04-20T10:59:19.756 INFO:tasks.workunit.client.0.smithi027.stdout:8/613: dread d9/de/d20/d53/f77 [0,4194304] 0 2024-04-20T10:59:19.756 INFO:tasks.workunit.client.0.smithi027.stdout:1/690: mknod d3/d7/d38/ddf/dcc/cfe 0 2024-04-20T10:59:19.756 INFO:tasks.workunit.client.0.smithi027.stdout:1/691: stat d3/d7/d38/db8 0 2024-04-20T10:59:19.756 INFO:tasks.workunit.client.0.smithi027.stdout:1/692: stat d3/d7/d38/ddf/dd9 0 2024-04-20T10:59:19.756 INFO:tasks.workunit.client.0.smithi027.stdout:1/693: write d3/d7/d38/ddf/d73/f7d [4513900,11376] 0 2024-04-20T10:59:19.756 INFO:tasks.workunit.client.0.smithi027.stdout:3/569: symlink d2/d18/lb4 0 2024-04-20T10:59:19.757 INFO:tasks.workunit.client.0.smithi027.stdout:4/516: mknod d0/d14/d6b/da3/d1b/d33/d5c/cc0 0 2024-04-20T10:59:19.757 INFO:tasks.workunit.client.0.smithi027.stdout:4/517: write d0/d14/d6b/da3/d1b/f9c [345342,42514] 0 2024-04-20T10:59:19.757 INFO:tasks.workunit.client.0.smithi027.stdout:9/458: creat d4/d74/f93 x:0 0 0 2024-04-20T10:59:19.757 INFO:tasks.workunit.client.0.smithi027.stdout:9/459: stat d4/d1b/d1f/d45/d8a/f6e 0 2024-04-20T10:59:19.757 INFO:tasks.workunit.client.0.smithi027.stdout:8/614: symlink d9/de/d1e/d4a/d6d/db7/d5b/lda 0 2024-04-20T10:59:19.758 INFO:tasks.workunit.client.0.smithi027.stdout:1/694: symlink d3/d20/dfb/lff 0 2024-04-20T10:59:19.759 INFO:tasks.workunit.client.0.smithi027.stdout:3/570: creat d2/dd/fb5 x:0 0 0 2024-04-20T10:59:19.760 INFO:tasks.workunit.client.0.smithi027.stdout:4/518: symlink d0/d14/d6b/da3/d1b/d33/d5c/d6f/d7b/lc1 0 2024-04-20T10:59:19.760 INFO:tasks.workunit.client.0.smithi027.stdout:4/519: truncate d0/d14/d6b/da3/d1b/f3d 636928 0 2024-04-20T10:59:19.761 INFO:tasks.workunit.client.0.smithi027.stdout:4/520: chown d0/d14/d6b 58125 1 2024-04-20T10:59:19.761 INFO:tasks.workunit.client.0.smithi027.stdout:4/521: chown d0/d14/d6b/da3/d1b/d2f/f67 57 1 2024-04-20T10:59:19.762 INFO:tasks.workunit.client.0.smithi027.stdout:1/695: rename d3/d7/fa to d3/d7/d28/f100 0 2024-04-20T10:59:19.763 INFO:tasks.workunit.client.0.smithi027.stdout:3/571: unlink d2/f14 0 2024-04-20T10:59:19.763 INFO:tasks.workunit.client.0.smithi027.stdout:3/572: fsync d2/dd/df/d23/d68/d88/f89 0 2024-04-20T10:59:19.768 INFO:tasks.workunit.client.0.smithi027.stdout:4/522: getdents d0/d14/d6b/da3/d53/d79 0 2024-04-20T10:59:19.769 INFO:tasks.workunit.client.0.smithi027.stdout:4/523: dread - d0/d14/d6b/da3/d18/f54 zero size 2024-04-20T10:59:19.769 INFO:tasks.workunit.client.0.smithi027.stdout:9/460: rename d4/d2a/f61 to d4/d1b/d31/d51/f94 0 2024-04-20T10:59:19.770 INFO:tasks.workunit.client.0.smithi027.stdout:3/573: symlink d2/d18/d2c/d4c/d67/lb6 0 2024-04-20T10:59:19.771 INFO:tasks.workunit.client.0.smithi027.stdout:3/574: fsync d2/d18/d2e/f35 0 2024-04-20T10:59:19.777 INFO:tasks.workunit.client.0.smithi027.stdout:4/524: dread d0/d14/d6b/da3/d53/d6a/f7c [0,4194304] 0 2024-04-20T10:59:19.778 INFO:tasks.workunit.client.0.smithi027.stdout:9/461: readlink d4/lf 0 2024-04-20T10:59:19.784 INFO:tasks.workunit.client.0.smithi027.stdout:9/462: write d4/f88 [1346774,31646] 0 2024-04-20T10:59:19.786 INFO:tasks.workunit.client.0.smithi027.stdout:3/575: getdents d2/dd/df 0 2024-04-20T10:59:19.788 INFO:tasks.workunit.client.0.smithi027.stdout:4/525: rmdir d0 39 2024-04-20T10:59:19.791 INFO:tasks.workunit.client.0.smithi027.stdout:5/512: sync 2024-04-20T10:59:19.791 INFO:tasks.workunit.client.0.smithi027.stdout:6/563: sync 2024-04-20T10:59:19.791 INFO:tasks.workunit.client.0.smithi027.stdout:5/513: chown d19/d1b/d1c/d1e/d58/f61 122859918 1 2024-04-20T10:59:19.791 INFO:tasks.workunit.client.0.smithi027.stdout:5/514: write d19/d1b/d1c/d1e/d25/fa5 [390197,96288] 0 2024-04-20T10:59:19.791 INFO:tasks.workunit.client.0.smithi027.stdout:5/515: read d19/d1b/d1c/d1e/d3c/f39 [23557,6034] 0 2024-04-20T10:59:19.796 INFO:tasks.workunit.client.0.smithi027.stdout:6/564: dread da/d21/dab/f10 [0,4194304] 0 2024-04-20T10:59:19.797 INFO:tasks.workunit.client.0.smithi027.stdout:5/516: dread d19/d1b/d1c/d1e/d3c/f31 [0,4194304] 0 2024-04-20T10:59:19.798 INFO:tasks.workunit.client.0.smithi027.stdout:3/576: unlink d2/dd/df/d33/f3c 0 2024-04-20T10:59:19.798 INFO:tasks.workunit.client.0.smithi027.stdout:3/577: stat d2/dd/df/d23/c5d 0 2024-04-20T10:59:19.799 INFO:tasks.workunit.client.0.smithi027.stdout:4/526: creat d0/d14/d6b/da3/d1b/d33/fc2 x:0 0 0 2024-04-20T10:59:19.800 INFO:tasks.workunit.client.0.smithi027.stdout:6/565: getdents da/d21/dab/d7e 0 2024-04-20T10:59:19.800 INFO:tasks.workunit.client.0.smithi027.stdout:6/566: write da/d21/dab/d19/d2e/d6b/f4e [4951314,74652] 0 2024-04-20T10:59:19.802 INFO:tasks.workunit.client.0.smithi027.stdout:9/463: rename d4/d14/c30 to d4/d14/d43/c95 0 2024-04-20T10:59:19.802 INFO:tasks.workunit.client.0.smithi027.stdout:5/517: mknod d19/d1b/ca8 0 2024-04-20T10:59:19.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/518: chown d19/d1b/d1c/d1e/d58/d9d 20 1 2024-04-20T10:59:19.804 INFO:tasks.workunit.client.0.smithi027.stdout:3/578: creat d2/fb7 x:0 0 0 2024-04-20T10:59:19.804 INFO:tasks.workunit.client.0.smithi027.stdout:3/579: fsync d2/dd/df/faa 0 2024-04-20T10:59:19.804 INFO:tasks.workunit.client.0.smithi027.stdout:4/527: mkdir d0/d14/d6b/da3/d1b/dc3 0 2024-04-20T10:59:19.807 INFO:tasks.workunit.client.0.smithi027.stdout:4/528: dread d0/d14/d6b/da3/d53/fb4 [0,4194304] 0 2024-04-20T10:59:19.807 INFO:tasks.workunit.client.0.smithi027.stdout:4/529: write d0/d5e/d6c/fbe [967956,129389] 0 2024-04-20T10:59:19.808 INFO:tasks.workunit.client.0.smithi027.stdout:4/530: dread - d0/d14/d6b/da3/d1a/f8a zero size 2024-04-20T10:59:19.812 INFO:tasks.workunit.client.0.smithi027.stdout:6/567: truncate da/d21/dab/d19/d2e/d32/f75 2388763 0 2024-04-20T10:59:19.815 INFO:tasks.workunit.client.0.smithi027.stdout:9/464: dread d4/f6b [0,4194304] 0 2024-04-20T10:59:19.815 INFO:tasks.workunit.client.0.smithi027.stdout:9/465: stat d4/d74/d72 0 2024-04-20T10:59:19.829 INFO:tasks.workunit.client.0.smithi027.stdout:5/519: dwrite d19/d1b/d1c/f6a [0,4194304] 0 2024-04-20T10:59:19.830 INFO:tasks.workunit.client.0.smithi027.stdout:4/531: truncate d0/d14/d6b/da3/d1b/d2f/f3f 546137 0 2024-04-20T10:59:19.830 INFO:tasks.workunit.client.0.smithi027.stdout:6/568: getdents da/dd 0 2024-04-20T10:59:19.833 INFO:tasks.workunit.client.0.smithi027.stdout:3/580: dwrite d2/dd/d22/f2f [4194304,4194304] 0 2024-04-20T10:59:19.835 INFO:tasks.workunit.client.0.smithi027.stdout:9/466: dwrite d4/d1b/d31/f67 [0,4194304] 0 2024-04-20T10:59:19.836 INFO:tasks.workunit.client.0.smithi027.stdout:5/520: getdents d19/d1b/d1c/d1e/d25/d40 0 2024-04-20T10:59:19.838 INFO:tasks.workunit.client.0.smithi027.stdout:4/532: creat d0/d14/fc4 x:0 0 0 2024-04-20T10:59:19.838 INFO:tasks.workunit.client.0.smithi027.stdout:4/533: dread - d0/d14/d6b/da3/d1a/f8a zero size 2024-04-20T10:59:19.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/569: mkdir da/dd/d3d/db9 0 2024-04-20T10:59:19.841 INFO:tasks.workunit.client.0.smithi027.stdout:5/521: mknod d19/d1b/d1c/d1e/d3c/ca9 0 2024-04-20T10:59:19.844 INFO:tasks.workunit.client.0.smithi027.stdout:1/696: dwrite d3/d7e/fd2 [4194304,4194304] 0 2024-04-20T10:59:19.845 INFO:tasks.workunit.client.0.smithi027.stdout:4/534: mkdir d0/d14/d6b/da3/d1b/dc3/dc5 0 2024-04-20T10:59:19.846 INFO:tasks.workunit.client.0.smithi027.stdout:6/570: creat da/d21/dab/d7e/d86/da3/fba x:0 0 0 2024-04-20T10:59:19.847 INFO:tasks.workunit.client.0.smithi027.stdout:5/522: unlink d19/d1b/d1c/c99 0 2024-04-20T10:59:19.848 INFO:tasks.workunit.client.0.smithi027.stdout:5/523: chown d19/d1b/d1c/f98 30869 1 2024-04-20T10:59:19.848 INFO:tasks.workunit.client.0.smithi027.stdout:5/524: fsync d19/d1b/d1c/d1e/d22/d47/f9f 0 2024-04-20T10:59:19.851 INFO:tasks.workunit.client.0.smithi027.stdout:9/467: dwrite d4/d1b/d1f/d45/f3f [0,4194304] 0 2024-04-20T10:59:19.854 INFO:tasks.workunit.client.0.smithi027.stdout:1/697: rename d3/d7/d38/ddf/l69 to d3/d20/dfb/l101 0 2024-04-20T10:59:19.854 INFO:tasks.workunit.client.0.smithi027.stdout:1/698: write d3/d20/f4d [1788988,36268] 0 2024-04-20T10:59:19.854 INFO:tasks.workunit.client.0.smithi027.stdout:6/571: dread da/d21/f95 [0,4194304] 0 2024-04-20T10:59:19.854 INFO:tasks.workunit.client.0.smithi027.stdout:4/535: mknod d0/d14/d6b/da3/d18/dbf/cc6 0 2024-04-20T10:59:19.854 INFO:tasks.workunit.client.0.smithi027.stdout:4/536: write d0/d14/d6b/da3/d18/fa8 [1405572,111152] 0 2024-04-20T10:59:19.862 INFO:tasks.workunit.client.0.smithi027.stdout:0/638: sync 2024-04-20T10:59:19.862 INFO:tasks.workunit.client.0.smithi027.stdout:8/615: sync 2024-04-20T10:59:19.862 INFO:tasks.workunit.client.0.smithi027.stdout:2/476: sync 2024-04-20T10:59:19.862 INFO:tasks.workunit.client.0.smithi027.stdout:5/525: creat d19/d1b/d1c/d1e/d3c/faa x:0 0 0 2024-04-20T10:59:19.862 INFO:tasks.workunit.client.0.smithi027.stdout:7/499: sync 2024-04-20T10:59:19.867 INFO:tasks.workunit.client.0.smithi027.stdout:8/616: dread d9/de/da3/d3c/f6f [0,4194304] 0 2024-04-20T10:59:19.868 INFO:tasks.workunit.client.0.smithi027.stdout:9/468: creat d4/d14/d43/d91/f96 x:0 0 0 2024-04-20T10:59:19.870 INFO:tasks.workunit.client.0.smithi027.stdout:2/477: rename d5/dc/d16/d2a/d4a/l63 to d5/d26/d47/d5e/l9e 0 2024-04-20T10:59:19.872 INFO:tasks.workunit.client.0.smithi027.stdout:0/639: creat d0/d11/d2c/d2e/d2b/d7b/d12/dd7/fdf x:0 0 0 2024-04-20T10:59:19.872 INFO:tasks.workunit.client.0.smithi027.stdout:8/617: rmdir d9/de/da3/dab 39 2024-04-20T10:59:19.872 INFO:tasks.workunit.client.0.smithi027.stdout:8/618: read - d9/de/d1e/fcb zero size 2024-04-20T10:59:19.875 INFO:tasks.workunit.client.0.smithi027.stdout:1/699: dwrite d3/d7e/f83 [0,4194304] 0 2024-04-20T10:59:19.876 INFO:tasks.workunit.client.0.smithi027.stdout:1/700: chown d3/d7/l71 4884 1 2024-04-20T10:59:19.881 INFO:tasks.workunit.client.0.smithi027.stdout:9/469: mknod d4/d74/d72/c97 0 2024-04-20T10:59:19.889 INFO:tasks.workunit.client.0.smithi027.stdout:7/500: dwrite d4/d1b/d4a/f68 [0,4194304] 0 2024-04-20T10:59:19.890 INFO:tasks.workunit.client.0.smithi027.stdout:0/640: mknod d0/d11/d2c/d2e/d2b/d7b/ce0 0 2024-04-20T10:59:19.893 INFO:tasks.workunit.client.0.smithi027.stdout:9/470: read d4/d1b/d1f/d45/f3f [2544278,63520] 0 2024-04-20T10:59:19.903 INFO:tasks.workunit.client.0.smithi027.stdout:2/478: dwrite d5/d26/d30/f3c [0,4194304] 0 2024-04-20T10:59:19.903 INFO:tasks.workunit.client.0.smithi027.stdout:8/619: dwrite d9/de/da3/d3c/fd3 [0,4194304] 0 2024-04-20T10:59:19.905 INFO:tasks.workunit.client.0.smithi027.stdout:7/501: unlink d4/d5/d8/d3b/f42 0 2024-04-20T10:59:19.905 INFO:tasks.workunit.client.0.smithi027.stdout:7/502: stat d4/d1b/d4a/d62/f6a 0 2024-04-20T10:59:19.905 INFO:tasks.workunit.client.0.smithi027.stdout:7/503: fdatasync d4/d5/d8/d21/f6e 0 2024-04-20T10:59:19.905 INFO:tasks.workunit.client.0.smithi027.stdout:1/701: dwrite d3/d7/d28/d35/f5e [4194304,4194304] 0 2024-04-20T10:59:19.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/641: creat d0/d11/d2c/d2e/d61/fe1 x:0 0 0 2024-04-20T10:59:19.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/479: creat d5/dc/d16/d2a/f9f x:0 0 0 2024-04-20T10:59:19.908 INFO:tasks.workunit.client.0.smithi027.stdout:2/480: fsync d5/dc/fe 0 2024-04-20T10:59:19.911 INFO:tasks.workunit.client.0.smithi027.stdout:1/702: dread d3/f17 [0,4194304] 0 2024-04-20T10:59:19.928 INFO:tasks.workunit.client.0.smithi027.stdout:7/504: dwrite d4/d5b/f67 [0,4194304] 0 2024-04-20T10:59:19.928 INFO:tasks.workunit.client.0.smithi027.stdout:7/505: write d4/d1b/f53 [1374110,61668] 0 2024-04-20T10:59:19.928 INFO:tasks.workunit.client.0.smithi027.stdout:2/481: dwrite d5/dc/d16/d2a/f40 [0,4194304] 0 2024-04-20T10:59:19.929 INFO:tasks.workunit.client.0.smithi027.stdout:8/620: unlink d9/de/d20/c96 0 2024-04-20T10:59:19.929 INFO:tasks.workunit.client.0.smithi027.stdout:8/621: truncate d9/de/d20/d48/d59/f99 1662584 0 2024-04-20T10:59:19.929 INFO:tasks.workunit.client.0.smithi027.stdout:8/622: chown d9/de/d1e/d4a/d6d/db7 11 1 2024-04-20T10:59:19.929 INFO:tasks.workunit.client.0.smithi027.stdout:8/623: stat d9/de/d1e/d4a/d6d/d79 0 2024-04-20T10:59:19.930 INFO:tasks.workunit.client.0.smithi027.stdout:0/642: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/fe2 x:0 0 0 2024-04-20T10:59:19.930 INFO:tasks.workunit.client.0.smithi027.stdout:1/703: rename d3/d7/d28/ca1 to d3/d7/d38/db8/c102 0 2024-04-20T10:59:19.932 INFO:tasks.workunit.client.0.smithi027.stdout:7/506: getdents d4/d5/d8/d3b 0 2024-04-20T10:59:19.933 INFO:tasks.workunit.client.0.smithi027.stdout:8/624: getdents d9/de/d1e/d4a/d6d/d7d 0 2024-04-20T10:59:19.934 INFO:tasks.workunit.client.0.smithi027.stdout:2/482: rename d5/d26/d47/d5e to d5/d26/d3d/da0 0 2024-04-20T10:59:19.949 INFO:tasks.workunit.client.0.smithi027.stdout:0/643: dwrite d0/d11/d2c/d2e/d2b/d7b/f27 [0,4194304] 0 2024-04-20T10:59:19.949 INFO:tasks.workunit.client.0.smithi027.stdout:0/644: write d0/d11/d2c/d2e/d2b/d3d/f84 [5312320,107764] 0 2024-04-20T10:59:19.950 INFO:tasks.workunit.client.0.smithi027.stdout:1/704: symlink d3/d7/d38/db8/l103 0 2024-04-20T10:59:19.950 INFO:tasks.workunit.client.0.smithi027.stdout:1/705: chown d3/d20/dfb/l101 19 1 2024-04-20T10:59:19.950 INFO:tasks.workunit.client.0.smithi027.stdout:7/507: creat d4/d1b/d4a/f9c x:0 0 0 2024-04-20T10:59:19.950 INFO:tasks.workunit.client.0.smithi027.stdout:2/483: symlink d5/d44/la1 0 2024-04-20T10:59:19.950 INFO:tasks.workunit.client.0.smithi027.stdout:7/508: chown d4/d5/d8/d3b 295 1 2024-04-20T10:59:19.950 INFO:tasks.workunit.client.0.smithi027.stdout:8/625: stat d9/de/f1b 0 2024-04-20T10:59:19.950 INFO:tasks.workunit.client.0.smithi027.stdout:8/626: fsync d9/de/da3/d3c/f9c 0 2024-04-20T10:59:19.967 INFO:tasks.workunit.client.0.smithi027.stdout:1/706: dwrite d3/d7/d38/ddf/fd8 [0,4194304] 0 2024-04-20T10:59:20.201 INFO:tasks.workunit.client.0.smithi027.stdout:2/484: unlink d5/d26/c8b 0 2024-04-20T10:59:20.202 INFO:tasks.workunit.client.0.smithi027.stdout:7/509: rmdir d4/d1b/d4a/d7a 39 2024-04-20T10:59:20.202 INFO:tasks.workunit.client.0.smithi027.stdout:2/485: write d5/dc/d16/d2a/d4a/f50 [5248561,14626] 0 2024-04-20T10:59:20.202 INFO:tasks.workunit.client.0.smithi027.stdout:0/645: creat d0/d11/d2c/da1/fe3 x:0 0 0 2024-04-20T10:59:20.202 INFO:tasks.workunit.client.0.smithi027.stdout:0/646: chown d0/d11/d2c/da1/fe3 275 1 2024-04-20T10:59:20.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/627: link d9/de/d1e/fcb d9/de/da3/dab/fdb 0 2024-04-20T10:59:20.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/628: chown d9/de/da3/dab/fdb 246037891 1 2024-04-20T10:59:20.204 INFO:tasks.workunit.client.0.smithi027.stdout:1/707: unlink d3/d7/d38/db8/c102 0 2024-04-20T10:59:20.204 INFO:tasks.workunit.client.0.smithi027.stdout:1/708: readlink d3/d45/d5d/la8 0 2024-04-20T10:59:20.204 INFO:tasks.workunit.client.0.smithi027.stdout:7/510: mknod d4/d4d/c9d 0 2024-04-20T10:59:20.205 INFO:tasks.workunit.client.0.smithi027.stdout:0/647: mknod d0/ce4 0 2024-04-20T10:59:20.205 INFO:tasks.workunit.client.0.smithi027.stdout:0/648: dread - d0/d11/d2c/d2e/d2b/d7b/d34/f43 zero size 2024-04-20T10:59:20.206 INFO:tasks.workunit.client.0.smithi027.stdout:8/629: dread d9/f34 [0,4194304] 0 2024-04-20T10:59:20.208 INFO:tasks.workunit.client.0.smithi027.stdout:7/511: link d4/d5/f1f d4/d5b/f9e 0 2024-04-20T10:59:20.208 INFO:tasks.workunit.client.0.smithi027.stdout:7/512: chown d4/f3d 45 1 2024-04-20T10:59:20.208 INFO:tasks.workunit.client.0.smithi027.stdout:7/513: fdatasync d4/d5/f1f 0 2024-04-20T10:59:20.209 INFO:tasks.workunit.client.0.smithi027.stdout:7/514: unlink d4/d5/l55 0 2024-04-20T10:59:20.211 INFO:tasks.workunit.client.0.smithi027.stdout:7/515: unlink d4/d1b/d4a/c39 0 2024-04-20T10:59:20.214 INFO:tasks.workunit.client.0.smithi027.stdout:0/649: write d0/d59/f73 [2015593,39649] 0 2024-04-20T10:59:20.214 INFO:tasks.workunit.client.0.smithi027.stdout:0/650: write d0/d11/d2c/d2e/d2b/d3d/f7e [766381,2176] 0 2024-04-20T10:59:20.215 INFO:tasks.workunit.client.0.smithi027.stdout:7/516: creat d4/d1b/d4a/d62/f9f x:0 0 0 2024-04-20T10:59:20.215 INFO:tasks.workunit.client.0.smithi027.stdout:7/517: chown d4/d5/d8/d3b 14846392 1 2024-04-20T10:59:20.217 INFO:tasks.workunit.client.0.smithi027.stdout:0/651: creat d0/d11/d2c/d2e/d2b/d7b/d64/fe5 x:0 0 0 2024-04-20T10:59:20.218 INFO:tasks.workunit.client.0.smithi027.stdout:0/652: read d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f50 [864666,8945] 0 2024-04-20T10:59:20.218 INFO:tasks.workunit.client.0.smithi027.stdout:7/518: getdents d4 0 2024-04-20T10:59:20.220 INFO:tasks.workunit.client.0.smithi027.stdout:7/519: creat d4/d1b/d4a/d62/fa0 x:0 0 0 2024-04-20T10:59:20.221 INFO:tasks.workunit.client.0.smithi027.stdout:0/653: symlink d0/d11/d2c/d2e/d35/le6 0 2024-04-20T10:59:20.222 INFO:tasks.workunit.client.0.smithi027.stdout:0/654: getdents d0/d11/d4b 0 2024-04-20T10:59:20.223 INFO:tasks.workunit.client.0.smithi027.stdout:0/655: creat d0/d11/d4b/d5b/d6f/fe7 x:0 0 0 2024-04-20T10:59:20.223 INFO:tasks.workunit.client.0.smithi027.stdout:0/656: stat d0/d11/d2c 0 2024-04-20T10:59:20.223 INFO:tasks.workunit.client.0.smithi027.stdout:0/657: fdatasync d0/d11/d2c/d2e/d61/f78 0 2024-04-20T10:59:20.223 INFO:tasks.workunit.client.0.smithi027.stdout:0/658: fdatasync d0/d11/d2c/d2e/d2b/d7b/d34/f43 0 2024-04-20T10:59:20.224 INFO:tasks.workunit.client.0.smithi027.stdout:2/486: dwrite d5/d1e/f9c [0,4194304] 0 2024-04-20T10:59:20.224 INFO:tasks.workunit.client.0.smithi027.stdout:2/487: truncate d5/dc/d16/d32/f5a 383811 0 2024-04-20T10:59:20.224 INFO:tasks.workunit.client.0.smithi027.stdout:0/659: link d0/d59/d9e/ld5 d0/le8 0 2024-04-20T10:59:20.224 INFO:tasks.workunit.client.0.smithi027.stdout:0/660: write d0/d11/d2c/d2e/d35/d8c/fa7 [134844,122804] 0 2024-04-20T10:59:20.225 INFO:tasks.workunit.client.0.smithi027.stdout:2/488: mknod d5/dc/d16/d2a/d4a/ca2 0 2024-04-20T10:59:20.226 INFO:tasks.workunit.client.0.smithi027.stdout:2/489: rename d5/d26/d47/f5f to d5/d26/d47/d8a/fa3 0 2024-04-20T10:59:20.227 INFO:tasks.workunit.client.0.smithi027.stdout:2/490: unlink d5/f86 0 2024-04-20T10:59:20.228 INFO:tasks.workunit.client.0.smithi027.stdout:1/709: dwrite d3/f1b [0,4194304] 0 2024-04-20T10:59:20.229 INFO:tasks.workunit.client.0.smithi027.stdout:8/630: dwrite d9/de/da3/d3c/f90 [0,4194304] 0 2024-04-20T10:59:20.239 INFO:tasks.workunit.client.0.smithi027.stdout:0/661: read d0/d11/d62/fb8 [2103378,118540] 0 2024-04-20T10:59:20.239 INFO:tasks.workunit.client.0.smithi027.stdout:7/520: dwrite d4/d1b/f63 [0,4194304] 0 2024-04-20T10:59:20.242 INFO:tasks.workunit.client.0.smithi027.stdout:0/662: rename d0/d11/d2c/d2e/d2b/d7b/d12/d25/l8b to d0/d11/d2c/le9 0 2024-04-20T10:59:20.242 INFO:tasks.workunit.client.0.smithi027.stdout:0/663: dread - d0/d59/d9e/fb9 zero size 2024-04-20T10:59:20.245 INFO:tasks.workunit.client.0.smithi027.stdout:0/664: truncate d0/d11/f54 2151515 0 2024-04-20T10:59:20.245 INFO:tasks.workunit.client.0.smithi027.stdout:0/665: chown d0/d11/d2c/d2e/d35/d8c/cc6 255889895 1 2024-04-20T10:59:20.245 INFO:tasks.workunit.client.0.smithi027.stdout:0/666: chown d0/d11/d2c/f52 1 1 2024-04-20T10:59:20.247 INFO:tasks.workunit.client.0.smithi027.stdout:0/667: creat d0/d11/d4b/d5b/d6f/fea x:0 0 0 2024-04-20T10:59:20.248 INFO:tasks.workunit.client.0.smithi027.stdout:0/668: mkdir d0/d11/d4b/d5b/d6f/d82/dd1/deb 0 2024-04-20T10:59:20.250 INFO:tasks.workunit.client.0.smithi027.stdout:7/521: dread d4/f3d [0,4194304] 0 2024-04-20T10:59:20.250 INFO:tasks.workunit.client.0.smithi027.stdout:7/522: stat d4/d5/d8/d3b/f74 0 2024-04-20T10:59:20.250 INFO:tasks.workunit.client.0.smithi027.stdout:2/491: dwrite d5/dc/d16/d37/f6b [0,4194304] 0 2024-04-20T10:59:20.250 INFO:tasks.workunit.client.0.smithi027.stdout:8/631: dwrite d9/de/fd2 [0,4194304] 0 2024-04-20T10:59:20.252 INFO:tasks.workunit.client.0.smithi027.stdout:2/492: dread f1 [0,4194304] 0 2024-04-20T10:59:20.252 INFO:tasks.workunit.client.0.smithi027.stdout:7/523: mknod d4/d5/d8/d21/d94/ca1 0 2024-04-20T10:59:20.253 INFO:tasks.workunit.client.0.smithi027.stdout:2/493: read d5/dc/f55 [714210,3997] 0 2024-04-20T10:59:20.253 INFO:tasks.workunit.client.0.smithi027.stdout:7/524: dread - d4/d1b/d4a/d62/d87/f9b zero size 2024-04-20T10:59:20.256 INFO:tasks.workunit.client.0.smithi027.stdout:2/494: truncate d5/d1e/f42 3038612 0 2024-04-20T10:59:20.258 INFO:tasks.workunit.client.0.smithi027.stdout:7/525: mkdir d4/da2 0 2024-04-20T10:59:20.259 INFO:tasks.workunit.client.0.smithi027.stdout:2/495: mkdir d5/dc/d16/d2a/d59/da4 0 2024-04-20T10:59:20.259 INFO:tasks.workunit.client.0.smithi027.stdout:2/496: dread - d5/dc/d16/d60/f69 zero size 2024-04-20T10:59:20.260 INFO:tasks.workunit.client.0.smithi027.stdout:6/572: dwrite da/d21/dab/d19/d2e/fb4 [0,4194304] 0 2024-04-20T10:59:20.261 INFO:tasks.workunit.client.0.smithi027.stdout:7/526: mknod d4/d5/ca3 0 2024-04-20T10:59:20.262 INFO:tasks.workunit.client.0.smithi027.stdout:2/497: dread d5/dc/d16/d2a/d4a/f67 [0,4194304] 0 2024-04-20T10:59:20.262 INFO:tasks.workunit.client.0.smithi027.stdout:2/498: chown d5/d1e/f9c 777288 1 2024-04-20T10:59:20.262 INFO:tasks.workunit.client.0.smithi027.stdout:2/499: fsync d5/d1e/d21/f3e 0 2024-04-20T10:59:20.263 INFO:tasks.workunit.client.0.smithi027.stdout:6/573: creat da/d21/d28/fbb x:0 0 0 2024-04-20T10:59:20.263 INFO:tasks.workunit.client.0.smithi027.stdout:6/574: truncate da/d21/d28/d54/f9f 603784 0 2024-04-20T10:59:20.264 INFO:tasks.workunit.client.0.smithi027.stdout:2/500: creat d5/d44/fa5 x:0 0 0 2024-04-20T10:59:20.268 INFO:tasks.workunit.client.0.smithi027.stdout:5/526: sync 2024-04-20T10:59:20.268 INFO:tasks.workunit.client.0.smithi027.stdout:4/537: sync 2024-04-20T10:59:20.268 INFO:tasks.workunit.client.0.smithi027.stdout:3/581: sync 2024-04-20T10:59:20.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/471: sync 2024-04-20T10:59:20.268 INFO:tasks.workunit.client.0.smithi027.stdout:4/538: dread - d0/d14/d6b/da3/f9e zero size 2024-04-20T10:59:20.268 INFO:tasks.workunit.client.0.smithi027.stdout:4/539: readlink d0/d14/d6b/da3/d1a/d3b/l3c 0 2024-04-20T10:59:20.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/472: write d4/d14/f38 [790319,68807] 0 2024-04-20T10:59:20.268 INFO:tasks.workunit.client.0.smithi027.stdout:4/540: write d0/d14/d6b/da3/d18/f2d [1928925,104511] 0 2024-04-20T10:59:20.270 INFO:tasks.workunit.client.0.smithi027.stdout:8/632: dread d9/de/d64/d6c/f9e [0,4194304] 0 2024-04-20T10:59:20.271 INFO:tasks.workunit.client.0.smithi027.stdout:3/582: truncate d2/f1f 63756 0 2024-04-20T10:59:20.271 INFO:tasks.workunit.client.0.smithi027.stdout:4/541: getdents d0/d14 0 2024-04-20T10:59:20.272 INFO:tasks.workunit.client.0.smithi027.stdout:9/473: creat d4/d74/d72/f98 x:0 0 0 2024-04-20T10:59:20.274 INFO:tasks.workunit.client.0.smithi027.stdout:8/633: rename d9/de/d1e/d4a/d6d/db7/d97/da5 to d9/de/d1e/ddc 0 2024-04-20T10:59:20.274 INFO:tasks.workunit.client.0.smithi027.stdout:7/527: write d4/d5b/f7e [2181803,33993] 0 2024-04-20T10:59:20.275 INFO:tasks.workunit.client.0.smithi027.stdout:3/583: link d2/d18/d2c/d4c/d67/fab d2/dd/d57/d8c/fb8 0 2024-04-20T10:59:20.275 INFO:tasks.workunit.client.0.smithi027.stdout:3/584: readlink d2/dd/d57/l92 0 2024-04-20T10:59:20.277 INFO:tasks.workunit.client.0.smithi027.stdout:4/542: creat d0/d14/d6b/d92/fc7 x:0 0 0 2024-04-20T10:59:20.277 INFO:tasks.workunit.client.0.smithi027.stdout:4/543: write d0/d5e/fba [78655,105125] 0 2024-04-20T10:59:20.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/474: unlink d4/d74/f93 0 2024-04-20T10:59:20.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/475: fsync d4/d1b/d8f/f90 0 2024-04-20T10:59:20.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/476: write d4/d14/d43/f86 [35478,43205] 0 2024-04-20T10:59:20.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/477: dread - d4/d1b/d8f/f90 zero size 2024-04-20T10:59:20.279 INFO:tasks.workunit.client.0.smithi027.stdout:2/501: write d5/d1e/f9c [1436091,73807] 0 2024-04-20T10:59:20.280 INFO:tasks.workunit.client.0.smithi027.stdout:8/634: creat d9/de/d1e/fdd x:0 0 0 2024-04-20T10:59:20.280 INFO:tasks.workunit.client.0.smithi027.stdout:8/635: chown d9/de/d18/f22 134484 1 2024-04-20T10:59:20.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/636: truncate d9/de/f2b 1140076 0 2024-04-20T10:59:20.281 INFO:tasks.workunit.client.0.smithi027.stdout:7/528: getdents d4/d5/d8/d21/d94 0 2024-04-20T10:59:20.282 INFO:tasks.workunit.client.0.smithi027.stdout:3/585: rmdir d2/d80 39 2024-04-20T10:59:20.283 INFO:tasks.workunit.client.0.smithi027.stdout:4/544: creat d0/d14/d6b/da3/d1a/d3b/fc8 x:0 0 0 2024-04-20T10:59:20.284 INFO:tasks.workunit.client.0.smithi027.stdout:9/478: rename d4/d1b/d31/d23/f36 to d4/d1b/d31/d23/f99 0 2024-04-20T10:59:20.288 INFO:tasks.workunit.client.0.smithi027.stdout:6/575: dwrite da/dd/d3d/f90 [0,4194304] 0 2024-04-20T10:59:20.290 INFO:tasks.workunit.client.0.smithi027.stdout:6/576: dread da/d21/dab/d7e/f85 [0,4194304] 0 2024-04-20T10:59:20.290 INFO:tasks.workunit.client.0.smithi027.stdout:6/577: chown da/d21/dab/d19/d2e/d32/f75 59093946 1 2024-04-20T10:59:20.290 INFO:tasks.workunit.client.0.smithi027.stdout:6/578: write da/dd/d79/fb7 [810928,71186] 0 2024-04-20T10:59:20.303 INFO:tasks.workunit.client.0.smithi027.stdout:3/586: dread d2/d18/d2e/f35 [0,4194304] 0 2024-04-20T10:59:20.306 INFO:tasks.workunit.client.0.smithi027.stdout:7/529: dwrite d4/d1b/ff [0,4194304] 0 2024-04-20T10:59:20.308 INFO:tasks.workunit.client.0.smithi027.stdout:9/479: dwrite d4/d1b/d1f/f7c [0,4194304] 0 2024-04-20T10:59:20.325 INFO:tasks.workunit.client.0.smithi027.stdout:8/637: dread d9/de/d1e/f65 [0,4194304] 0 2024-04-20T10:59:20.325 INFO:tasks.workunit.client.0.smithi027.stdout:8/638: chown d9/f33 541 1 2024-04-20T10:59:20.545 INFO:tasks.workunit.client.0.smithi027.stdout:4/545: rename d0/d14/d6b/da3/d53/d6a/f7c to d0/d14/d6b/da3/d18/db9/fc9 0 2024-04-20T10:59:20.546 INFO:tasks.workunit.client.0.smithi027.stdout:3/587: getdents d2/dd 0 2024-04-20T10:59:20.547 INFO:tasks.workunit.client.0.smithi027.stdout:7/530: mknod d4/d5b/ca4 0 2024-04-20T10:59:20.547 INFO:tasks.workunit.client.0.smithi027.stdout:7/531: write d4/d1b/d4a/f9c [22437,65558] 0 2024-04-20T10:59:20.547 INFO:tasks.workunit.client.0.smithi027.stdout:3/588: dread d2/dd/f77 [0,4194304] 0 2024-04-20T10:59:20.548 INFO:tasks.workunit.client.0.smithi027.stdout:3/589: stat d2/f1f 0 2024-04-20T10:59:20.553 INFO:tasks.workunit.client.0.smithi027.stdout:8/639: creat d9/de/d1e/d4a/fde x:0 0 0 2024-04-20T10:59:20.557 INFO:tasks.workunit.client.0.smithi027.stdout:6/579: rename da/dd/f91 to da/d21/dab/d19/d2e/d6b/d65/fbc 0 2024-04-20T10:59:20.558 INFO:tasks.workunit.client.0.smithi027.stdout:7/532: creat d4/d1b/d4a/d62/fa5 x:0 0 0 2024-04-20T10:59:20.558 INFO:tasks.workunit.client.0.smithi027.stdout:3/590: symlink d2/dd/df/d23/d68/d88/lb9 0 2024-04-20T10:59:20.559 INFO:tasks.workunit.client.0.smithi027.stdout:3/591: truncate d2/d18/d2c/f69 4776829 0 2024-04-20T10:59:20.560 INFO:tasks.workunit.client.0.smithi027.stdout:8/640: creat d9/de/d1e/d4a/d6d/db7/d97/fdf x:0 0 0 2024-04-20T10:59:20.560 INFO:tasks.workunit.client.0.smithi027.stdout:6/580: getdents da/dd/d79 0 2024-04-20T10:59:20.561 INFO:tasks.workunit.client.0.smithi027.stdout:7/533: creat d4/d1b/d4a/fa6 x:0 0 0 2024-04-20T10:59:20.562 INFO:tasks.workunit.client.0.smithi027.stdout:8/641: creat d9/de/d20/d48/d59/da0/fe0 x:0 0 0 2024-04-20T10:59:20.564 INFO:tasks.workunit.client.0.smithi027.stdout:6/581: rename da/d21/dab/d19/d2e/l57 to da/d21/dab/lbd 0 2024-04-20T10:59:20.564 INFO:tasks.workunit.client.0.smithi027.stdout:8/642: write d9/de/f2b [665736,40353] 0 2024-04-20T10:59:20.564 INFO:tasks.workunit.client.0.smithi027.stdout:1/710: sync 2024-04-20T10:59:20.564 INFO:tasks.workunit.client.0.smithi027.stdout:2/502: sync 2024-04-20T10:59:20.565 INFO:tasks.workunit.client.0.smithi027.stdout:0/669: sync 2024-04-20T10:59:20.565 INFO:tasks.workunit.client.0.smithi027.stdout:5/527: sync 2024-04-20T10:59:20.565 INFO:tasks.workunit.client.0.smithi027.stdout:9/480: sync 2024-04-20T10:59:20.565 INFO:tasks.workunit.client.0.smithi027.stdout:5/528: readlink d19/d1b/d1c/d1e/d25/l37 0 2024-04-20T10:59:20.565 INFO:tasks.workunit.client.0.smithi027.stdout:0/670: truncate d0/d11/d2c/fcd 625089 0 2024-04-20T10:59:20.566 INFO:tasks.workunit.client.0.smithi027.stdout:8/643: creat d9/de/d20/d48/d59/fe1 x:0 0 0 2024-04-20T10:59:20.566 INFO:tasks.workunit.client.0.smithi027.stdout:6/582: dread da/dd/d3c/fa8 [0,4194304] 0 2024-04-20T10:59:20.567 INFO:tasks.workunit.client.0.smithi027.stdout:1/711: creat d3/d20/f104 x:0 0 0 2024-04-20T10:59:20.567 INFO:tasks.workunit.client.0.smithi027.stdout:4/546: dwrite d0/d14/d6b/da3/f1c [0,4194304] 0 2024-04-20T10:59:20.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/481: write d4/f1e [5485710,122706] 0 2024-04-20T10:59:20.571 INFO:tasks.workunit.client.0.smithi027.stdout:3/592: dread d2/dd/d22/f2f [4194304,4194304] 0 2024-04-20T10:59:20.571 INFO:tasks.workunit.client.0.smithi027.stdout:5/529: unlink d19/d1b/d1c/d1e/d22/d47/f9a 0 2024-04-20T10:59:20.571 INFO:tasks.workunit.client.0.smithi027.stdout:5/530: chown d19/d1b/d1c/d1e/d25/d40/d82 0 1 2024-04-20T10:59:20.572 INFO:tasks.workunit.client.0.smithi027.stdout:0/671: symlink d0/d11/d2c/d2e/d61/lec 0 2024-04-20T10:59:20.573 INFO:tasks.workunit.client.0.smithi027.stdout:8/644: unlink d9/de/da3/d3c/d4b/cca 0 2024-04-20T10:59:20.576 INFO:tasks.workunit.client.0.smithi027.stdout:1/712: symlink d3/d7/l105 0 2024-04-20T10:59:20.576 INFO:tasks.workunit.client.0.smithi027.stdout:1/713: getdents d3/d7/d38/ddf/d73 0 2024-04-20T10:59:20.576 INFO:tasks.workunit.client.0.smithi027.stdout:1/714: chown d3/d7/ff6 28191 1 2024-04-20T10:59:20.579 INFO:tasks.workunit.client.0.smithi027.stdout:4/547: truncate d0/d14/f4a 63098 0 2024-04-20T10:59:20.580 INFO:tasks.workunit.client.0.smithi027.stdout:9/482: rmdir d4/d1b/d50 39 2024-04-20T10:59:20.580 INFO:tasks.workunit.client.0.smithi027.stdout:9/483: write d4/d14/f44 [8746686,71222] 0 2024-04-20T10:59:20.581 INFO:tasks.workunit.client.0.smithi027.stdout:9/484: chown d4/d1b/d1f 908260 1 2024-04-20T10:59:20.581 INFO:tasks.workunit.client.0.smithi027.stdout:0/672: creat d0/d11/d2c/d2e/d2b/d7b/dcf/fed x:0 0 0 2024-04-20T10:59:20.582 INFO:tasks.workunit.client.0.smithi027.stdout:0/673: chown d0/d11/d2c/d2e/d2b/d7b/d12/d1a/c41 9493 1 2024-04-20T10:59:20.583 INFO:tasks.workunit.client.0.smithi027.stdout:1/715: rename d3/d41/c66 to d3/d7/d28/d35/c106 0 2024-04-20T10:59:20.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/485: creat d4/d1b/d31/f9a x:0 0 0 2024-04-20T10:59:20.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/486: fdatasync d4/d74/d72/f73 0 2024-04-20T10:59:20.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/487: chown d4/d14/l37 42032 1 2024-04-20T10:59:20.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/488: truncate d4/d1b/d1f/d45/f7d 1564914 0 2024-04-20T10:59:20.584 INFO:tasks.workunit.client.0.smithi027.stdout:9/489: fdatasync d4/d1b/d1f/f7c 0 2024-04-20T10:59:20.588 INFO:tasks.workunit.client.0.smithi027.stdout:0/674: symlink d0/d11/d4b/d5b/d6f/d82/dd1/deb/lee 0 2024-04-20T10:59:20.588 INFO:tasks.workunit.client.0.smithi027.stdout:1/716: mkdir d3/d20/d107 0 2024-04-20T10:59:20.588 INFO:tasks.workunit.client.0.smithi027.stdout:1/717: rmdir d3/d7e 39 2024-04-20T10:59:20.589 INFO:tasks.workunit.client.0.smithi027.stdout:0/675: symlink d0/d11/d2c/d2e/d55/lef 0 2024-04-20T10:59:20.590 INFO:tasks.workunit.client.0.smithi027.stdout:1/718: mknod d3/d7/c108 0 2024-04-20T10:59:20.591 INFO:tasks.workunit.client.0.smithi027.stdout:4/548: dread d0/d14/d6b/da3/d18/f4b [0,4194304] 0 2024-04-20T10:59:20.595 INFO:tasks.workunit.client.0.smithi027.stdout:6/583: dwrite da/d21/dab/f10 [4194304,4194304] 0 2024-04-20T10:59:20.595 INFO:tasks.workunit.client.0.smithi027.stdout:6/584: stat da/d21/dab/d1f 0 2024-04-20T10:59:20.595 INFO:tasks.workunit.client.0.smithi027.stdout:4/549: creat d0/d14/d6b/da3/d1b/d33/d97/fca x:0 0 0 2024-04-20T10:59:20.595 INFO:tasks.workunit.client.0.smithi027.stdout:8/645: write d9/de/f37 [1315952,109429] 0 2024-04-20T10:59:20.596 INFO:tasks.workunit.client.0.smithi027.stdout:6/585: dread da/dd/d3c/fa8 [0,4194304] 0 2024-04-20T10:59:20.596 INFO:tasks.workunit.client.0.smithi027.stdout:1/719: dread d3/d7/d28/f59 [0,4194304] 0 2024-04-20T10:59:20.596 INFO:tasks.workunit.client.0.smithi027.stdout:6/586: fdatasync da/d21/dab/d19/f93 0 2024-04-20T10:59:20.596 INFO:tasks.workunit.client.0.smithi027.stdout:6/587: chown da/d21/d28/d54 0 1 2024-04-20T10:59:20.596 INFO:tasks.workunit.client.0.smithi027.stdout:1/720: readlink d3/d7/d38/ddf/l4f 0 2024-04-20T10:59:20.596 INFO:tasks.workunit.client.0.smithi027.stdout:2/503: dwrite d5/d1e/f70 [0,4194304] 0 2024-04-20T10:59:20.597 INFO:tasks.workunit.client.0.smithi027.stdout:8/646: symlink d9/le2 0 2024-04-20T10:59:20.599 INFO:tasks.workunit.client.0.smithi027.stdout:6/588: creat da/d21/dab/d19/d2e/d6b/d65/fbe x:0 0 0 2024-04-20T10:59:20.599 INFO:tasks.workunit.client.0.smithi027.stdout:6/589: dread - da/d21/fb2 zero size 2024-04-20T10:59:20.599 INFO:tasks.workunit.client.0.smithi027.stdout:0/676: write d0/d11/f21 [1624356,95006] 0 2024-04-20T10:59:20.599 INFO:tasks.workunit.client.0.smithi027.stdout:2/504: symlink d5/dc/d16/d2a/d59/la6 0 2024-04-20T10:59:20.599 INFO:tasks.workunit.client.0.smithi027.stdout:0/677: chown d0/d11/d4b/d5b/d6f/d82/lc5 0 1 2024-04-20T10:59:20.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/678: dread - d0/d11/d4b/d5b/d6f/d82/fa0 zero size 2024-04-20T10:59:20.600 INFO:tasks.workunit.client.0.smithi027.stdout:8/647: unlink d9/fb5 0 2024-04-20T10:59:20.601 INFO:tasks.workunit.client.0.smithi027.stdout:2/505: rename d5/d26/d3d to d5/d44/da7 0 2024-04-20T10:59:20.603 INFO:tasks.workunit.client.0.smithi027.stdout:8/648: unlink d9/de/d64/d6c/f9e 0 2024-04-20T10:59:20.603 INFO:tasks.workunit.client.0.smithi027.stdout:8/649: write d9/de/f2b [353290,30664] 0 2024-04-20T10:59:20.603 INFO:tasks.workunit.client.0.smithi027.stdout:3/593: dwrite d2/dd/d22/fa2 [0,4194304] 0 2024-04-20T10:59:20.603 INFO:tasks.workunit.client.0.smithi027.stdout:3/594: fsync d2/d18/d2c/d4c/d67/fab 0 2024-04-20T10:59:20.603 INFO:tasks.workunit.client.0.smithi027.stdout:5/531: dwrite d19/d1b/d1c/d1e/d3c/f31 [0,4194304] 0 2024-04-20T10:59:20.603 INFO:tasks.workunit.client.0.smithi027.stdout:1/721: dread d3/d7/d28/f59 [0,4194304] 0 2024-04-20T10:59:20.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/550: write d0/f27 [2942293,9293] 0 2024-04-20T10:59:20.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/506: read d5/dc/d16/d2a/d59/f81 [187171,83997] 0 2024-04-20T10:59:20.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/507: fdatasync d5/dc/d16/d37/f46 0 2024-04-20T10:59:20.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/508: chown d5/dc/d16/d2a/d4a/f67 16076946 1 2024-04-20T10:59:20.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/509: dread - d5/dc/d16/d37/f4e zero size 2024-04-20T10:59:20.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/510: readlink d5/dc/l12 0 2024-04-20T10:59:20.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/511: chown d5/d44/f65 11985 1 2024-04-20T10:59:20.623 INFO:tasks.workunit.client.0.smithi027.stdout:6/590: dwrite da/d21/dab/d19/f6a [0,4194304] 0 2024-04-20T10:59:20.628 INFO:tasks.workunit.client.0.smithi027.stdout:0/679: dwrite d0/d11/d2c/d2e/d2b/d7b/fce [0,4194304] 0 2024-04-20T10:59:20.857 INFO:tasks.workunit.client.0.smithi027.stdout:8/650: creat d9/de/d1e/d4a/d6d/db7/fe3 x:0 0 0 2024-04-20T10:59:20.858 INFO:tasks.workunit.client.0.smithi027.stdout:3/595: creat d2/dd/d57/d8d/fba x:0 0 0 2024-04-20T10:59:20.858 INFO:tasks.workunit.client.0.smithi027.stdout:8/651: write d9/de/da3/d3c/f9c [6219889,96610] 0 2024-04-20T10:59:20.858 INFO:tasks.workunit.client.0.smithi027.stdout:9/490: truncate d4/f1e 2482282 0 2024-04-20T10:59:20.858 INFO:tasks.workunit.client.0.smithi027.stdout:5/532: symlink d19/d1b/d1c/d1e/d22/d68/d75/lab 0 2024-04-20T10:59:20.859 INFO:tasks.workunit.client.0.smithi027.stdout:1/722: rename d3/d45/fee to d3/d7/d28/d92/f109 0 2024-04-20T10:59:20.860 INFO:tasks.workunit.client.0.smithi027.stdout:9/491: dread d4/d14/d43/f52 [0,4194304] 0 2024-04-20T10:59:20.861 INFO:tasks.workunit.client.0.smithi027.stdout:4/551: creat d0/d14/d6b/da3/d53/d6a/da1/db0/fcb x:0 0 0 2024-04-20T10:59:20.861 INFO:tasks.workunit.client.0.smithi027.stdout:2/512: getdents d5/d26 0 2024-04-20T10:59:20.861 INFO:tasks.workunit.client.0.smithi027.stdout:6/591: mkdir da/d21/dab/d1f/dbf 0 2024-04-20T10:59:20.867 INFO:tasks.workunit.client.0.smithi027.stdout:5/533: truncate f2 3634013 0 2024-04-20T10:59:20.867 INFO:tasks.workunit.client.0.smithi027.stdout:9/492: rmdir d4/d1b/d1f/d45/d8a 39 2024-04-20T10:59:20.867 INFO:tasks.workunit.client.0.smithi027.stdout:9/493: write d4/d1b/d1f/d45/f71 [702559,44264] 0 2024-04-20T10:59:20.867 INFO:tasks.workunit.client.0.smithi027.stdout:9/494: fsync d4/d1b/d31/f3c 0 2024-04-20T10:59:20.867 INFO:tasks.workunit.client.0.smithi027.stdout:9/495: fdatasync d4/f88 0 2024-04-20T10:59:20.868 INFO:tasks.workunit.client.0.smithi027.stdout:1/723: write d3/d7/d28/d92/fb5 [3026441,17348] 0 2024-04-20T10:59:20.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/552: truncate d0/d14/d6b/da3/d1b/d33/d5c/f98 274203 0 2024-04-20T10:59:20.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/553: chown d0/d14/d6b/da3/d18/db9 1177079 1 2024-04-20T10:59:20.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/554: fdatasync d0/d14/d6b/da3/d1b/d2f/f82 0 2024-04-20T10:59:20.868 INFO:tasks.workunit.client.0.smithi027.stdout:2/513: mknod d5/d1e/d21/ca8 0 2024-04-20T10:59:20.869 INFO:tasks.workunit.client.0.smithi027.stdout:9/496: write d4/f6b [330153,45962] 0 2024-04-20T10:59:20.870 INFO:tasks.workunit.client.0.smithi027.stdout:6/592: unlink da/d21/dab/d19/d2e/d6b/f4e 0 2024-04-20T10:59:20.874 INFO:tasks.workunit.client.0.smithi027.stdout:2/514: write d5/dc/d16/d2a/d4a/f50 [3299942,13057] 0 2024-04-20T10:59:20.874 INFO:tasks.workunit.client.0.smithi027.stdout:2/515: chown d5/d44/la1 601527 1 2024-04-20T10:59:20.875 INFO:tasks.workunit.client.0.smithi027.stdout:6/593: dread da/f84 [0,4194304] 0 2024-04-20T10:59:20.875 INFO:tasks.workunit.client.0.smithi027.stdout:4/555: rename d0/d48/lad to d0/d14/d6b/da3/d1b/lcc 0 2024-04-20T10:59:20.877 INFO:tasks.workunit.client.0.smithi027.stdout:9/497: creat d4/d1b/d1f/f9b x:0 0 0 2024-04-20T10:59:20.877 INFO:tasks.workunit.client.0.smithi027.stdout:9/498: readlink d4/d1b/d31/d51/l62 0 2024-04-20T10:59:20.878 INFO:tasks.workunit.client.0.smithi027.stdout:3/596: dwrite d2/dd/d57/d8d/fa3 [0,4194304] 0 2024-04-20T10:59:20.879 INFO:tasks.workunit.client.0.smithi027.stdout:1/724: write d3/d7/d28/d92/dad/fbf [3424705,73084] 0 2024-04-20T10:59:20.885 INFO:tasks.workunit.client.0.smithi027.stdout:4/556: mkdir d0/d14/d6b/d92/dcd 0 2024-04-20T10:59:20.885 INFO:tasks.workunit.client.0.smithi027.stdout:9/499: rmdir d4/d1b/d31/d23 39 2024-04-20T10:59:20.885 INFO:tasks.workunit.client.0.smithi027.stdout:0/680: sync 2024-04-20T10:59:20.885 INFO:tasks.workunit.client.0.smithi027.stdout:4/557: chown d0/d14/d6b/da3/l25 0 1 2024-04-20T10:59:20.885 INFO:tasks.workunit.client.0.smithi027.stdout:7/534: sync 2024-04-20T10:59:20.885 INFO:tasks.workunit.client.0.smithi027.stdout:4/558: truncate d0/d14/d6b/da3/d1a/fa4 772366 0 2024-04-20T10:59:20.885 INFO:tasks.workunit.client.0.smithi027.stdout:5/534: truncate d19/d1b/d1c/d1e/d3c/f31 8738658 0 2024-04-20T10:59:20.885 INFO:tasks.workunit.client.0.smithi027.stdout:5/535: dread - d19/d1b/d1c/d1e/d25/fa0 zero size 2024-04-20T10:59:20.887 INFO:tasks.workunit.client.0.smithi027.stdout:1/725: mkdir d3/d20/d85/d10a 0 2024-04-20T10:59:20.887 INFO:tasks.workunit.client.0.smithi027.stdout:1/726: dread - d3/d7/ff6 zero size 2024-04-20T10:59:20.887 INFO:tasks.workunit.client.0.smithi027.stdout:1/727: stat d3/d7e/cb1 0 2024-04-20T10:59:20.887 INFO:tasks.workunit.client.0.smithi027.stdout:1/728: chown d3/d7/d38/da4/dac/fc0 21 1 2024-04-20T10:59:20.887 INFO:tasks.workunit.client.0.smithi027.stdout:1/729: chown d3/d7 4 1 2024-04-20T10:59:20.887 INFO:tasks.workunit.client.0.smithi027.stdout:7/535: stat d4/d5/d8/d21/l41 0 2024-04-20T10:59:20.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/681: getdents d0/d11/d2c/d2e/d2b/d7b/d34 0 2024-04-20T10:59:20.891 INFO:tasks.workunit.client.0.smithi027.stdout:7/536: mkdir d4/d5/d8/d21/d94/da7 0 2024-04-20T10:59:20.891 INFO:tasks.workunit.client.0.smithi027.stdout:7/537: stat d4/d5b 0 2024-04-20T10:59:20.891 INFO:tasks.workunit.client.0.smithi027.stdout:5/536: link d19/d1b/d1c/l2e d19/d1b/d1c/d1e/d22/d68/lac 0 2024-04-20T10:59:20.894 INFO:tasks.workunit.client.0.smithi027.stdout:0/682: rmdir d0/d11/d2c/d2e/d2b/d7b/d12/dd7 39 2024-04-20T10:59:20.894 INFO:tasks.workunit.client.0.smithi027.stdout:0/683: write d0/d59/f6a [1435634,77754] 0 2024-04-20T10:59:20.894 INFO:tasks.workunit.client.0.smithi027.stdout:7/538: creat d4/d5/d8/d21/d94/fa8 x:0 0 0 2024-04-20T10:59:20.895 INFO:tasks.workunit.client.0.smithi027.stdout:0/684: mkdir d0/d11/d5c/df0 0 2024-04-20T10:59:20.896 INFO:tasks.workunit.client.0.smithi027.stdout:0/685: dread d0/d11/d4b/fcb [0,4194304] 0 2024-04-20T10:59:20.897 INFO:tasks.workunit.client.0.smithi027.stdout:7/539: mkdir d4/d1b/d4a/d62/d87/da9 0 2024-04-20T10:59:20.897 INFO:tasks.workunit.client.0.smithi027.stdout:7/540: write d4/d1b/d4a/d62/fa5 [335719,109605] 0 2024-04-20T10:59:20.897 INFO:tasks.workunit.client.0.smithi027.stdout:7/541: write d4/d1b/d4a/d62/f9f [1037064,75554] 0 2024-04-20T10:59:20.911 INFO:tasks.workunit.client.0.smithi027.stdout:2/516: dwrite d5/d26/d47/d8a/f9a [0,4194304] 0 2024-04-20T10:59:20.911 INFO:tasks.workunit.client.0.smithi027.stdout:6/594: dwrite da/dd/d79/f87 [0,4194304] 0 2024-04-20T10:59:20.913 INFO:tasks.workunit.client.0.smithi027.stdout:9/500: dwrite d4/f1c [4194304,4194304] 0 2024-04-20T10:59:20.913 INFO:tasks.workunit.client.0.smithi027.stdout:9/501: fsync d4/d1b/d1f/f9b 0 2024-04-20T10:59:20.914 INFO:tasks.workunit.client.0.smithi027.stdout:3/597: dwrite d2/d18/d2e/fa7 [0,4194304] 0 2024-04-20T10:59:20.932 INFO:tasks.workunit.client.0.smithi027.stdout:4/559: dwrite d0/d14/d6b/da3/d53/d6a/da1/db0/fbb [0,4194304] 0 2024-04-20T10:59:20.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/517: dwrite f1 [0,4194304] 0 2024-04-20T10:59:20.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/518: chown d5/d26/d47 1488545754 1 2024-04-20T10:59:20.935 INFO:tasks.workunit.client.0.smithi027.stdout:5/537: dwrite d19/d1b/d1c/d1e/d25/d40/f81 [4194304,4194304] 0 2024-04-20T10:59:20.935 INFO:tasks.workunit.client.0.smithi027.stdout:1/730: dwrite d3/d41/dc6/fd3 [0,4194304] 0 2024-04-20T10:59:20.939 INFO:tasks.workunit.client.0.smithi027.stdout:4/560: write d0/d14/d6b/da3/d1b/d2f/f82 [43252,121500] 0 2024-04-20T10:59:20.959 INFO:tasks.workunit.client.0.smithi027.stdout:2/519: dwrite d5/f28 [0,4194304] 0 2024-04-20T10:59:20.959 INFO:tasks.workunit.client.0.smithi027.stdout:1/731: dread d3/d7/d38/ddf/f5a [0,4194304] 0 2024-04-20T10:59:20.959 INFO:tasks.workunit.client.0.smithi027.stdout:1/732: chown d3/d7/d28/f3d 1740183 1 2024-04-20T10:59:20.959 INFO:tasks.workunit.client.0.smithi027.stdout:1/733: write d3/d7/d28/f70 [539758,62152] 0 2024-04-20T10:59:21.154 INFO:tasks.workunit.client.0.smithi027.stdout:0/686: creat d0/d11/d2c/d2e/d2b/d7b/d12/ff1 x:0 0 0 2024-04-20T10:59:21.154 INFO:tasks.workunit.client.0.smithi027.stdout:7/542: rename d4/d5/d8/d45/l84 to d4/d5/d8/d3b/laa 0 2024-04-20T10:59:21.154 INFO:tasks.workunit.client.0.smithi027.stdout:7/543: fdatasync d4/d5/f1f 0 2024-04-20T10:59:21.157 INFO:tasks.workunit.client.0.smithi027.stdout:6/595: mknod da/d21/dab/d19/cc0 0 2024-04-20T10:59:21.157 INFO:tasks.workunit.client.0.smithi027.stdout:6/596: chown da/d21/dab/d1f 1951 1 2024-04-20T10:59:21.158 INFO:tasks.workunit.client.0.smithi027.stdout:9/502: symlink d4/d1b/l9c 0 2024-04-20T10:59:21.160 INFO:tasks.workunit.client.0.smithi027.stdout:3/598: creat d2/dd/df/d23/d68/fbb x:0 0 0 2024-04-20T10:59:21.164 INFO:tasks.workunit.client.0.smithi027.stdout:5/538: creat d19/d1b/d1c/fad x:0 0 0 2024-04-20T10:59:21.164 INFO:tasks.workunit.client.0.smithi027.stdout:5/539: write d19/d1b/d1c/d1e/d22/f89 [126869,97470] 0 2024-04-20T10:59:21.166 INFO:tasks.workunit.client.0.smithi027.stdout:5/540: dread d19/d1b/d1c/d1e/d58/f6c [0,4194304] 0 2024-04-20T10:59:21.166 INFO:tasks.workunit.client.0.smithi027.stdout:4/561: symlink d0/d14/d6b/lce 0 2024-04-20T10:59:21.173 INFO:tasks.workunit.client.0.smithi027.stdout:2/520: mknod d5/dc/d16/d37/ca9 0 2024-04-20T10:59:21.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/734: link d3/d7/lae d3/d7/d38/l10b 0 2024-04-20T10:59:21.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/735: write d3/d41/dc6/de6/fea [411970,76520] 0 2024-04-20T10:59:21.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/736: write d3/d7/d28/f59 [4857845,36558] 0 2024-04-20T10:59:21.175 INFO:tasks.workunit.client.0.smithi027.stdout:0/687: creat d0/d11/d2c/d2e/d2b/ff2 x:0 0 0 2024-04-20T10:59:21.175 INFO:tasks.workunit.client.0.smithi027.stdout:0/688: dread - d0/d11/d2c/d2e/d2b/d7b/dcf/fed zero size 2024-04-20T10:59:21.175 INFO:tasks.workunit.client.0.smithi027.stdout:1/737: dread d3/d45/d5d/d6c/f72 [0,4194304] 0 2024-04-20T10:59:21.175 INFO:tasks.workunit.client.0.smithi027.stdout:7/544: mknod d4/d5/d8/d46/cab 0 2024-04-20T10:59:21.176 INFO:tasks.workunit.client.0.smithi027.stdout:7/545: truncate d4/d5b/f9e 1939821 0 2024-04-20T10:59:21.176 INFO:tasks.workunit.client.0.smithi027.stdout:6/597: mknod da/d21/dab/d19/d2e/d32/cc1 0 2024-04-20T10:59:21.177 INFO:tasks.workunit.client.0.smithi027.stdout:9/503: creat d4/d1b/d50/d5d/f9d x:0 0 0 2024-04-20T10:59:21.178 INFO:tasks.workunit.client.0.smithi027.stdout:3/599: getdents d2/dd/df/d33 0 2024-04-20T10:59:21.178 INFO:tasks.workunit.client.0.smithi027.stdout:5/541: creat d19/d1b/d1c/d1e/d25/d40/d82/fae x:0 0 0 2024-04-20T10:59:21.178 INFO:tasks.workunit.client.0.smithi027.stdout:5/542: read d19/d1b/d1c/d1e/d58/f60 [41913,121333] 0 2024-04-20T10:59:21.180 INFO:tasks.workunit.client.0.smithi027.stdout:2/521: dread d5/d44/da7/f48 [0,4194304] 0 2024-04-20T10:59:21.180 INFO:tasks.workunit.client.0.smithi027.stdout:2/522: chown d5/dc/d16/d2a/d59 11746 1 2024-04-20T10:59:21.180 INFO:tasks.workunit.client.0.smithi027.stdout:4/562: unlink d0/d48/f91 0 2024-04-20T10:59:21.180 INFO:tasks.workunit.client.0.smithi027.stdout:4/563: fdatasync d0/d14/d6b/da3/d1b/d33/f9a 0 2024-04-20T10:59:21.181 INFO:tasks.workunit.client.0.smithi027.stdout:4/564: chown d0/d14/d6b/da3/d18/db9 579031399 1 2024-04-20T10:59:21.181 INFO:tasks.workunit.client.0.smithi027.stdout:0/689: getdents d0/d11/d2c/d2e/d2b/d7b/dcf 0 2024-04-20T10:59:21.182 INFO:tasks.workunit.client.0.smithi027.stdout:1/738: truncate d3/d7/d38/ddf/fd8 953768 0 2024-04-20T10:59:21.182 INFO:tasks.workunit.client.0.smithi027.stdout:1/739: fsync d3/d7/d28/f100 0 2024-04-20T10:59:21.185 INFO:tasks.workunit.client.0.smithi027.stdout:6/598: rmdir da/d21 39 2024-04-20T10:59:21.186 INFO:tasks.workunit.client.0.smithi027.stdout:9/504: unlink d4/d1b/d31/d23/f7a 0 2024-04-20T10:59:21.186 INFO:tasks.workunit.client.0.smithi027.stdout:7/546: dread d4/d5b/f67 [0,4194304] 0 2024-04-20T10:59:21.186 INFO:tasks.workunit.client.0.smithi027.stdout:7/547: readlink d4/d5/d8/d45/l7d 0 2024-04-20T10:59:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:7/548: dread d4/f2e [0,4194304] 0 2024-04-20T10:59:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:7/549: write d4/d5/f1f [967426,117643] 0 2024-04-20T10:59:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:7/550: chown d4/d1b/d58/f75 4 1 2024-04-20T10:59:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:3/600: truncate d2/f16 5333134 0 2024-04-20T10:59:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:2/523: creat d5/d44/da7/da0/faa x:0 0 0 2024-04-20T10:59:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:0/690: symlink d0/d11/d2c/d2e/d2b/d7b/d34/lf3 0 2024-04-20T10:59:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:5/543: write d19/d1b/d1c/d1e/d3c/f96 [87945,81390] 0 2024-04-20T10:59:21.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/740: rmdir d3/d7/d38/da4/dbc 39 2024-04-20T10:59:21.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/741: dread - d3/d7/d38/ddf/f84 zero size 2024-04-20T10:59:21.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/742: chown d3/f1c 7520 1 2024-04-20T10:59:21.191 INFO:tasks.workunit.client.0.smithi027.stdout:8/652: sync 2024-04-20T10:59:21.192 INFO:tasks.workunit.client.0.smithi027.stdout:9/505: mkdir d4/d1b/d1f/d45/d9e 0 2024-04-20T10:59:21.192 INFO:tasks.workunit.client.0.smithi027.stdout:9/506: read d4/d1b/d1f/d45/d8a/f6e [99813,96624] 0 2024-04-20T10:59:21.193 INFO:tasks.workunit.client.0.smithi027.stdout:7/551: creat d4/d5/d8/fac x:0 0 0 2024-04-20T10:59:21.195 INFO:tasks.workunit.client.0.smithi027.stdout:2/524: mkdir d5/dc/d16/d32/dab 0 2024-04-20T10:59:21.195 INFO:tasks.workunit.client.0.smithi027.stdout:2/525: fsync d5/dc/d16/d2a/f84 0 2024-04-20T10:59:21.196 INFO:tasks.workunit.client.0.smithi027.stdout:0/691: creat d0/d11/d2c/d2e/d2b/d7b/d12/d25/ff4 x:0 0 0 2024-04-20T10:59:21.196 INFO:tasks.workunit.client.0.smithi027.stdout:0/692: stat d0/d11/d2c/d2e/d35 0 2024-04-20T10:59:21.196 INFO:tasks.workunit.client.0.smithi027.stdout:0/693: read - d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/fbe zero size 2024-04-20T10:59:21.198 INFO:tasks.workunit.client.0.smithi027.stdout:5/544: creat d19/d1b/d1c/d1e/d25/faf x:0 0 0 2024-04-20T10:59:21.198 INFO:tasks.workunit.client.0.smithi027.stdout:4/565: dwrite d0/fd [0,4194304] 0 2024-04-20T10:59:21.199 INFO:tasks.workunit.client.0.smithi027.stdout:9/507: creat d4/f9f x:0 0 0 2024-04-20T10:59:21.200 INFO:tasks.workunit.client.0.smithi027.stdout:7/552: mknod d4/d5/cad 0 2024-04-20T10:59:21.200 INFO:tasks.workunit.client.0.smithi027.stdout:7/553: chown d4/d5/d8/f85 39022935 1 2024-04-20T10:59:21.201 INFO:tasks.workunit.client.0.smithi027.stdout:0/694: creat d0/d59/dbf/ff5 x:0 0 0 2024-04-20T10:59:21.202 INFO:tasks.workunit.client.0.smithi027.stdout:4/566: dread d0/d14/d6b/da3/d1b/f21 [4194304,4194304] 0 2024-04-20T10:59:21.203 INFO:tasks.workunit.client.0.smithi027.stdout:5/545: mknod d19/d1b/d1c/cb0 0 2024-04-20T10:59:21.203 INFO:tasks.workunit.client.0.smithi027.stdout:5/546: truncate d19/d1b/d1c/d1e/d3c/f96 524149 0 2024-04-20T10:59:21.203 INFO:tasks.workunit.client.0.smithi027.stdout:8/653: dread d9/fc5 [0,4194304] 0 2024-04-20T10:59:21.206 INFO:tasks.workunit.client.0.smithi027.stdout:9/508: chown d4/d2a/c2f 1 1 2024-04-20T10:59:21.206 INFO:tasks.workunit.client.0.smithi027.stdout:7/554: creat d4/d5b/fae x:0 0 0 2024-04-20T10:59:21.206 INFO:tasks.workunit.client.0.smithi027.stdout:9/509: chown d4/d1b/d31/d23/d29/d7f/l84 9 1 2024-04-20T10:59:21.209 INFO:tasks.workunit.client.0.smithi027.stdout:5/547: rename d19/d1b/d1c/d1e/d22/d47 to d19/d1b/d1c/d1e/d22/db1 0 2024-04-20T10:59:21.209 INFO:tasks.workunit.client.0.smithi027.stdout:9/510: getdents d4/d74/d72 0 2024-04-20T10:59:21.211 INFO:tasks.workunit.client.0.smithi027.stdout:7/555: creat d4/d1b/d4a/d62/faf x:0 0 0 2024-04-20T10:59:21.212 INFO:tasks.workunit.client.0.smithi027.stdout:7/556: creat d4/d1b/fb0 x:0 0 0 2024-04-20T10:59:21.215 INFO:tasks.workunit.client.0.smithi027.stdout:7/557: unlink d4/d5/d8/d3b/l76 0 2024-04-20T10:59:21.216 INFO:tasks.workunit.client.0.smithi027.stdout:4/567: dread d0/d14/d6b/da3/d1b/f30 [0,4194304] 0 2024-04-20T10:59:21.216 INFO:tasks.workunit.client.0.smithi027.stdout:4/568: write d0/d14/d6b/da3/d18/f54 [723863,14031] 0 2024-04-20T10:59:21.217 INFO:tasks.workunit.client.0.smithi027.stdout:4/569: write d0/d14/d6b/da3/d1b/f4c [766322,45486] 0 2024-04-20T10:59:21.218 INFO:tasks.workunit.client.0.smithi027.stdout:4/570: creat d0/fcf x:0 0 0 2024-04-20T10:59:21.220 INFO:tasks.workunit.client.0.smithi027.stdout:4/571: link d0/d14/d6b/da3/d1b/d2f/f82 d0/d14/d6b/da3/d1b/d33/fd0 0 2024-04-20T10:59:21.228 INFO:tasks.workunit.client.0.smithi027.stdout:9/511: write d4/d2a/f69 [3971265,128499] 0 2024-04-20T10:59:21.228 INFO:tasks.workunit.client.0.smithi027.stdout:9/512: chown d4/d1b/d31/f75 0 1 2024-04-20T10:59:21.228 INFO:tasks.workunit.client.0.smithi027.stdout:9/513: write d4/d1b/d50/d5d/f8c [560386,85112] 0 2024-04-20T10:59:21.228 INFO:tasks.workunit.client.0.smithi027.stdout:9/514: readlink d4/d1b/d31/l8b 0 2024-04-20T10:59:21.231 INFO:tasks.workunit.client.0.smithi027.stdout:5/548: dwrite fc [0,4194304] 0 2024-04-20T10:59:21.233 INFO:tasks.workunit.client.0.smithi027.stdout:5/549: symlink d19/d1b/d1c/d1e/d25/d40/d78/lb2 0 2024-04-20T10:59:21.234 INFO:tasks.workunit.client.0.smithi027.stdout:5/550: write d19/d1b/d1c/d1e/f88 [986431,125056] 0 2024-04-20T10:59:21.236 INFO:tasks.workunit.client.0.smithi027.stdout:4/572: dread d0/d5e/d6c/fbc [0,4194304] 0 2024-04-20T10:59:21.247 INFO:tasks.workunit.client.0.smithi027.stdout:9/515: read d4/d1b/d1f/d5a/f79 [2492472,55999] 0 2024-04-20T10:59:21.248 INFO:tasks.workunit.client.0.smithi027.stdout:9/516: symlink d4/d1b/d1f/la0 0 2024-04-20T10:59:21.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/517: truncate d4/d2a/f77 127823 0 2024-04-20T10:59:21.249 INFO:tasks.workunit.client.0.smithi027.stdout:6/599: dwrite da/d21/dab/f1a [0,4194304] 0 2024-04-20T10:59:21.249 INFO:tasks.workunit.client.0.smithi027.stdout:6/600: dread - da/d21/dab/fb5 zero size 2024-04-20T10:59:21.249 INFO:tasks.workunit.client.0.smithi027.stdout:6/601: fsync da/d21/d28/fad 0 2024-04-20T10:59:21.250 INFO:tasks.workunit.client.0.smithi027.stdout:3/601: dwrite d2/dd/d57/d8c/fb8 [0,4194304] 0 2024-04-20T10:59:21.250 INFO:tasks.workunit.client.0.smithi027.stdout:3/602: fdatasync d2/d18/d2e/f75 0 2024-04-20T10:59:21.250 INFO:tasks.workunit.client.0.smithi027.stdout:8/654: dwrite d9/de/d1e/d4a/f45 [0,4194304] 0 2024-04-20T10:59:21.250 INFO:tasks.workunit.client.0.smithi027.stdout:1/743: dwrite d3/d7/d38/ddf/d73/fc4 [0,4194304] 0 2024-04-20T10:59:21.252 INFO:tasks.workunit.client.0.smithi027.stdout:0/695: dwrite d0/d11/d2c/d2e/d55/fac [4194304,4194304] 0 2024-04-20T10:59:21.252 INFO:tasks.workunit.client.0.smithi027.stdout:0/696: read - d0/d59/dbf/ff5 zero size 2024-04-20T10:59:21.257 INFO:tasks.workunit.client.0.smithi027.stdout:9/518: truncate d4/d1b/d31/d51/f94 1913678 0 2024-04-20T10:59:21.257 INFO:tasks.workunit.client.0.smithi027.stdout:3/603: creat d2/d80/fbc x:0 0 0 2024-04-20T10:59:21.257 INFO:tasks.workunit.client.0.smithi027.stdout:8/655: creat d9/de/d1e/d4a/d6d/d79/fe4 x:0 0 0 2024-04-20T10:59:21.257 INFO:tasks.workunit.client.0.smithi027.stdout:8/656: write d9/de/d1e/f65 [996319,41783] 0 2024-04-20T10:59:21.264 INFO:tasks.workunit.client.0.smithi027.stdout:1/744: symlink d3/d7/d38/l10c 0 2024-04-20T10:59:21.264 INFO:tasks.workunit.client.0.smithi027.stdout:0/697: unlink d0/d11/d2c/f9d 0 2024-04-20T10:59:21.264 INFO:tasks.workunit.client.0.smithi027.stdout:9/519: unlink d4/d14/f38 0 2024-04-20T10:59:21.264 INFO:tasks.workunit.client.0.smithi027.stdout:9/520: read - d4/d1b/f53 zero size 2024-04-20T10:59:21.264 INFO:tasks.workunit.client.0.smithi027.stdout:0/698: truncate d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/f69 899563 0 2024-04-20T10:59:21.264 INFO:tasks.workunit.client.0.smithi027.stdout:0/699: write d0/d11/d4b/d5b/d6f/fe7 [711906,84812] 0 2024-04-20T10:59:21.265 INFO:tasks.workunit.client.0.smithi027.stdout:5/551: rmdir d19/d1b/d1c/d1e/d22/db1 39 2024-04-20T10:59:21.265 INFO:tasks.workunit.client.0.smithi027.stdout:1/745: creat d3/d7/d38/ddf/de9/f10d x:0 0 0 2024-04-20T10:59:21.265 INFO:tasks.workunit.client.0.smithi027.stdout:2/526: dwrite d5/dc/d16/d60/f69 [0,4194304] 0 2024-04-20T10:59:21.265 INFO:tasks.workunit.client.0.smithi027.stdout:7/558: dwrite d4/d5/f33 [0,4194304] 0 2024-04-20T10:59:21.266 INFO:tasks.workunit.client.0.smithi027.stdout:0/700: dread d0/d11/d2c/fcd [0,4194304] 0 2024-04-20T10:59:21.266 INFO:tasks.workunit.client.0.smithi027.stdout:1/746: unlink d3/d7/d38/ddf/l6d 0 2024-04-20T10:59:21.266 INFO:tasks.workunit.client.0.smithi027.stdout:1/747: truncate d3/d7/d38/ddf/f5a 4484095 0 2024-04-20T10:59:21.271 INFO:tasks.workunit.client.0.smithi027.stdout:7/559: mknod d4/d1b/d58/cb1 0 2024-04-20T10:59:21.271 INFO:tasks.workunit.client.0.smithi027.stdout:9/521: dread d4/d1b/d31/f3d [0,4194304] 0 2024-04-20T10:59:21.278 INFO:tasks.workunit.client.0.smithi027.stdout:0/701: link d0/d11/d2c/d2e/l46 d0/d11/d2c/d2e/d55/lf6 0 2024-04-20T10:59:21.279 INFO:tasks.workunit.client.0.smithi027.stdout:1/748: creat d3/d7e/f10e x:0 0 0 2024-04-20T10:59:21.282 INFO:tasks.workunit.client.0.smithi027.stdout:7/560: mknod d4/d1b/d4a/d7a/d93/cb2 0 2024-04-20T10:59:21.282 INFO:tasks.workunit.client.0.smithi027.stdout:7/561: fdatasync d4/d5b/f7e 0 2024-04-20T10:59:21.283 INFO:tasks.workunit.client.0.smithi027.stdout:7/562: write d4/d5/f1f [1646140,95446] 0 2024-04-20T10:59:21.283 INFO:tasks.workunit.client.0.smithi027.stdout:7/563: stat d4/d5/d8/d45 0 2024-04-20T10:59:21.283 INFO:tasks.workunit.client.0.smithi027.stdout:7/564: chown d4/d1b/c23 11810 1 2024-04-20T10:59:21.286 INFO:tasks.workunit.client.0.smithi027.stdout:7/565: rename d4/d5b/f7e to d4/d1b/d4a/fb3 0 2024-04-20T10:59:21.288 INFO:tasks.workunit.client.0.smithi027.stdout:7/566: link d4/d1b/d4a/fa6 d4/fb4 0 2024-04-20T10:59:21.290 INFO:tasks.workunit.client.0.smithi027.stdout:9/522: dread d4/d14/f49 [0,4194304] 0 2024-04-20T10:59:21.291 INFO:tasks.workunit.client.0.smithi027.stdout:7/567: truncate d4/d5/f33 440118 0 2024-04-20T10:59:21.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/568: creat d4/da2/fb5 x:0 0 0 2024-04-20T10:59:21.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/569: write d4/d5/d81/f8e [4542335,69378] 0 2024-04-20T10:59:21.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/570: read - d4/d5/d8/d21/f96 zero size 2024-04-20T10:59:21.294 INFO:tasks.workunit.client.0.smithi027.stdout:7/571: creat d4/d1b/fb6 x:0 0 0 2024-04-20T10:59:21.294 INFO:tasks.workunit.client.0.smithi027.stdout:7/572: chown d4/d5/d8/d3b/laa 0 1 2024-04-20T10:59:21.294 INFO:tasks.workunit.client.0.smithi027.stdout:0/702: dread d0/d11/d2c/d2e/d2b/d7b/d34/f3b [0,4194304] 0 2024-04-20T10:59:21.295 INFO:tasks.workunit.client.0.smithi027.stdout:0/703: write d0/d11/d2c/d2e/d55/f6b [443744,99248] 0 2024-04-20T10:59:21.295 INFO:tasks.workunit.client.0.smithi027.stdout:7/573: mknod d4/d1b/d4a/d7a/cb7 0 2024-04-20T10:59:21.296 INFO:tasks.workunit.client.0.smithi027.stdout:0/704: creat d0/d11/d2c/ff7 x:0 0 0 2024-04-20T10:59:21.297 INFO:tasks.workunit.client.0.smithi027.stdout:7/574: creat d4/d1b/d58/fb8 x:0 0 0 2024-04-20T10:59:21.298 INFO:tasks.workunit.client.0.smithi027.stdout:0/705: unlink d0/d11/f54 0 2024-04-20T10:59:21.298 INFO:tasks.workunit.client.0.smithi027.stdout:0/706: write d0/d59/dbf/fd2 [656667,19177] 0 2024-04-20T10:59:21.299 INFO:tasks.workunit.client.0.smithi027.stdout:3/604: dwrite d2/d7f/fa4 [0,4194304] 0 2024-04-20T10:59:21.299 INFO:tasks.workunit.client.0.smithi027.stdout:2/527: dwrite d5/dc/d16/d2a/f75 [0,4194304] 0 2024-04-20T10:59:21.300 INFO:tasks.workunit.client.0.smithi027.stdout:3/605: write d2/d18/d2c/f69 [4947663,78651] 0 2024-04-20T10:59:21.300 INFO:tasks.workunit.client.0.smithi027.stdout:6/602: dwrite da/d21/dab/d19/d2e/f38 [0,4194304] 0 2024-04-20T10:59:21.300 INFO:tasks.workunit.client.0.smithi027.stdout:6/603: write da/d21/d28/fbb [614899,92080] 0 2024-04-20T10:59:21.300 INFO:tasks.workunit.client.0.smithi027.stdout:7/575: rename d4/d1b/d4a/fa6 to d4/d5/d8/d21/d94/da7/fb9 0 2024-04-20T10:59:21.301 INFO:tasks.workunit.client.0.smithi027.stdout:5/552: dwrite d19/d1b/d1c/f98 [0,4194304] 0 2024-04-20T10:59:21.302 INFO:tasks.workunit.client.0.smithi027.stdout:0/707: rmdir d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f 39 2024-04-20T10:59:21.302 INFO:tasks.workunit.client.0.smithi027.stdout:0/708: readlink d0/d11/d2c/d2e/d35/d8c/lbc 0 2024-04-20T10:59:21.305 INFO:tasks.workunit.client.0.smithi027.stdout:1/749: dwrite d3/d7/d38/ddf/f3f [0,4194304] 0 2024-04-20T10:59:21.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/606: mknod d2/dd/df/d23/d68/d88/cbd 0 2024-04-20T10:59:21.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/607: stat d2/dd/d22/fa2 0 2024-04-20T10:59:21.309 INFO:tasks.workunit.client.0.smithi027.stdout:2/528: rename d5/dc/d16/d2a/d4a to d5/dc/d16/d60/dac 0 2024-04-20T10:59:21.309 INFO:tasks.workunit.client.0.smithi027.stdout:6/604: rename da/d21/dab to da/d21/dab/d1f/dc2 22 2024-04-20T10:59:21.309 INFO:tasks.workunit.client.0.smithi027.stdout:2/529: chown d5/d26/d78/f7b 720650365 1 2024-04-20T10:59:21.310 INFO:tasks.workunit.client.0.smithi027.stdout:5/553: creat d19/d1b/d1c/d1e/d22/db1/fb3 x:0 0 0 2024-04-20T10:59:21.312 INFO:tasks.workunit.client.0.smithi027.stdout:3/608: mknod d2/dd/d57/cbe 0 2024-04-20T10:59:21.313 INFO:tasks.workunit.client.0.smithi027.stdout:5/554: fdatasync d19/d1b/d1c/d1e/d3c/f74 0 2024-04-20T10:59:21.317 INFO:tasks.workunit.client.0.smithi027.stdout:0/709: write d0/d11/d2c/d2e/d35/fb5 [5735025,19572] 0 2024-04-20T10:59:21.317 INFO:tasks.workunit.client.0.smithi027.stdout:6/605: dread da/d21/dab/d19/f5d [0,4194304] 0 2024-04-20T10:59:21.317 INFO:tasks.workunit.client.0.smithi027.stdout:6/606: fdatasync da/d21/d28/fad 0 2024-04-20T10:59:21.317 INFO:tasks.workunit.client.0.smithi027.stdout:6/607: dread - da/f99 zero size 2024-04-20T10:59:21.318 INFO:tasks.workunit.client.0.smithi027.stdout:0/710: mknod d0/d11/d2c/d2e/d2b/d7b/d34/d5a/cf8 0 2024-04-20T10:59:21.318 INFO:tasks.workunit.client.0.smithi027.stdout:0/711: fsync d0/d11/d2c/d2e/d61/fe1 0 2024-04-20T10:59:21.318 INFO:tasks.workunit.client.0.smithi027.stdout:9/523: dwrite d4/d1b/d8f/f90 [0,4194304] 0 2024-04-20T10:59:21.320 INFO:tasks.workunit.client.0.smithi027.stdout:0/712: creat d0/d11/d4b/d5b/d6f/d82/dd1/deb/ff9 x:0 0 0 2024-04-20T10:59:21.322 INFO:tasks.workunit.client.0.smithi027.stdout:3/609: dread d2/f16 [0,4194304] 0 2024-04-20T10:59:21.322 INFO:tasks.workunit.client.0.smithi027.stdout:0/713: truncate d0/d11/d2c/d2e/d55/fac 3476304 0 2024-04-20T10:59:21.323 INFO:tasks.workunit.client.0.smithi027.stdout:3/610: creat d2/dd/d22/fbf x:0 0 0 2024-04-20T10:59:21.326 INFO:tasks.workunit.client.0.smithi027.stdout:0/714: unlink d0/d11/d4b/d5b/d6f/d82/fa0 0 2024-04-20T10:59:21.326 INFO:tasks.workunit.client.0.smithi027.stdout:0/715: dread - d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/fe2 zero size 2024-04-20T10:59:21.327 INFO:tasks.workunit.client.0.smithi027.stdout:3/611: creat d2/dd/fc0 x:0 0 0 2024-04-20T10:59:21.327 INFO:tasks.workunit.client.0.smithi027.stdout:9/524: dread d4/d2a/f66 [0,4194304] 0 2024-04-20T10:59:21.327 INFO:tasks.workunit.client.0.smithi027.stdout:0/716: creat d0/d59/dbf/ffa x:0 0 0 2024-04-20T10:59:21.333 INFO:tasks.workunit.client.0.smithi027.stdout:3/612: symlink d2/dd/df/lc1 0 2024-04-20T10:59:21.333 INFO:tasks.workunit.client.0.smithi027.stdout:3/613: write d2/f17 [2508894,31567] 0 2024-04-20T10:59:21.333 INFO:tasks.workunit.client.0.smithi027.stdout:3/614: write d2/dd/fc0 [336957,3399] 0 2024-04-20T10:59:21.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/525: getdents d4/d1b/d1f/d5a 0 2024-04-20T10:59:21.333 INFO:tasks.workunit.client.0.smithi027.stdout:9/526: chown d4/d14/d43/c95 3698 1 2024-04-20T10:59:21.334 INFO:tasks.workunit.client.0.smithi027.stdout:7/576: dwrite d4/d1b/d58/fb8 [0,4194304] 0 2024-04-20T10:59:21.334 INFO:tasks.workunit.client.0.smithi027.stdout:7/577: stat d4/d1b/f35 0 2024-04-20T10:59:21.334 INFO:tasks.workunit.client.0.smithi027.stdout:5/555: dread f17 [0,4194304] 0 2024-04-20T10:59:21.334 INFO:tasks.workunit.client.0.smithi027.stdout:5/556: fdatasync d19/d1b/d1c/d1e/d25/f2f 0 2024-04-20T10:59:21.335 INFO:tasks.workunit.client.0.smithi027.stdout:5/557: readlink d19/d1b/d1c/d1e/d22/d68/l80 0 2024-04-20T10:59:21.335 INFO:tasks.workunit.client.0.smithi027.stdout:5/558: write d19/d1b/d1c/d1e/d22/d68/d75/f77 [861661,62477] 0 2024-04-20T10:59:21.335 INFO:tasks.workunit.client.0.smithi027.stdout:1/750: dwrite d3/d7/d28/f70 [0,4194304] 0 2024-04-20T10:59:21.336 INFO:tasks.workunit.client.0.smithi027.stdout:1/751: dread - d3/d20/f96 zero size 2024-04-20T10:59:21.336 INFO:tasks.workunit.client.0.smithi027.stdout:3/615: link d2/dd/f77 d2/dd/d57/d8d/fc2 0 2024-04-20T10:59:21.336 INFO:tasks.workunit.client.0.smithi027.stdout:3/616: stat d2/dd/df/faa 0 2024-04-20T10:59:21.336 INFO:tasks.workunit.client.0.smithi027.stdout:2/530: dwrite d5/dc/d16/d60/dac/f94 [0,4194304] 0 2024-04-20T10:59:21.337 INFO:tasks.workunit.client.0.smithi027.stdout:1/752: dread d3/d7/d38/da4/dac/fc0 [0,4194304] 0 2024-04-20T10:59:21.337 INFO:tasks.workunit.client.0.smithi027.stdout:1/753: fsync d3/d7/d38/ddf/dcc/fd0 0 2024-04-20T10:59:21.337 INFO:tasks.workunit.client.0.smithi027.stdout:6/608: dread da/d21/d28/f3a [0,4194304] 0 2024-04-20T10:59:21.337 INFO:tasks.workunit.client.0.smithi027.stdout:6/609: dread da/dd/d79/fa5 [0,4194304] 0 2024-04-20T10:59:21.340 INFO:tasks.workunit.client.0.smithi027.stdout:7/578: link d4/c3a d4/d5/d8/d45/cba 0 2024-04-20T10:59:21.340 INFO:tasks.workunit.client.0.smithi027.stdout:5/559: mkdir d19/d1b/d1c/d1e/d58/db4 0 2024-04-20T10:59:21.345 INFO:tasks.workunit.client.0.smithi027.stdout:3/617: rename d2/d80/f6f to d2/d18/d2e/fc3 0 2024-04-20T10:59:21.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/717: dwrite d0/d11/d2c/da1/fe3 [0,4194304] 0 2024-04-20T10:59:21.346 INFO:tasks.workunit.client.0.smithi027.stdout:0/718: chown d0/d59/f73 7 1 2024-04-20T10:59:21.346 INFO:tasks.workunit.client.0.smithi027.stdout:9/527: link d4/d1b/d31/d23/d29/f82 d4/d14/d43/fa1 0 2024-04-20T10:59:21.349 INFO:tasks.workunit.client.0.smithi027.stdout:2/531: creat d5/dc/d16/d2a/d59/fad x:0 0 0 2024-04-20T10:59:21.350 INFO:tasks.workunit.client.0.smithi027.stdout:1/754: symlink d3/d20/d107/l10f 0 2024-04-20T10:59:21.351 INFO:tasks.workunit.client.0.smithi027.stdout:6/610: mkdir da/d21/d28/d2d/dc3 0 2024-04-20T10:59:21.354 INFO:tasks.workunit.client.0.smithi027.stdout:3/618: mkdir d2/d94/dc4 0 2024-04-20T10:59:21.355 INFO:tasks.workunit.client.0.smithi027.stdout:0/719: mknod d0/d11/d2c/d2e/d2b/d7b/cfb 0 2024-04-20T10:59:21.356 INFO:tasks.workunit.client.0.smithi027.stdout:9/528: creat d4/d1b/d50/fa2 x:0 0 0 2024-04-20T10:59:21.357 INFO:tasks.workunit.client.0.smithi027.stdout:8/657: sync 2024-04-20T10:59:21.357 INFO:tasks.workunit.client.0.smithi027.stdout:8/658: stat d9/de/d20/d48/d59/c7a 0 2024-04-20T10:59:21.358 INFO:tasks.workunit.client.0.smithi027.stdout:1/755: creat d3/d7/d28/db0/f110 x:0 0 0 2024-04-20T10:59:21.358 INFO:tasks.workunit.client.0.smithi027.stdout:4/573: sync 2024-04-20T10:59:21.358 INFO:tasks.workunit.client.0.smithi027.stdout:4/574: write d0/d14/fc4 [299336,21361] 0 2024-04-20T10:59:21.359 INFO:tasks.workunit.client.0.smithi027.stdout:3/619: dread d2/d94/f95 [0,4194304] 0 2024-04-20T10:59:21.360 INFO:tasks.workunit.client.0.smithi027.stdout:4/575: unlink d0/d14/d6b/da3/f16 0 2024-04-20T10:59:21.362 INFO:tasks.workunit.client.0.smithi027.stdout:4/576: write d0/d14/d6b/da3/d1b/f9c [259912,75983] 0 2024-04-20T10:59:21.362 INFO:tasks.workunit.client.0.smithi027.stdout:4/577: readlink d0/d14/d6b/da3/d18/l81 0 2024-04-20T10:59:21.362 INFO:tasks.workunit.client.0.smithi027.stdout:4/578: write d0/d14/d6b/da3/d1b/d33/d5c/d6f/f74 [600209,12109] 0 2024-04-20T10:59:21.365 INFO:tasks.workunit.client.0.smithi027.stdout:4/579: mkdir d0/d14/d6b/da3/d1b/d33/d5c/d6f/dd1 0 2024-04-20T10:59:21.365 INFO:tasks.workunit.client.0.smithi027.stdout:4/580: readlink d0/d14/d6b/da3/d18/lb3 0 2024-04-20T10:59:21.365 INFO:tasks.workunit.client.0.smithi027.stdout:4/581: chown d0/fb 5 1 2024-04-20T10:59:21.367 INFO:tasks.workunit.client.0.smithi027.stdout:7/579: dwrite d4/d1b/d4a/d62/d87/f9b [0,4194304] 0 2024-04-20T10:59:21.369 INFO:tasks.workunit.client.0.smithi027.stdout:7/580: creat d4/d5/d8/d21/fbb x:0 0 0 2024-04-20T10:59:21.369 INFO:tasks.workunit.client.0.smithi027.stdout:7/581: write d4/d1b/f88 [44496,65427] 0 2024-04-20T10:59:21.369 INFO:tasks.workunit.client.0.smithi027.stdout:7/582: readlink d4/d5/d8/d3b/l51 0 2024-04-20T10:59:21.369 INFO:tasks.workunit.client.0.smithi027.stdout:7/583: fsync d4/d1b/d58/f75 0 2024-04-20T10:59:21.370 INFO:tasks.workunit.client.0.smithi027.stdout:7/584: write d4/d1b/d4a/d62/faf [243313,19743] 0 2024-04-20T10:59:21.370 INFO:tasks.workunit.client.0.smithi027.stdout:4/582: dread d0/f3a [0,4194304] 0 2024-04-20T10:59:21.371 INFO:tasks.workunit.client.0.smithi027.stdout:4/583: chown d0/d14/l2b 0 1 2024-04-20T10:59:21.371 INFO:tasks.workunit.client.0.smithi027.stdout:4/584: chown d0/d14/d6b/da3/d53/la6 14542327 1 2024-04-20T10:59:21.371 INFO:tasks.workunit.client.0.smithi027.stdout:7/585: unlink d4/d5/d8/f85 0 2024-04-20T10:59:21.371 INFO:tasks.workunit.client.0.smithi027.stdout:2/532: dwrite d5/d26/d30/f3c [0,4194304] 0 2024-04-20T10:59:21.372 INFO:tasks.workunit.client.0.smithi027.stdout:4/585: creat d0/d14/d6b/da3/d1b/d33/d5c/fd2 x:0 0 0 2024-04-20T10:59:21.372 INFO:tasks.workunit.client.0.smithi027.stdout:4/586: write d0/d14/d6b/da3/d1b/d33/d5c/fd2 [542575,29833] 0 2024-04-20T10:59:21.372 INFO:tasks.workunit.client.0.smithi027.stdout:4/587: chown d0/d14/d6b/da3/f9e 84 1 2024-04-20T10:59:21.372 INFO:tasks.workunit.client.0.smithi027.stdout:4/588: readlink d0/d14/d6b/da3/d1b/d2f/l32 0 2024-04-20T10:59:21.372 INFO:tasks.workunit.client.0.smithi027.stdout:7/586: truncate d4/f3d 370908 0 2024-04-20T10:59:21.372 INFO:tasks.workunit.client.0.smithi027.stdout:7/587: fsync d4/d5/d8/f4c 0 2024-04-20T10:59:21.373 INFO:tasks.workunit.client.0.smithi027.stdout:2/533: rmdir d5/dc/d16/d60 39 2024-04-20T10:59:21.373 INFO:tasks.workunit.client.0.smithi027.stdout:2/534: fdatasync d5/d44/fa5 0 2024-04-20T10:59:21.373 INFO:tasks.workunit.client.0.smithi027.stdout:1/756: write d3/d7/fc [2063422,39765] 0 2024-04-20T10:59:21.373 INFO:tasks.workunit.client.0.smithi027.stdout:4/589: symlink d0/d14/d6b/da3/d1b/d33/ld3 0 2024-04-20T10:59:21.374 INFO:tasks.workunit.client.0.smithi027.stdout:7/588: creat d4/d4d/fbc x:0 0 0 2024-04-20T10:59:21.374 INFO:tasks.workunit.client.0.smithi027.stdout:2/535: symlink d5/lae 0 2024-04-20T10:59:21.374 INFO:tasks.workunit.client.0.smithi027.stdout:2/536: truncate d5/d1e/f70 4322384 0 2024-04-20T10:59:21.375 INFO:tasks.workunit.client.0.smithi027.stdout:1/757: mknod d3/d45/d5d/c111 0 2024-04-20T10:59:21.375 INFO:tasks.workunit.client.0.smithi027.stdout:1/758: write d3/d7/f7a [198729,30210] 0 2024-04-20T10:59:21.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/537: mknod d5/dc/d16/d2a/d59/d66/caf 0 2024-04-20T10:59:21.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/538: write d5/dc/d16/d2a/f9f [583962,80332] 0 2024-04-20T10:59:21.378 INFO:tasks.workunit.client.0.smithi027.stdout:1/759: rename d3/d45/fc8 to d3/d7/d38/f112 0 2024-04-20T10:59:21.378 INFO:tasks.workunit.client.0.smithi027.stdout:1/760: write d3/d7e/f9f [258540,118367] 0 2024-04-20T10:59:21.378 INFO:tasks.workunit.client.0.smithi027.stdout:1/761: fsync d3/f1b 0 2024-04-20T10:59:21.378 INFO:tasks.workunit.client.0.smithi027.stdout:1/762: chown d3/d7/d28/d92 251592 1 2024-04-20T10:59:21.378 INFO:tasks.workunit.client.0.smithi027.stdout:1/763: fsync d3/d7/d38/d62/fb9 0 2024-04-20T10:59:21.379 INFO:tasks.workunit.client.0.smithi027.stdout:2/539: unlink d5/dc/d16/d37/f4e 0 2024-04-20T10:59:21.379 INFO:tasks.workunit.client.0.smithi027.stdout:2/540: read - d5/dc/d16/d2a/d59/fad zero size 2024-04-20T10:59:21.379 INFO:tasks.workunit.client.0.smithi027.stdout:8/659: write d9/de/d1e/d4a/d6d/d75/f83 [1601220,42489] 0 2024-04-20T10:59:21.381 INFO:tasks.workunit.client.0.smithi027.stdout:1/764: creat d3/d41/dc6/de6/f113 x:0 0 0 2024-04-20T10:59:21.385 INFO:tasks.workunit.client.0.smithi027.stdout:4/590: read d0/d14/d6b/da3/d1b/d33/f35 [1049212,62647] 0 2024-04-20T10:59:21.386 INFO:tasks.workunit.client.0.smithi027.stdout:0/720: dwrite d0/d11/d2c/fda [0,4194304] 0 2024-04-20T10:59:21.386 INFO:tasks.workunit.client.0.smithi027.stdout:0/721: write d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/fe2 [497950,55145] 0 2024-04-20T10:59:21.386 INFO:tasks.workunit.client.0.smithi027.stdout:0/722: chown d0/d11/d5c/f95 847 1 2024-04-20T10:59:21.386 INFO:tasks.workunit.client.0.smithi027.stdout:5/560: dwrite d19/d1b/d1c/d1e/d22/db1/f50 [0,4194304] 0 2024-04-20T10:59:21.389 INFO:tasks.workunit.client.0.smithi027.stdout:4/591: truncate d0/d14/d6b/da3/d1b/d2f/f73 3297070 0 2024-04-20T10:59:21.389 INFO:tasks.workunit.client.0.smithi027.stdout:9/529: dwrite d4/d2a/f77 [0,4194304] 0 2024-04-20T10:59:21.389 INFO:tasks.workunit.client.0.smithi027.stdout:0/723: dread d0/d11/d2c/d2e/d2b/d3d/f84 [4194304,4194304] 0 2024-04-20T10:59:21.390 INFO:tasks.workunit.client.0.smithi027.stdout:0/724: chown d0/ld0 81 1 2024-04-20T10:59:21.391 INFO:tasks.workunit.client.0.smithi027.stdout:2/541: dread d5/dc/d16/d37/f6b [0,4194304] 0 2024-04-20T10:59:21.392 INFO:tasks.workunit.client.0.smithi027.stdout:6/611: dwrite da/d21/dab/d19/d2e/d6b/d65/fbe [0,4194304] 0 2024-04-20T10:59:21.392 INFO:tasks.workunit.client.0.smithi027.stdout:6/612: write da/dd/d3c/fa8 [1022672,49405] 0 2024-04-20T10:59:21.393 INFO:tasks.workunit.client.0.smithi027.stdout:1/765: getdents d3/d7/d28/db0 0 2024-04-20T10:59:21.393 INFO:tasks.workunit.client.0.smithi027.stdout:1/766: dread - d3/d7/d28/db0/f110 zero size 2024-04-20T10:59:21.394 INFO:tasks.workunit.client.0.smithi027.stdout:1/767: write d3/d20/f46 [1329784,71630] 0 2024-04-20T10:59:21.394 INFO:tasks.workunit.client.0.smithi027.stdout:1/768: chown d3/d7/c19 1422 1 2024-04-20T10:59:21.394 INFO:tasks.workunit.client.0.smithi027.stdout:1/769: write d3/d45/d5d/fd5 [4856612,79972] 0 2024-04-20T10:59:21.395 INFO:tasks.workunit.client.0.smithi027.stdout:9/530: dread d4/d1b/d1f/d45/f7d [0,4194304] 0 2024-04-20T10:59:21.395 INFO:tasks.workunit.client.0.smithi027.stdout:9/531: write d4/d74/d72/f98 [658000,98658] 0 2024-04-20T10:59:21.395 INFO:tasks.workunit.client.0.smithi027.stdout:9/532: write d4/d1b/d8f/f90 [4344345,43857] 0 2024-04-20T10:59:21.398 INFO:tasks.workunit.client.0.smithi027.stdout:7/589: dwrite d4/d1b/fb0 [0,4194304] 0 2024-04-20T10:59:21.400 INFO:tasks.workunit.client.0.smithi027.stdout:0/725: rename d0/d11/d2c/d2e/d61/fe1 to d0/d11/d2c/d2e/d55/ffc 0 2024-04-20T10:59:21.403 INFO:tasks.workunit.client.0.smithi027.stdout:6/613: creat da/d21/dab/d1f/dbf/fc4 x:0 0 0 2024-04-20T10:59:21.403 INFO:tasks.workunit.client.0.smithi027.stdout:1/770: rmdir d3/d41 39 2024-04-20T10:59:21.404 INFO:tasks.workunit.client.0.smithi027.stdout:5/561: write d19/d1b/d1c/f98 [2436068,25306] 0 2024-04-20T10:59:21.404 INFO:tasks.workunit.client.0.smithi027.stdout:9/533: creat d4/d14/fa3 x:0 0 0 2024-04-20T10:59:21.405 INFO:tasks.workunit.client.0.smithi027.stdout:2/542: write d5/d26/f3a [2893154,130213] 0 2024-04-20T10:59:21.408 INFO:tasks.workunit.client.0.smithi027.stdout:7/590: creat d4/d5/d8/d21/d94/fbd x:0 0 0 2024-04-20T10:59:21.414 INFO:tasks.workunit.client.0.smithi027.stdout:6/614: truncate da/d21/d28/d54/f7c 860016 0 2024-04-20T10:59:21.415 INFO:tasks.workunit.client.0.smithi027.stdout:6/615: stat da/d21/dab/d1f/c97 0 2024-04-20T10:59:21.418 INFO:tasks.workunit.client.0.smithi027.stdout:9/534: unlink d4/d14/c2e 0 2024-04-20T10:59:21.418 INFO:tasks.workunit.client.0.smithi027.stdout:7/591: link d4/d5/d8/c25 d4/d1b/d4a/d7a/d93/cbe 0 2024-04-20T10:59:21.418 INFO:tasks.workunit.client.0.smithi027.stdout:6/616: rename da/dd/l50 to da/db8/lc5 0 2024-04-20T10:59:21.418 INFO:tasks.workunit.client.0.smithi027.stdout:5/562: truncate d19/d1b/d1c/d1e/d3c/f39 143403 0 2024-04-20T10:59:21.418 INFO:tasks.workunit.client.0.smithi027.stdout:5/563: dread - d19/d1b/d1c/d1e/d22/d68/f90 zero size 2024-04-20T10:59:21.418 INFO:tasks.workunit.client.0.smithi027.stdout:2/543: dread d5/d26/f3a [0,4194304] 0 2024-04-20T10:59:21.418 INFO:tasks.workunit.client.0.smithi027.stdout:7/592: getdents d4/d1b/d4a/d7a 0 2024-04-20T10:59:21.419 INFO:tasks.workunit.client.0.smithi027.stdout:7/593: read d4/d1b/d4a/f9c [12876,21465] 0 2024-04-20T10:59:21.419 INFO:tasks.workunit.client.0.smithi027.stdout:7/594: readlink d4/d1b/d4a/d7a/l8b 0 2024-04-20T10:59:21.419 INFO:tasks.workunit.client.0.smithi027.stdout:7/595: readlink d4/d5/d8/d45/l7d 0 2024-04-20T10:59:21.420 INFO:tasks.workunit.client.0.smithi027.stdout:6/617: mknod da/dd/d3c/d9e/cc6 0 2024-04-20T10:59:21.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/544: rename d5/c76 to d5/d26/cb0 0 2024-04-20T10:59:21.422 INFO:tasks.workunit.client.0.smithi027.stdout:7/596: mknod d4/d1b/d4a/d62/d87/cbf 0 2024-04-20T10:59:21.424 INFO:tasks.workunit.client.0.smithi027.stdout:6/618: creat da/d21/dab/fc7 x:0 0 0 2024-04-20T10:59:21.425 INFO:tasks.workunit.client.0.smithi027.stdout:7/597: creat d4/d5/d8/d21/d94/da7/fc0 x:0 0 0 2024-04-20T10:59:21.426 INFO:tasks.workunit.client.0.smithi027.stdout:6/619: symlink da/d21/dab/d7e/lc8 0 2024-04-20T10:59:21.427 INFO:tasks.workunit.client.0.smithi027.stdout:6/620: chown da/d21/dab/c23 15471 1 2024-04-20T10:59:21.427 INFO:tasks.workunit.client.0.smithi027.stdout:7/598: mknod d4/d1b/d4a/d62/cc1 0 2024-04-20T10:59:21.427 INFO:tasks.workunit.client.0.smithi027.stdout:7/599: write d4/d5b/f9e [2447948,96280] 0 2024-04-20T10:59:21.428 INFO:tasks.workunit.client.0.smithi027.stdout:6/621: symlink da/d21/d28/d2d/d53/d9a/lc9 0 2024-04-20T10:59:21.428 INFO:tasks.workunit.client.0.smithi027.stdout:6/622: chown da/d21/dab 535380495 1 2024-04-20T10:59:21.428 INFO:tasks.workunit.client.0.smithi027.stdout:6/623: chown da/d21/d28/d2d/fa6 1989 1 2024-04-20T10:59:21.428 INFO:tasks.workunit.client.0.smithi027.stdout:0/726: getdents d0/d11/d2c/d2e/d55 0 2024-04-20T10:59:21.429 INFO:tasks.workunit.client.0.smithi027.stdout:0/727: chown d0/d11/d2c/d2e/d2b/d7b/d12/dd7 1201 1 2024-04-20T10:59:21.429 INFO:tasks.workunit.client.0.smithi027.stdout:0/728: fdatasync d0/d11/d2c/d2e/d2b/d7b/d12/fc1 0 2024-04-20T10:59:21.429 INFO:tasks.workunit.client.0.smithi027.stdout:0/729: fsync d0/d11/d2c/d2e/d2b/d7b/f27 0 2024-04-20T10:59:21.429 INFO:tasks.workunit.client.0.smithi027.stdout:0/730: fdatasync d0/d11/d2c/d2e/d2b/d7b/d12/ff1 0 2024-04-20T10:59:21.430 INFO:tasks.workunit.client.0.smithi027.stdout:7/600: mkdir d4/d5/d8/d21/d94/da7/dc2 0 2024-04-20T10:59:21.430 INFO:tasks.workunit.client.0.smithi027.stdout:7/601: fsync d4/d5/d8/d45/f71 0 2024-04-20T10:59:21.431 INFO:tasks.workunit.client.0.smithi027.stdout:6/624: mknod da/dd/cca 0 2024-04-20T10:59:21.431 INFO:tasks.workunit.client.0.smithi027.stdout:6/625: fsync da/d21/dab/d7e/d86/da3/fac 0 2024-04-20T10:59:21.432 INFO:tasks.workunit.client.0.smithi027.stdout:4/592: dwrite d0/d14/d6b/da3/f1e [0,4194304] 0 2024-04-20T10:59:21.432 INFO:tasks.workunit.client.0.smithi027.stdout:0/731: mknod d0/d11/d2c/d2e/d2b/d7b/d12/d1a/cfd 0 2024-04-20T10:59:21.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/626: symlink da/d21/d28/d2d/d53/d9a/lcb 0 2024-04-20T10:59:21.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/627: getdents da/dd/d79 0 2024-04-20T10:59:21.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/628: fsync da/dd/d79/fb7 0 2024-04-20T10:59:21.434 INFO:tasks.workunit.client.0.smithi027.stdout:7/602: dread d4/d1b/d4a/d62/fa5 [0,4194304] 0 2024-04-20T10:59:21.437 INFO:tasks.workunit.client.0.smithi027.stdout:7/603: rename d4/d1b/d4a/d7a/l8b to d4/d5/d81/lc3 0 2024-04-20T10:59:21.437 INFO:tasks.workunit.client.0.smithi027.stdout:7/604: read d4/d1b/f72 [3586847,103103] 0 2024-04-20T10:59:21.438 INFO:tasks.workunit.client.0.smithi027.stdout:2/545: dwrite d5/f4b [0,4194304] 0 2024-04-20T10:59:21.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/535: dwrite d4/d1b/d31/f75 [0,4194304] 0 2024-04-20T10:59:21.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/536: fsync d4/d74/d72/f73 0 2024-04-20T10:59:21.439 INFO:tasks.workunit.client.0.smithi027.stdout:4/593: link d0/d14/d6b/lce d0/d14/d6b/da3/d1a/dae/ld4 0 2024-04-20T10:59:21.441 INFO:tasks.workunit.client.0.smithi027.stdout:7/605: creat d4/d5/d8/d21/d94/da7/dc2/fc4 x:0 0 0 2024-04-20T10:59:21.445 INFO:tasks.workunit.client.0.smithi027.stdout:9/537: symlink d4/d1b/la4 0 2024-04-20T10:59:21.447 INFO:tasks.workunit.client.0.smithi027.stdout:4/594: link d0/d14/d6b/da3/d1a/f8a d0/d14/d6b/da3/d53/d6a/fd5 0 2024-04-20T10:59:21.451 INFO:tasks.workunit.client.0.smithi027.stdout:0/732: dwrite d0/d11/d2c/d2e/d35/d8c/fa7 [0,4194304] 0 2024-04-20T10:59:21.451 INFO:tasks.workunit.client.0.smithi027.stdout:7/606: dread d4/d1b/d4a/d62/d87/f9b [0,4194304] 0 2024-04-20T10:59:21.453 INFO:tasks.workunit.client.0.smithi027.stdout:4/595: creat d0/d14/d6b/da3/d18/db9/fd6 x:0 0 0 2024-04-20T10:59:21.454 INFO:tasks.workunit.client.0.smithi027.stdout:0/733: getdents d0/d11 0 2024-04-20T10:59:21.455 INFO:tasks.workunit.client.0.smithi027.stdout:7/607: creat d4/d4d/fc5 x:0 0 0 2024-04-20T10:59:21.455 INFO:tasks.workunit.client.0.smithi027.stdout:7/608: readlink d4/d1b/l1c 0 2024-04-20T10:59:21.456 INFO:tasks.workunit.client.0.smithi027.stdout:4/596: rename d0/d5e/ca7 to d0/d14/d6b/d92/dcd/cd7 0 2024-04-20T10:59:21.456 INFO:tasks.workunit.client.0.smithi027.stdout:4/597: write d0/d14/d6b/da3/d1b/d33/f9a [4990011,73651] 0 2024-04-20T10:59:21.456 INFO:tasks.workunit.client.0.smithi027.stdout:0/734: getdents d0/d11/d2c/d2e/d2b/d7b/d12/d25 0 2024-04-20T10:59:21.456 INFO:tasks.workunit.client.0.smithi027.stdout:0/735: read - d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/fbe zero size 2024-04-20T10:59:21.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/598: mkdir d0/d5e/d6c/dd8 0 2024-04-20T10:59:21.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/599: chown d0/d14/d6b/da3/l60 8237661 1 2024-04-20T10:59:21.459 INFO:tasks.workunit.client.0.smithi027.stdout:0/736: link d0/d11/d2c/d2e/d35/d8c/lbc d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/lfe 0 2024-04-20T10:59:21.460 INFO:tasks.workunit.client.0.smithi027.stdout:4/600: symlink d0/d14/d6b/da3/d18/dbf/ld9 0 2024-04-20T10:59:21.461 INFO:tasks.workunit.client.0.smithi027.stdout:9/538: dread d4/d1b/d1f/f7c [0,4194304] 0 2024-04-20T10:59:21.461 INFO:tasks.workunit.client.0.smithi027.stdout:0/737: unlink d0/d11/d2c/d2e/d2b/d7b/ce0 0 2024-04-20T10:59:21.461 INFO:tasks.workunit.client.0.smithi027.stdout:1/771: dwrite d3/d7/d38/f112 [0,4194304] 0 2024-04-20T10:59:21.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/629: dwrite da/d21/dab/d19/d2e/d32/fb3 [0,4194304] 0 2024-04-20T10:59:21.462 INFO:tasks.workunit.client.0.smithi027.stdout:4/601: mkdir d0/d14/d6b/da3/d1a/d3b/dda 0 2024-04-20T10:59:21.462 INFO:tasks.workunit.client.0.smithi027.stdout:4/602: write d0/d14/d6b/da3/f31 [1217905,40948] 0 2024-04-20T10:59:21.463 INFO:tasks.workunit.client.0.smithi027.stdout:9/539: creat d4/d1b/fa5 x:0 0 0 2024-04-20T10:59:21.463 INFO:tasks.workunit.client.0.smithi027.stdout:9/540: fsync d4/d74/d72/f98 0 2024-04-20T10:59:21.467 INFO:tasks.workunit.client.0.smithi027.stdout:4/603: symlink d0/d14/d6b/da3/d18/dbf/ldb 0 2024-04-20T10:59:21.469 INFO:tasks.workunit.client.0.smithi027.stdout:9/541: rename d4/d2a/l87 to d4/d1b/d8f/la6 0 2024-04-20T10:59:21.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/564: write d19/d1b/d1c/d1e/d22/db1/f55 [1743451,87964] 0 2024-04-20T10:59:21.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/630: mkdir da/dcc 0 2024-04-20T10:59:21.473 INFO:tasks.workunit.client.0.smithi027.stdout:6/631: stat da/d21/dab/c26 0 2024-04-20T10:59:21.473 INFO:tasks.workunit.client.0.smithi027.stdout:6/632: write da/dd/d79/fa5 [515419,123827] 0 2024-04-20T10:59:21.473 INFO:tasks.workunit.client.0.smithi027.stdout:4/604: symlink d0/d14/d6b/d92/ldc 0 2024-04-20T10:59:21.475 INFO:tasks.workunit.client.0.smithi027.stdout:5/565: link d19/d1b/d1c/d1e/d25/la6 d19/d1b/d1c/d1e/d22/db1/lb5 0 2024-04-20T10:59:21.476 INFO:tasks.workunit.client.0.smithi027.stdout:6/633: mkdir da/d21/dab/d7e/d86/dcd 0 2024-04-20T10:59:21.477 INFO:tasks.workunit.client.0.smithi027.stdout:4/605: mknod d0/d14/d6b/d92/dcd/cdd 0 2024-04-20T10:59:21.479 INFO:tasks.workunit.client.0.smithi027.stdout:1/772: dwrite d3/d7/d38/da4/dac/ff9 [0,4194304] 0 2024-04-20T10:59:21.480 INFO:tasks.workunit.client.0.smithi027.stdout:6/634: rename da/d21/dab/d7e/d86/da3/fba to da/dcc/fce 0 2024-04-20T10:59:21.486 INFO:tasks.workunit.client.0.smithi027.stdout:1/773: rmdir d3 39 2024-04-20T10:59:21.486 INFO:tasks.workunit.client.0.smithi027.stdout:1/774: stat d3/d20/faf 0 2024-04-20T10:59:21.486 INFO:tasks.workunit.client.0.smithi027.stdout:1/775: readlink d3/d20/lb7 0 2024-04-20T10:59:21.487 INFO:tasks.workunit.client.0.smithi027.stdout:0/738: write d0/d11/d2c/d2e/d55/fac [1820896,50644] 0 2024-04-20T10:59:21.490 INFO:tasks.workunit.client.0.smithi027.stdout:1/776: link d3/d7/d38/db8/lec d3/d7/d38/db8/l114 0 2024-04-20T10:59:21.490 INFO:tasks.workunit.client.0.smithi027.stdout:1/777: write d3/d20/d85/f86 [396700,111628] 0 2024-04-20T10:59:21.498 INFO:tasks.workunit.client.0.smithi027.stdout:9/542: dwrite d4/d1b/d1f/f9b [0,4194304] 0 2024-04-20T10:59:21.499 INFO:tasks.workunit.client.0.smithi027.stdout:9/543: write d4/d1b/f89 [859276,40756] 0 2024-04-20T10:59:21.500 INFO:tasks.workunit.client.0.smithi027.stdout:0/739: rmdir d0/d11/d2c/da1/da6 0 2024-04-20T10:59:21.500 INFO:tasks.workunit.client.0.smithi027.stdout:0/740: getdents d0/d11/d4b/d5b/d6f/d82/dd3 0 2024-04-20T10:59:21.506 INFO:tasks.workunit.client.0.smithi027.stdout:6/635: read da/d21/dab/d7e/f81 [756777,16468] 0 2024-04-20T10:59:21.506 INFO:tasks.workunit.client.0.smithi027.stdout:6/636: fdatasync da/dd/d3d/f90 0 2024-04-20T10:59:21.506 INFO:tasks.workunit.client.0.smithi027.stdout:1/778: dread d3/d7e/f9f [0,4194304] 0 2024-04-20T10:59:21.507 INFO:tasks.workunit.client.0.smithi027.stdout:5/566: dwrite d19/d1b/d1c/d1e/f57 [4194304,4194304] 0 2024-04-20T10:59:21.508 INFO:tasks.workunit.client.0.smithi027.stdout:4/606: dwrite d0/d14/d6b/da3/d18/f2d [4194304,4194304] 0 2024-04-20T10:59:21.508 INFO:tasks.workunit.client.0.smithi027.stdout:1/779: dread d3/d7/d38/d62/f6e [0,4194304] 0 2024-04-20T10:59:21.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/567: getdents d19/d1b/d1c/d1e/d58 0 2024-04-20T10:59:21.512 INFO:tasks.workunit.client.0.smithi027.stdout:0/741: write d0/d11/d2c/d2e/d2b/d3d/fb3 [1773787,117609] 0 2024-04-20T10:59:21.512 INFO:tasks.workunit.client.0.smithi027.stdout:0/742: getdents d0/d11/d2c/d2e/d2b/d7b/dcf 0 2024-04-20T10:59:21.514 INFO:tasks.workunit.client.0.smithi027.stdout:1/780: rename d3/d20/d85/l9e to d3/d20/d85/l115 0 2024-04-20T10:59:21.516 INFO:tasks.workunit.client.0.smithi027.stdout:0/743: symlink d0/d11/d5c/df0/lff 0 2024-04-20T10:59:21.516 INFO:tasks.workunit.client.0.smithi027.stdout:0/744: chown d0/d11/d2c/d2e/d2b/d7b/d64 384443 1 2024-04-20T10:59:21.516 INFO:tasks.workunit.client.0.smithi027.stdout:6/637: dread da/d21/dab/d19/f74 [0,4194304] 0 2024-04-20T10:59:21.518 INFO:tasks.workunit.client.0.smithi027.stdout:1/781: creat d3/d45/f116 x:0 0 0 2024-04-20T10:59:21.520 INFO:tasks.workunit.client.0.smithi027.stdout:0/745: dread d0/d11/d2c/d2e/d2b/d7b/d12/fad [0,4194304] 0 2024-04-20T10:59:21.528 INFO:tasks.workunit.client.0.smithi027.stdout:0/746: creat d0/d11/d4b/d5b/d6f/d82/dd1/deb/f100 x:0 0 0 2024-04-20T10:59:21.545 INFO:tasks.workunit.client.0.smithi027.stdout:9/544: dwrite d4/d1b/d8f/f90 [4194304,4194304] 0 2024-04-20T10:59:21.545 INFO:tasks.workunit.client.0.smithi027.stdout:9/545: write d4/d1b/d1f/d45/d8a/f6e [1559167,75614] 0 2024-04-20T10:59:21.545 INFO:tasks.workunit.client.0.smithi027.stdout:4/607: dwrite d0/d14/d6b/da3/d1a/d3b/fc8 [0,4194304] 0 2024-04-20T10:59:21.545 INFO:tasks.workunit.client.0.smithi027.stdout:0/747: mkdir d0/d11/d4b/d5b/d6f/d82/dd3/d101 0 2024-04-20T10:59:21.546 INFO:tasks.workunit.client.0.smithi027.stdout:9/546: link d4/d14/f44 d4/d1b/d8f/fa7 0 2024-04-20T10:59:21.548 INFO:tasks.workunit.client.0.smithi027.stdout:6/638: dwrite da/d21/dab/fa2 [0,4194304] 0 2024-04-20T10:59:21.548 INFO:tasks.workunit.client.0.smithi027.stdout:7/609: dwrite d4/d1b/d4a/d62/f9f [0,4194304] 0 2024-04-20T10:59:21.548 INFO:tasks.workunit.client.0.smithi027.stdout:7/610: truncate d4/d5/d8/d21/f48 525770 0 2024-04-20T10:59:21.548 INFO:tasks.workunit.client.0.smithi027.stdout:4/608: symlink d0/d14/d6b/da3/d18/dbf/lde 0 2024-04-20T10:59:21.549 INFO:tasks.workunit.client.0.smithi027.stdout:9/547: rmdir d4/d1b/d1f/d45 39 2024-04-20T10:59:21.550 INFO:tasks.workunit.client.0.smithi027.stdout:1/782: dwrite d3/d20/f46 [0,4194304] 0 2024-04-20T10:59:21.554 INFO:tasks.workunit.client.0.smithi027.stdout:6/639: dread da/d21/dab/d7e/f81 [0,4194304] 0 2024-04-20T10:59:21.558 INFO:tasks.workunit.client.0.smithi027.stdout:7/611: creat d4/d5/d6f/fc6 x:0 0 0 2024-04-20T10:59:21.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/548: mkdir d4/d14/da8 0 2024-04-20T10:59:21.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/549: chown d4/d14/d43/c95 985375922 1 2024-04-20T10:59:21.559 INFO:tasks.workunit.client.0.smithi027.stdout:6/640: dread da/d21/dab/d7e/f81 [0,4194304] 0 2024-04-20T10:59:21.561 INFO:tasks.workunit.client.0.smithi027.stdout:7/612: dread d4/d1b/f88 [0,4194304] 0 2024-04-20T10:59:21.561 INFO:tasks.workunit.client.0.smithi027.stdout:9/550: mknod d4/d14/ca9 0 2024-04-20T10:59:21.563 INFO:tasks.workunit.client.0.smithi027.stdout:6/641: link da/d21/dab/d1f/dbf/fc4 da/d21/dab/d19/fcf 0 2024-04-20T10:59:21.563 INFO:tasks.workunit.client.0.smithi027.stdout:6/642: chown da/f2a 63486 1 2024-04-20T10:59:21.563 INFO:tasks.workunit.client.0.smithi027.stdout:6/643: write da/f2a [2857983,6855] 0 2024-04-20T10:59:21.566 INFO:tasks.workunit.client.0.smithi027.stdout:0/748: dwrite d0/d11/f21 [0,4194304] 0 2024-04-20T10:59:21.568 INFO:tasks.workunit.client.0.smithi027.stdout:9/551: rename d4/d2a/c2f to d4/d1b/caa 0 2024-04-20T10:59:21.569 INFO:tasks.workunit.client.0.smithi027.stdout:4/609: dwrite d0/f37 [0,4194304] 0 2024-04-20T10:59:21.572 INFO:tasks.workunit.client.0.smithi027.stdout:9/552: rename d4/d1b/d8f/fa7 to d4/d1b/d31/fab 0 2024-04-20T10:59:21.586 INFO:tasks.workunit.client.0.smithi027.stdout:1/783: dwrite d3/d7/d28/f48 [0,4194304] 0 2024-04-20T10:59:21.586 INFO:tasks.workunit.client.0.smithi027.stdout:1/784: write d3/d7/d38/fa7 [182977,85494] 0 2024-04-20T10:59:21.587 INFO:tasks.workunit.client.0.smithi027.stdout:4/610: getdents d0/d14/d6b 0 2024-04-20T10:59:21.587 INFO:tasks.workunit.client.0.smithi027.stdout:7/613: dwrite d4/d5/d8/d21/f96 [0,4194304] 0 2024-04-20T10:59:21.587 INFO:tasks.workunit.client.0.smithi027.stdout:1/785: mknod d3/d7/d38/db8/c117 0 2024-04-20T10:59:21.590 INFO:tasks.workunit.client.0.smithi027.stdout:7/614: mknod d4/cc7 0 2024-04-20T10:59:21.590 INFO:tasks.workunit.client.0.smithi027.stdout:7/615: write d4/d1b/d4a/d62/f56 [137933,84815] 0 2024-04-20T10:59:21.591 INFO:tasks.workunit.client.0.smithi027.stdout:8/660: sync 2024-04-20T10:59:21.591 INFO:tasks.workunit.client.0.smithi027.stdout:3/620: sync 2024-04-20T10:59:21.591 INFO:tasks.workunit.client.0.smithi027.stdout:2/546: sync 2024-04-20T10:59:21.593 INFO:tasks.workunit.client.0.smithi027.stdout:4/611: mknod d0/d14/d6b/da3/d53/d6a/cdf 0 2024-04-20T10:59:21.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/749: dwrite d0/d59/dbf/ff5 [0,4194304] 0 2024-04-20T10:59:21.593 INFO:tasks.workunit.client.0.smithi027.stdout:8/661: unlink d9/de/d64/c76 0 2024-04-20T10:59:21.593 INFO:tasks.workunit.client.0.smithi027.stdout:8/662: stat d9/de/d1e/d4a/f45 0 2024-04-20T10:59:21.593 INFO:tasks.workunit.client.0.smithi027.stdout:8/663: write d9/de/d1e/d4a/d6d/db7/d5b/f5d [569475,42562] 0 2024-04-20T10:59:21.594 INFO:tasks.workunit.client.0.smithi027.stdout:2/547: mkdir d5/dc/d16/db1 0 2024-04-20T10:59:21.595 INFO:tasks.workunit.client.0.smithi027.stdout:3/621: creat d2/fc5 x:0 0 0 2024-04-20T10:59:21.595 INFO:tasks.workunit.client.0.smithi027.stdout:3/622: chown d2/dd/df 30668 1 2024-04-20T10:59:21.595 INFO:tasks.workunit.client.0.smithi027.stdout:3/623: truncate d2/f66 298247 0 2024-04-20T10:59:21.595 INFO:tasks.workunit.client.0.smithi027.stdout:3/624: stat d2/d18/d2e/l37 0 2024-04-20T10:59:21.595 INFO:tasks.workunit.client.0.smithi027.stdout:3/625: fsync d2/d18/d2c/d4c/f6e 0 2024-04-20T10:59:21.595 INFO:tasks.workunit.client.0.smithi027.stdout:4/612: creat d0/d14/d6b/da3/fe0 x:0 0 0 2024-04-20T10:59:21.599 INFO:tasks.workunit.client.0.smithi027.stdout:8/664: creat d9/de/d20/d48/d62/fe5 x:0 0 0 2024-04-20T10:59:21.599 INFO:tasks.workunit.client.0.smithi027.stdout:0/750: creat d0/d11/d5c/df0/f102 x:0 0 0 2024-04-20T10:59:21.599 INFO:tasks.workunit.client.0.smithi027.stdout:0/751: chown d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/fe2 523008 1 2024-04-20T10:59:21.600 INFO:tasks.workunit.client.0.smithi027.stdout:3/626: getdents d2/dd/df/d23/d68/d88 0 2024-04-20T10:59:21.602 INFO:tasks.workunit.client.0.smithi027.stdout:6/644: dwrite da/dd/d79/f87 [0,4194304] 0 2024-04-20T10:59:21.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/752: creat d0/d11/d4b/f103 x:0 0 0 2024-04-20T10:59:21.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/753: truncate d0/d11/d2c/d2e/f2a 1430898 0 2024-04-20T10:59:21.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/754: write d0/d11/d4b/d5b/d6f/d82/dd1/deb/ff9 [531708,107453] 0 2024-04-20T10:59:21.604 INFO:tasks.workunit.client.0.smithi027.stdout:2/548: dread d5/d26/d47/d8a/fa3 [0,4194304] 0 2024-04-20T10:59:21.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/613: dread d0/d14/d6b/da3/d1b/d2f/f67 [0,4194304] 0 2024-04-20T10:59:21.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/614: fsync d0/d14/d6b/da3/d1b/d33/f9a 0 2024-04-20T10:59:21.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/615: readlink d0/d14/d6b/da3/d53/d6a/da1/lab 0 2024-04-20T10:59:21.610 INFO:tasks.workunit.client.0.smithi027.stdout:7/616: dwrite d4/da2/fb5 [0,4194304] 0 2024-04-20T10:59:21.610 INFO:tasks.workunit.client.0.smithi027.stdout:7/617: write d4/d5/d8/f82 [376547,3163] 0 2024-04-20T10:59:21.610 INFO:tasks.workunit.client.0.smithi027.stdout:6/645: symlink da/d21/dab/d1f/ld0 0 2024-04-20T10:59:21.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/616: dread d0/d14/d6b/da3/d1a/fa4 [0,4194304] 0 2024-04-20T10:59:21.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/617: chown d0/d14/d6b/da3/d18/fa8 0 1 2024-04-20T10:59:21.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/618: readlink d0/d14/d6b/da3/d1a/d3b/l3c 0 2024-04-20T10:59:21.612 INFO:tasks.workunit.client.0.smithi027.stdout:7/618: symlink d4/d5/d8/d21/d94/da7/dc2/lc8 0 2024-04-20T10:59:21.613 INFO:tasks.workunit.client.0.smithi027.stdout:7/619: chown d4/d5/d8/d21/d94/fa8 184033807 1 2024-04-20T10:59:21.613 INFO:tasks.workunit.client.0.smithi027.stdout:6/646: rmdir da/dd/d3d 39 2024-04-20T10:59:21.613 INFO:tasks.workunit.client.0.smithi027.stdout:3/627: dread d2/d18/f76 [0,4194304] 0 2024-04-20T10:59:21.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/619: mkdir d0/d14/d6b/da3/d53/d6a/da1/de1 0 2024-04-20T10:59:21.615 INFO:tasks.workunit.client.0.smithi027.stdout:3/628: mkdir d2/dd/d57/d8d/dc6 0 2024-04-20T10:59:21.615 INFO:tasks.workunit.client.0.smithi027.stdout:3/629: write d2/dd/df/d23/d68/fbb [793175,39390] 0 2024-04-20T10:59:21.616 INFO:tasks.workunit.client.0.smithi027.stdout:4/620: write d0/d14/d6b/da3/d18/f4b [1405315,75264] 0 2024-04-20T10:59:21.617 INFO:tasks.workunit.client.0.smithi027.stdout:6/647: mknod da/d21/dab/d19/d2e/cd1 0 2024-04-20T10:59:21.618 INFO:tasks.workunit.client.0.smithi027.stdout:4/621: mkdir d0/d48/d68/d76/de2 0 2024-04-20T10:59:21.619 INFO:tasks.workunit.client.0.smithi027.stdout:6/648: rename da/d21/f9b to da/d21/dab/d19/d2e/d6b/fd2 0 2024-04-20T10:59:21.620 INFO:tasks.workunit.client.0.smithi027.stdout:4/622: creat d0/d14/d6b/da3/d1b/d33/d5c/d6f/d7b/fe3 x:0 0 0 2024-04-20T10:59:21.621 INFO:tasks.workunit.client.0.smithi027.stdout:1/786: dwrite d3/f17 [4194304,4194304] 0 2024-04-20T10:59:21.623 INFO:tasks.workunit.client.0.smithi027.stdout:8/665: dwrite d9/de/da3/f30 [4194304,4194304] 0 2024-04-20T10:59:21.623 INFO:tasks.workunit.client.0.smithi027.stdout:6/649: creat da/d21/d28/d2d/d53/fd3 x:0 0 0 2024-04-20T10:59:21.624 INFO:tasks.workunit.client.0.smithi027.stdout:6/650: read da/d21/dab/d7e/f85 [64795,109707] 0 2024-04-20T10:59:21.625 INFO:tasks.workunit.client.0.smithi027.stdout:2/549: dwrite d5/f4b [0,4194304] 0 2024-04-20T10:59:21.629 INFO:tasks.workunit.client.0.smithi027.stdout:4/623: truncate d0/d14/d6b/da3/d1b/d33/d5c/fd2 225408 0 2024-04-20T10:59:21.630 INFO:tasks.workunit.client.0.smithi027.stdout:6/651: getdents da/d21/dab/d7e/d86 0 2024-04-20T10:59:21.632 INFO:tasks.workunit.client.0.smithi027.stdout:2/550: dread d5/dc/f55 [0,4194304] 0 2024-04-20T10:59:21.634 INFO:tasks.workunit.client.0.smithi027.stdout:4/624: unlink d0/c5 0 2024-04-20T10:59:21.636 INFO:tasks.workunit.client.0.smithi027.stdout:3/630: dwrite d2/d18/f29 [0,4194304] 0 2024-04-20T10:59:21.636 INFO:tasks.workunit.client.0.smithi027.stdout:2/551: link d5/d26/l2d d5/dc/d16/d60/dac/lb2 0 2024-04-20T10:59:21.643 INFO:tasks.workunit.client.0.smithi027.stdout:7/620: dwrite d4/d5/d8/d21/d94/da7/dc2/fc4 [0,4194304] 0 2024-04-20T10:59:21.643 INFO:tasks.workunit.client.0.smithi027.stdout:7/621: stat d4/d5/d8/d21/d94/da7 0 2024-04-20T10:59:21.649 INFO:tasks.workunit.client.0.smithi027.stdout:6/652: write da/d21/dab/d19/f93 [2088154,37996] 0 2024-04-20T10:59:21.651 INFO:tasks.workunit.client.0.smithi027.stdout:3/631: dread d2/d18/d2c/f30 [0,4194304] 0 2024-04-20T10:59:21.651 INFO:tasks.workunit.client.0.smithi027.stdout:6/653: creat da/d21/d28/d2d/fd4 x:0 0 0 2024-04-20T10:59:21.651 INFO:tasks.workunit.client.0.smithi027.stdout:6/654: stat da/d21/dab/d19/d2e/d6b/f62 0 2024-04-20T10:59:21.652 INFO:tasks.workunit.client.0.smithi027.stdout:6/655: write da/d21/dab/f4f [4339006,54528] 0 2024-04-20T10:59:21.653 INFO:tasks.workunit.client.0.smithi027.stdout:6/656: creat da/d21/dab/d7e/d86/fd5 x:0 0 0 2024-04-20T10:59:21.654 INFO:tasks.workunit.client.0.smithi027.stdout:6/657: rmdir da/d21/d28/d2d 39 2024-04-20T10:59:21.655 INFO:tasks.workunit.client.0.smithi027.stdout:6/658: symlink da/dd/d3c/ld6 0 2024-04-20T10:59:21.656 INFO:tasks.workunit.client.0.smithi027.stdout:6/659: creat da/d21/dab/d1f/dbf/fd7 x:0 0 0 2024-04-20T10:59:21.656 INFO:tasks.workunit.client.0.smithi027.stdout:6/660: symlink da/d21/dab/d19/d2e/d6b/ld8 0 2024-04-20T10:59:21.658 INFO:tasks.workunit.client.0.smithi027.stdout:6/661: creat da/d21/fd9 x:0 0 0 2024-04-20T10:59:21.662 INFO:tasks.workunit.client.0.smithi027.stdout:8/666: dwrite d9/de/faf [0,4194304] 0 2024-04-20T10:59:21.663 INFO:tasks.workunit.client.0.smithi027.stdout:8/667: chown d9/de/f2f 266004954 1 2024-04-20T10:59:21.665 INFO:tasks.workunit.client.0.smithi027.stdout:4/625: dwrite d0/d14/d6b/da3/d18/db9/fc9 [0,4194304] 0 2024-04-20T10:59:21.665 INFO:tasks.workunit.client.0.smithi027.stdout:8/668: unlink d9/de/d20/fb1 0 2024-04-20T10:59:21.667 INFO:tasks.workunit.client.0.smithi027.stdout:8/669: mkdir d9/de/d20/d48/d59/de6 0 2024-04-20T10:59:21.667 INFO:tasks.workunit.client.0.smithi027.stdout:1/787: dwrite d3/d7/d28/f100 [0,4194304] 0 2024-04-20T10:59:21.669 INFO:tasks.workunit.client.0.smithi027.stdout:7/622: dwrite d4/f2e [0,4194304] 0 2024-04-20T10:59:21.669 INFO:tasks.workunit.client.0.smithi027.stdout:7/623: chown d4/d5/d8/d45/c90 662 1 2024-04-20T10:59:21.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/552: dwrite d5/dc/d16/d60/dac/f94 [4194304,4194304] 0 2024-04-20T10:59:21.670 INFO:tasks.workunit.client.0.smithi027.stdout:1/788: mkdir d3/d7/d28/d92/dad/d118 0 2024-04-20T10:59:21.673 INFO:tasks.workunit.client.0.smithi027.stdout:2/553: truncate d5/d26/f2c 69292 0 2024-04-20T10:59:21.675 INFO:tasks.workunit.client.0.smithi027.stdout:1/789: creat d3/d7/d28/d92/dad/d118/f119 x:0 0 0 2024-04-20T10:59:21.677 INFO:tasks.workunit.client.0.smithi027.stdout:3/632: dwrite d2/d18/f5b [0,4194304] 0 2024-04-20T10:59:21.678 INFO:tasks.workunit.client.0.smithi027.stdout:3/633: chown d2/dd/fc0 85 1 2024-04-20T10:59:21.678 INFO:tasks.workunit.client.0.smithi027.stdout:2/554: creat d5/d26/fb3 x:0 0 0 2024-04-20T10:59:21.678 INFO:tasks.workunit.client.0.smithi027.stdout:6/662: dwrite da/f99 [0,4194304] 0 2024-04-20T10:59:21.678 INFO:tasks.workunit.client.0.smithi027.stdout:3/634: symlink d2/dd/d57/d8d/lc7 0 2024-04-20T10:59:21.679 INFO:tasks.workunit.client.0.smithi027.stdout:2/555: mkdir d5/d26/d30/db4 0 2024-04-20T10:59:21.679 INFO:tasks.workunit.client.0.smithi027.stdout:2/556: truncate d5/d44/fa5 965649 0 2024-04-20T10:59:21.682 INFO:tasks.workunit.client.0.smithi027.stdout:1/790: dread d3/d7/f57 [0,4194304] 0 2024-04-20T10:59:21.684 INFO:tasks.workunit.client.0.smithi027.stdout:3/635: creat d2/dd/d57/d8d/dc6/fc8 x:0 0 0 2024-04-20T10:59:21.684 INFO:tasks.workunit.client.0.smithi027.stdout:3/636: dread - d2/d18/d2c/d4c/d67/d6c/f87 zero size 2024-04-20T10:59:21.684 INFO:tasks.workunit.client.0.smithi027.stdout:3/637: fdatasync d2/f21 0 2024-04-20T10:59:21.684 INFO:tasks.workunit.client.0.smithi027.stdout:3/638: truncate d2/dd/df/faa 31004 0 2024-04-20T10:59:21.686 INFO:tasks.workunit.client.0.smithi027.stdout:3/639: link d2/dd/df/d23/c91 d2/cc9 0 2024-04-20T10:59:21.686 INFO:tasks.workunit.client.0.smithi027.stdout:6/663: dread da/d21/dab/f1a [0,4194304] 0 2024-04-20T10:59:21.686 INFO:tasks.workunit.client.0.smithi027.stdout:6/664: dread - da/d21/d28/d2d/fd4 zero size 2024-04-20T10:59:21.687 INFO:tasks.workunit.client.0.smithi027.stdout:8/670: dwrite d9/de/d1e/d4a/d6d/db7/d5b/f5d [0,4194304] 0 2024-04-20T10:59:21.687 INFO:tasks.workunit.client.0.smithi027.stdout:8/671: chown d9/de/d20/d95/c98 15627960 1 2024-04-20T10:59:21.687 INFO:tasks.workunit.client.0.smithi027.stdout:8/672: chown d9/f15 853432 1 2024-04-20T10:59:21.688 INFO:tasks.workunit.client.0.smithi027.stdout:3/640: unlink d2/d80/c64 0 2024-04-20T10:59:21.688 INFO:tasks.workunit.client.0.smithi027.stdout:3/641: write d2/dd/fb5 [489121,48025] 0 2024-04-20T10:59:21.688 INFO:tasks.workunit.client.0.smithi027.stdout:3/642: stat d2/d80/c49 0 2024-04-20T10:59:21.688 INFO:tasks.workunit.client.0.smithi027.stdout:1/791: dread d3/d7e/fdc [0,4194304] 0 2024-04-20T10:59:21.688 INFO:tasks.workunit.client.0.smithi027.stdout:1/792: dread - d3/d7/ffd zero size 2024-04-20T10:59:21.688 INFO:tasks.workunit.client.0.smithi027.stdout:6/665: getdents da/d21/dab/d1f/dbf 0 2024-04-20T10:59:21.692 INFO:tasks.workunit.client.0.smithi027.stdout:8/673: symlink d9/de/da3/dab/le7 0 2024-04-20T10:59:21.692 INFO:tasks.workunit.client.0.smithi027.stdout:8/674: fdatasync d9/f15 0 2024-04-20T10:59:21.693 INFO:tasks.workunit.client.0.smithi027.stdout:5/568: sync 2024-04-20T10:59:21.694 INFO:tasks.workunit.client.0.smithi027.stdout:1/793: creat d3/d7/d38/da4/dac/f11a x:0 0 0 2024-04-20T10:59:21.695 INFO:tasks.workunit.client.0.smithi027.stdout:5/569: creat d19/d1b/d1c/d9c/fb6 x:0 0 0 2024-04-20T10:59:21.695 INFO:tasks.workunit.client.0.smithi027.stdout:5/570: stat d19/d1b/d1c/d1e/d22/l3a 0 2024-04-20T10:59:21.697 INFO:tasks.workunit.client.0.smithi027.stdout:3/643: write d2/d80/f5a [2833355,8908] 0 2024-04-20T10:59:21.699 INFO:tasks.workunit.client.0.smithi027.stdout:5/571: dread f18 [0,4194304] 0 2024-04-20T10:59:21.699 INFO:tasks.workunit.client.0.smithi027.stdout:5/572: dread - d19/d1b/d1c/d1e/d22/d68/f90 zero size 2024-04-20T10:59:21.699 INFO:tasks.workunit.client.0.smithi027.stdout:5/573: dread - d19/d1b/d1c/fad zero size 2024-04-20T10:59:21.699 INFO:tasks.workunit.client.0.smithi027.stdout:5/574: readlink d19/d1b/d1c/d1e/d22/l6f 0 2024-04-20T10:59:21.699 INFO:tasks.workunit.client.0.smithi027.stdout:5/575: write d19/d1b/d1c/d1e/d22/f5a [265749,113973] 0 2024-04-20T10:59:21.700 INFO:tasks.workunit.client.0.smithi027.stdout:1/794: rename d3/d7/d28/db0/lce to d3/d7/d38/da4/dbc/l11b 0 2024-04-20T10:59:21.700 INFO:tasks.workunit.client.0.smithi027.stdout:1/795: write d3/d7/f1f [565213,122803] 0 2024-04-20T10:59:21.700 INFO:tasks.workunit.client.0.smithi027.stdout:1/796: fsync d3/d7/d28/f59 0 2024-04-20T10:59:21.700 INFO:tasks.workunit.client.0.smithi027.stdout:9/553: sync 2024-04-20T10:59:21.700 INFO:tasks.workunit.client.0.smithi027.stdout:0/755: sync 2024-04-20T10:59:21.700 INFO:tasks.workunit.client.0.smithi027.stdout:0/756: readlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/l5f 0 2024-04-20T10:59:21.701 INFO:tasks.workunit.client.0.smithi027.stdout:1/797: dread d3/d7/d38/d62/f6e [0,4194304] 0 2024-04-20T10:59:21.701 INFO:tasks.workunit.client.0.smithi027.stdout:3/644: creat d2/dd/df/d33/fca x:0 0 0 2024-04-20T10:59:21.701 INFO:tasks.workunit.client.0.smithi027.stdout:3/645: fsync d2/dd/f39 0 2024-04-20T10:59:21.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/666: read da/d21/f5b [3262780,47714] 0 2024-04-20T10:59:21.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/667: write da/d21/d28/f3a [2137613,31402] 0 2024-04-20T10:59:21.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/668: stat da/d21/d28/fbb 0 2024-04-20T10:59:21.702 INFO:tasks.workunit.client.0.smithi027.stdout:2/557: dwrite d5/dc/d16/d2a/f40 [0,4194304] 0 2024-04-20T10:59:21.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/576: mkdir d19/d1b/d1c/d1e/d22/d68/d75/db7 0 2024-04-20T10:59:21.703 INFO:tasks.workunit.client.0.smithi027.stdout:9/554: mknod d4/d1b/cac 0 2024-04-20T10:59:21.703 INFO:tasks.workunit.client.0.smithi027.stdout:9/555: readlink d4/d1b/l9c 0 2024-04-20T10:59:21.703 INFO:tasks.workunit.client.0.smithi027.stdout:0/757: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/f104 x:0 0 0 2024-04-20T10:59:21.703 INFO:tasks.workunit.client.0.smithi027.stdout:0/758: truncate d0/d59/dbf/ffa 850552 0 2024-04-20T10:59:21.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/798: mkdir d3/d7/d28/db0/d11c 0 2024-04-20T10:59:21.705 INFO:tasks.workunit.client.0.smithi027.stdout:3/646: creat d2/fcb x:0 0 0 2024-04-20T10:59:21.706 INFO:tasks.workunit.client.0.smithi027.stdout:2/558: creat d5/dc/d16/d60/d8e/fb5 x:0 0 0 2024-04-20T10:59:21.707 INFO:tasks.workunit.client.0.smithi027.stdout:5/577: symlink d19/d1b/d1c/d1e/d58/db4/lb8 0 2024-04-20T10:59:21.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/556: creat d4/d1b/d31/d23/d29/d7f/fad x:0 0 0 2024-04-20T10:59:21.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/557: readlink d4/d14/l37 0 2024-04-20T10:59:21.709 INFO:tasks.workunit.client.0.smithi027.stdout:0/759: link d0/d11/d2c/d2e/d61/f78 d0/d11/d2c/f105 0 2024-04-20T10:59:21.711 INFO:tasks.workunit.client.0.smithi027.stdout:1/799: rename d3/d7e/f10e to d3/d7/d38/da4/dbc/f11d 0 2024-04-20T10:59:21.711 INFO:tasks.workunit.client.0.smithi027.stdout:1/800: readlink d3/d7/l71 0 2024-04-20T10:59:21.711 INFO:tasks.workunit.client.0.smithi027.stdout:2/559: symlink d5/d26/d78/lb6 0 2024-04-20T10:59:21.712 INFO:tasks.workunit.client.0.smithi027.stdout:6/669: dread da/dd/f59 [0,4194304] 0 2024-04-20T10:59:21.712 INFO:tasks.workunit.client.0.smithi027.stdout:5/578: mkdir d19/d1b/d1c/d1e/da3/db9 0 2024-04-20T10:59:21.712 INFO:tasks.workunit.client.0.smithi027.stdout:0/760: rmdir d0/d11/d4b/d5b/d6f/d82 39 2024-04-20T10:59:21.712 INFO:tasks.workunit.client.0.smithi027.stdout:0/761: write d0/d11/d2c/d2e/d2b/d7b/d64/fb4 [219083,55475] 0 2024-04-20T10:59:21.713 INFO:tasks.workunit.client.0.smithi027.stdout:1/801: unlink d3/d45/d5d/d6c/f72 0 2024-04-20T10:59:21.713 INFO:tasks.workunit.client.0.smithi027.stdout:2/560: truncate d5/dc/d16/d2a/f75 1488378 0 2024-04-20T10:59:21.713 INFO:tasks.workunit.client.0.smithi027.stdout:2/561: stat d5/dc/d16/d32 0 2024-04-20T10:59:21.714 INFO:tasks.workunit.client.0.smithi027.stdout:6/670: truncate da/d21/dab/d1f/f3f 556478 0 2024-04-20T10:59:21.715 INFO:tasks.workunit.client.0.smithi027.stdout:5/579: creat d19/d1b/d1c/d1e/d3c/fba x:0 0 0 2024-04-20T10:59:21.715 INFO:tasks.workunit.client.0.smithi027.stdout:5/580: write d19/d1b/d1c/d1e/d22/db1/fb3 [115863,95079] 0 2024-04-20T10:59:21.716 INFO:tasks.workunit.client.0.smithi027.stdout:9/558: write d4/d1b/d31/d23/f47 [3017239,57289] 0 2024-04-20T10:59:21.716 INFO:tasks.workunit.client.0.smithi027.stdout:1/802: creat d3/d7/d28/d35/f11e x:0 0 0 2024-04-20T10:59:21.718 INFO:tasks.workunit.client.0.smithi027.stdout:2/562: truncate d5/dc/d16/d60/dac/f67 116413 0 2024-04-20T10:59:21.720 INFO:tasks.workunit.client.0.smithi027.stdout:6/671: getdents da/d21/dab/d19/d2e/d6b 0 2024-04-20T10:59:21.721 INFO:tasks.workunit.client.0.smithi027.stdout:5/581: getdents d19/d1b/d1c/d1e 0 2024-04-20T10:59:21.722 INFO:tasks.workunit.client.0.smithi027.stdout:5/582: dread d19/d1b/d1c/d1e/d58/f6c [0,4194304] 0 2024-04-20T10:59:21.722 INFO:tasks.workunit.client.0.smithi027.stdout:5/583: fdatasync d19/d1b/d1c/d1e/d25/fa0 0 2024-04-20T10:59:21.727 INFO:tasks.workunit.client.0.smithi027.stdout:1/803: write d3/d7/d28/d35/f5e [2972091,14643] 0 2024-04-20T10:59:21.730 INFO:tasks.workunit.client.0.smithi027.stdout:2/563: dread d5/d1e/d21/f33 [0,4194304] 0 2024-04-20T10:59:21.732 INFO:tasks.workunit.client.0.smithi027.stdout:5/584: dread d19/d1b/f8d [4194304,4194304] 0 2024-04-20T10:59:21.732 INFO:tasks.workunit.client.0.smithi027.stdout:9/559: dwrite d4/d14/f27 [0,4194304] 0 2024-04-20T10:59:21.733 INFO:tasks.workunit.client.0.smithi027.stdout:6/672: mknod da/cda 0 2024-04-20T10:59:21.735 INFO:tasks.workunit.client.0.smithi027.stdout:5/585: mknod d19/d1b/d1c/d1e/d22/cbb 0 2024-04-20T10:59:21.735 INFO:tasks.workunit.client.0.smithi027.stdout:5/586: readlink d19/d1b/d1c/d1e/d25/l37 0 2024-04-20T10:59:21.737 INFO:tasks.workunit.client.0.smithi027.stdout:2/564: dread d5/dc/d16/d2a/f9f [0,4194304] 0 2024-04-20T10:59:21.737 INFO:tasks.workunit.client.0.smithi027.stdout:2/565: fdatasync d5/d26/f3a 0 2024-04-20T10:59:21.740 INFO:tasks.workunit.client.0.smithi027.stdout:5/587: creat d19/d1b/d1c/d1e/d25/d40/d82/fbc x:0 0 0 2024-04-20T10:59:21.742 INFO:tasks.workunit.client.0.smithi027.stdout:2/566: truncate d5/dc/d16/d37/f6b 1804869 0 2024-04-20T10:59:21.746 INFO:tasks.workunit.client.0.smithi027.stdout:1/804: dwrite d3/d7/d28/ff1 [0,4194304] 0 2024-04-20T10:59:21.746 INFO:tasks.workunit.client.0.smithi027.stdout:1/805: readlink d3/d7/d28/l31 0 2024-04-20T10:59:21.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/588: mkdir d19/d1b/d1c/dbd 0 2024-04-20T10:59:21.749 INFO:tasks.workunit.client.0.smithi027.stdout:1/806: creat d3/d20/dfb/f11f x:0 0 0 2024-04-20T10:59:21.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/589: rename d19/d1b/d1c/d9c to d19/d1b/d1c/dbd/dbe 0 2024-04-20T10:59:21.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/590: write d19/d1b/d1c/d1e/d25/d40/d82/fae [921021,60466] 0 2024-04-20T10:59:21.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/591: readlink d19/d1b/d1c/d1e/d22/d68/l80 0 2024-04-20T10:59:21.760 INFO:tasks.workunit.client.0.smithi027.stdout:6/673: dwrite da/dd/f69 [0,4194304] 0 2024-04-20T10:59:21.760 INFO:tasks.workunit.client.0.smithi027.stdout:6/674: fdatasync da/d21/d28/d2d/fa6 0 2024-04-20T10:59:21.760 INFO:tasks.workunit.client.0.smithi027.stdout:6/675: chown da/d21/dab/d1f/f3f 3 1 2024-04-20T10:59:21.760 INFO:tasks.workunit.client.0.smithi027.stdout:9/560: dwrite d4/d74/d72/f98 [0,4194304] 0 2024-04-20T10:59:21.760 INFO:tasks.workunit.client.0.smithi027.stdout:9/561: fsync d4/f7 0 2024-04-20T10:59:21.767 INFO:tasks.workunit.client.0.smithi027.stdout:1/807: dwrite d3/d7/d38/d62/fe0 [0,4194304] 0 2024-04-20T10:59:21.768 INFO:tasks.workunit.client.0.smithi027.stdout:6/676: dread da/d21/dab/d1f/f3f [0,4194304] 0 2024-04-20T10:59:21.768 INFO:tasks.workunit.client.0.smithi027.stdout:6/677: write da/d21/f8f [938005,109284] 0 2024-04-20T10:59:21.768 INFO:tasks.workunit.client.0.smithi027.stdout:6/678: write da/dd/d79/fa5 [1247962,120409] 0 2024-04-20T10:59:21.768 INFO:tasks.workunit.client.0.smithi027.stdout:9/562: mkdir d4/d74/dae 0 2024-04-20T10:59:21.769 INFO:tasks.workunit.client.0.smithi027.stdout:9/563: dread d4/d14/d43/f86 [0,4194304] 0 2024-04-20T10:59:21.769 INFO:tasks.workunit.client.0.smithi027.stdout:9/564: truncate d4/d1b/d31/f9a 49851 0 2024-04-20T10:59:21.769 INFO:tasks.workunit.client.0.smithi027.stdout:9/565: chown d4/d1b/d31/d51/c5b 3413 1 2024-04-20T10:59:21.770 INFO:tasks.workunit.client.0.smithi027.stdout:7/624: sync 2024-04-20T10:59:21.770 INFO:tasks.workunit.client.0.smithi027.stdout:4/626: sync 2024-04-20T10:59:21.770 INFO:tasks.workunit.client.0.smithi027.stdout:1/808: link d3/d7/d28/f48 d3/d7/d38/ddf/d73/f120 0 2024-04-20T10:59:21.772 INFO:tasks.workunit.client.0.smithi027.stdout:9/566: mkdir d4/d1b/daf 0 2024-04-20T10:59:21.772 INFO:tasks.workunit.client.0.smithi027.stdout:9/567: read - d4/d1b/fa5 zero size 2024-04-20T10:59:21.774 INFO:tasks.workunit.client.0.smithi027.stdout:7/625: unlink d4/d5/d8/d46/c6d 0 2024-04-20T10:59:21.776 INFO:tasks.workunit.client.0.smithi027.stdout:1/809: creat d3/d7/d38/ddf/dcc/f121 x:0 0 0 2024-04-20T10:59:21.776 INFO:tasks.workunit.client.0.smithi027.stdout:1/810: stat d3/d7/d38/d62/f6e 0 2024-04-20T10:59:21.776 INFO:tasks.workunit.client.0.smithi027.stdout:1/811: chown d3/d7/c2b 0 1 2024-04-20T10:59:21.777 INFO:tasks.workunit.client.0.smithi027.stdout:5/592: dwrite d19/d1b/d1c/d1e/d3c/f24 [0,4194304] 0 2024-04-20T10:59:21.778 INFO:tasks.workunit.client.0.smithi027.stdout:6/679: write da/d21/dab/d19/d2e/fb4 [3235992,115478] 0 2024-04-20T10:59:21.779 INFO:tasks.workunit.client.0.smithi027.stdout:7/626: truncate d4/d4d/f60 360975 0 2024-04-20T10:59:21.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/627: dread d4/d1b/f88 [0,4194304] 0 2024-04-20T10:59:21.782 INFO:tasks.workunit.client.0.smithi027.stdout:6/680: truncate da/dd/f25 495577 0 2024-04-20T10:59:21.784 INFO:tasks.workunit.client.0.smithi027.stdout:1/812: write d3/d7e/f83 [2163583,39893] 0 2024-04-20T10:59:21.784 INFO:tasks.workunit.client.0.smithi027.stdout:5/593: write d19/d1b/d1c/d1e/f88 [597988,24571] 0 2024-04-20T10:59:21.784 INFO:tasks.workunit.client.0.smithi027.stdout:5/594: readlink d19/d1b/d1c/d1e/d25/d40/d82/l8e 0 2024-04-20T10:59:21.786 INFO:tasks.workunit.client.0.smithi027.stdout:7/628: rmdir d4/d5/d8 39 2024-04-20T10:59:21.790 INFO:tasks.workunit.client.0.smithi027.stdout:1/813: creat d3/d7/d38/d62/f122 x:0 0 0 2024-04-20T10:59:21.790 INFO:tasks.workunit.client.0.smithi027.stdout:1/814: write d3/d7/d38/da4/fda [274462,36303] 0 2024-04-20T10:59:21.790 INFO:tasks.workunit.client.0.smithi027.stdout:5/595: unlink d19/d1b/d1c/d1e/d22/f30 0 2024-04-20T10:59:21.798 INFO:tasks.workunit.client.0.smithi027.stdout:9/568: dwrite d4/d1b/d31/f75 [4194304,4194304] 0 2024-04-20T10:59:21.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/627: dwrite d0/d14/d6b/da3/d53/fb4 [0,4194304] 0 2024-04-20T10:59:21.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/628: chown d0/d14/d6b/da3/d1b/c69 15 1 2024-04-20T10:59:21.798 INFO:tasks.workunit.client.0.smithi027.stdout:1/815: rmdir d3/d20/d85 39 2024-04-20T10:59:21.799 INFO:tasks.workunit.client.0.smithi027.stdout:5/596: dread f0 [0,4194304] 0 2024-04-20T10:59:21.800 INFO:tasks.workunit.client.0.smithi027.stdout:4/629: rmdir d0/d14/d6b/da3/d1b/d33/d5c/d6f/d7b 39 2024-04-20T10:59:21.800 INFO:tasks.workunit.client.0.smithi027.stdout:1/816: mkdir d3/d7e/d123 0 2024-04-20T10:59:21.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/597: creat d19/d1b/d1c/dbd/dbe/fbf x:0 0 0 2024-04-20T10:59:21.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/598: dread - d19/d1b/d1c/d1e/d22/d68/f90 zero size 2024-04-20T10:59:21.806 INFO:tasks.workunit.client.0.smithi027.stdout:7/629: dwrite d4/d5/d8/f82 [0,4194304] 0 2024-04-20T10:59:21.806 INFO:tasks.workunit.client.0.smithi027.stdout:7/630: read d4/d5/f1f [941314,32967] 0 2024-04-20T10:59:21.807 INFO:tasks.workunit.client.0.smithi027.stdout:4/630: creat d0/d14/d71/fe4 x:0 0 0 2024-04-20T10:59:21.808 INFO:tasks.workunit.client.0.smithi027.stdout:7/631: mknod d4/d5/d8/d3b/cc9 0 2024-04-20T10:59:21.813 INFO:tasks.workunit.client.0.smithi027.stdout:6/681: dwrite da/dd/f69 [0,4194304] 0 2024-04-20T10:59:21.814 INFO:tasks.workunit.client.0.smithi027.stdout:6/682: chown da/l20 1 1 2024-04-20T10:59:21.814 INFO:tasks.workunit.client.0.smithi027.stdout:4/631: creat d0/d14/d6b/da3/fe5 x:0 0 0 2024-04-20T10:59:21.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/683: creat da/d21/d28/d54/fdb x:0 0 0 2024-04-20T10:59:21.818 INFO:tasks.workunit.client.0.smithi027.stdout:8/675: sync 2024-04-20T10:59:21.819 INFO:tasks.workunit.client.0.smithi027.stdout:4/632: mkdir d0/d14/d6b/da3/d1b/d2f/de6 0 2024-04-20T10:59:21.820 INFO:tasks.workunit.client.0.smithi027.stdout:8/676: mkdir d9/de/d20/d48/d59/d74/de8 0 2024-04-20T10:59:21.820 INFO:tasks.workunit.client.0.smithi027.stdout:8/677: read - d9/de/da3/dab/fc9 zero size 2024-04-20T10:59:21.821 INFO:tasks.workunit.client.0.smithi027.stdout:5/599: dwrite d19/d1b/d1c/d1e/d58/f94 [0,4194304] 0 2024-04-20T10:59:21.821 INFO:tasks.workunit.client.0.smithi027.stdout:4/633: creat d0/d14/d6b/da3/d1b/d33/fe7 x:0 0 0 2024-04-20T10:59:21.822 INFO:tasks.workunit.client.0.smithi027.stdout:8/678: symlink d9/de/d64/le9 0 2024-04-20T10:59:21.822 INFO:tasks.workunit.client.0.smithi027.stdout:6/684: dread da/d21/dab/d19/d2e/fb4 [0,4194304] 0 2024-04-20T10:59:21.822 INFO:tasks.workunit.client.0.smithi027.stdout:5/600: creat d19/d1b/d1c/d1e/d25/fc0 x:0 0 0 2024-04-20T10:59:21.824 INFO:tasks.workunit.client.0.smithi027.stdout:6/685: symlink da/d21/d28/d54/ldc 0 2024-04-20T10:59:21.824 INFO:tasks.workunit.client.0.smithi027.stdout:6/686: truncate da/d21/dab/f29 1553833 0 2024-04-20T10:59:21.824 INFO:tasks.workunit.client.0.smithi027.stdout:5/601: symlink d19/d1b/d1c/d1e/d83/lc1 0 2024-04-20T10:59:21.825 INFO:tasks.workunit.client.0.smithi027.stdout:7/632: dwrite d4/d5b/fae [0,4194304] 0 2024-04-20T10:59:21.825 INFO:tasks.workunit.client.0.smithi027.stdout:7/633: fdatasync d4/d4d/f4f 0 2024-04-20T10:59:21.842 INFO:tasks.workunit.client.0.smithi027.stdout:7/634: dwrite d4/d5/d8/d46/f8f [0,4194304] 0 2024-04-20T10:59:21.842 INFO:tasks.workunit.client.0.smithi027.stdout:7/635: chown d4/d5/d8/d3b/cc9 96 1 2024-04-20T10:59:21.842 INFO:tasks.workunit.client.0.smithi027.stdout:7/636: dread - d4/fb4 zero size 2024-04-20T10:59:21.989 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:22.081 INFO:tasks.workunit.client.0.smithi027.stdout:6/687: creat da/dd/d3d/db9/fdd x:0 0 0 2024-04-20T10:59:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:6/688: chown da/dd/d3c 6335 1 2024-04-20T10:59:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:6/689: fsync da/d21/dab/f11 0 2024-04-20T10:59:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/679: rename d9/de/d20/d48 to d9/de/d1e/d4a/d6d/dea 0 2024-04-20T10:59:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/680: chown d9/de/l10 0 1 2024-04-20T10:59:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/681: chown d9/de/d20/d53/c6b 16 1 2024-04-20T10:59:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/682: stat d9/de/d1e/d4a/d6d/d75/d7c/fbc 0 2024-04-20T10:59:22.082 INFO:tasks.workunit.client.0.smithi027.stdout:5/602: truncate d19/d1b/f8d 2750968 0 2024-04-20T10:59:22.083 INFO:tasks.workunit.client.0.smithi027.stdout:7/637: creat d4/d5/d6f/fca x:0 0 0 2024-04-20T10:59:22.085 INFO:tasks.workunit.client.0.smithi027.stdout:6/690: creat da/d21/d28/d54/fde x:0 0 0 2024-04-20T10:59:22.085 INFO:tasks.workunit.client.0.smithi027.stdout:8/683: creat d9/de/d1e/d4a/d6d/d79/feb x:0 0 0 2024-04-20T10:59:22.086 INFO:tasks.workunit.client.0.smithi027.stdout:7/638: creat d4/d1b/d4a/d7a/fcb x:0 0 0 2024-04-20T10:59:22.088 INFO:tasks.workunit.client.0.smithi027.stdout:6/691: mkdir da/d21/d28/d2d/d53/dae/ddf 0 2024-04-20T10:59:22.090 INFO:tasks.workunit.client.0.smithi027.stdout:8/684: unlink d9/de/f37 0 2024-04-20T10:59:22.090 INFO:tasks.workunit.client.0.smithi027.stdout:7/639: dread d4/d5/f40 [0,4194304] 0 2024-04-20T10:59:22.090 INFO:tasks.workunit.client.0.smithi027.stdout:7/640: readlink d4/d5/d8/d3b/l51 0 2024-04-20T10:59:22.091 INFO:tasks.workunit.client.0.smithi027.stdout:5/603: write f18 [2602572,9453] 0 2024-04-20T10:59:22.092 INFO:tasks.workunit.client.0.smithi027.stdout:6/692: rmdir da/d21/d28/d2d/dc3 0 2024-04-20T10:59:22.092 INFO:tasks.workunit.client.0.smithi027.stdout:6/693: truncate da/d21/d28/d2d/d53/fd3 639117 0 2024-04-20T10:59:22.092 INFO:tasks.workunit.client.0.smithi027.stdout:6/694: truncate da/d21/d28/d54/fde 393043 0 2024-04-20T10:59:22.092 INFO:tasks.workunit.client.0.smithi027.stdout:7/641: mknod d4/d5/d8/d46/ccc 0 2024-04-20T10:59:22.092 INFO:tasks.workunit.client.0.smithi027.stdout:7/642: truncate d4/d5/d8/d46/d77/d7c/f8c 268565 0 2024-04-20T10:59:22.092 INFO:tasks.workunit.client.0.smithi027.stdout:7/643: chown d4/d1b/d4a/d7a/c95 10095103 1 2024-04-20T10:59:22.093 INFO:tasks.workunit.client.0.smithi027.stdout:6/695: unlink da/d21/d28/d2d/d53/fd3 0 2024-04-20T10:59:22.094 INFO:tasks.workunit.client.0.smithi027.stdout:7/644: mkdir d4/d1b/d4a/d62/dcd 0 2024-04-20T10:59:22.095 INFO:tasks.workunit.client.0.smithi027.stdout:6/696: unlink da/dd/c73 0 2024-04-20T10:59:22.096 INFO:tasks.workunit.client.0.smithi027.stdout:7/645: unlink d4/d4d/l73 0 2024-04-20T10:59:22.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.096+0000 7f8d34b04640 1 Processor -- start 2024-04-20T10:59:22.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.096+0000 7f8d34b04640 1 -- start start 2024-04-20T10:59:22.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.098+0000 7f8d34b04640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d300c7440 0x7f8d3015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.098+0000 7f8d34b04640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 0x7f8d300ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.098+0000 7f8d34b04640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d3006d910 con 0x7f8d3006e9a0 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.098+0000 7f8d34b04640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8d300ca650 con 0x7f8d300c7440 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.098+0000 7f8d34b04640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8d300c7820 con 0x7f8d300c9cc0 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.098+0000 7f8d2dd9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8d3006e9a0 0x7f8d30140680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56708/0 (socket says 172.21.15.27:56708) 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.099+0000 7f8d2e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8d300c7440 0x7f8d3015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.099+0000 7f8d2dd9b640 1 -- 172.21.15.27:0/3645656779 learned_addr learned my addr 172.21.15.27:0/3645656779 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.099+0000 7f8d2ed9d640 1 --2- 172.21.15.27:0/3645656779 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 0x7f8d300ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.099+0000 7f8d2e59c640 1 -- 172.21.15.27:0/3645656779 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 msgr2=0x7f8d300ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.099+0000 7f8d2e59c640 1 --2- 172.21.15.27:0/3645656779 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 0x7f8d300ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.099+0000 7f8d2e59c640 1 -- 172.21.15.27:0/3645656779 >> v1:172.21.15.27:6789/0 conn(0x7f8d3006e9a0 legacy=0x7f8d30140680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.099+0000 7f8d2d59a640 1 -- 172.21.15.27:0/3645656779 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407641162 0 0) 0x7f8d3006d910 con 0x7f8d3006e9a0 2024-04-20T10:59:22.101 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.099+0000 7f8d2e59c640 1 -- 172.21.15.27:0/3645656779 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d30071a40 con 0x7f8d300c7440 2024-04-20T10:59:22.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.100+0000 7f8d2e59c640 1 --2- 172.21.15.27:0/3645656779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 0x7f8d3015bfd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d200097a0 tx=0x7f8d2002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6369cd37e09a66c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:22.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.101+0000 7f8d2d59a640 1 -- 172.21.15.27:0/3645656779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20034070 con 0x7f8d300c7440 2024-04-20T10:59:22.102 INFO:tasks.workunit.client.0.smithi027.stdout:5/604: dread f3 [0,4194304] 0 2024-04-20T10:59:22.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.101+0000 7f8d2d59a640 1 -- 172.21.15.27:0/3645656779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d20030610 con 0x7f8d300c7440 2024-04-20T10:59:22.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.101+0000 7f8d2d59a640 1 -- 172.21.15.27:0/3645656779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d200309e0 con 0x7f8d300c7440 2024-04-20T10:59:22.103 INFO:tasks.workunit.client.0.smithi027.stdout:1/817: sync 2024-04-20T10:59:22.103 INFO:tasks.workunit.client.0.smithi027.stdout:4/634: sync 2024-04-20T10:59:22.103 INFO:tasks.workunit.client.0.smithi027.stdout:3/647: sync 2024-04-20T10:59:22.104 INFO:tasks.workunit.client.0.smithi027.stdout:4/635: chown d0/d14/d6b/da3/d53/d6a 261981611 1 2024-04-20T10:59:22.104 INFO:tasks.workunit.client.0.smithi027.stdout:0/762: sync 2024-04-20T10:59:22.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.102+0000 7f8d34b04640 1 -- 172.21.15.27:0/3645656779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 msgr2=0x7f8d3015bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.102+0000 7f8d34b04640 1 --2- 172.21.15.27:0/3645656779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 0x7f8d3015bfd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d200097a0 tx=0x7f8d2002b960 comp rx=0 tx=0).stop 2024-04-20T10:59:22.104 INFO:tasks.workunit.client.0.smithi027.stdout:9/569: sync 2024-04-20T10:59:22.104 INFO:tasks.workunit.client.0.smithi027.stdout:0/763: chown d0/d11/d2c/d2e/l45 150025749 1 2024-04-20T10:59:22.105 INFO:tasks.workunit.client.0.smithi027.stdout:2/567: sync 2024-04-20T10:59:22.105 INFO:tasks.workunit.client.0.smithi027.stdout:3/648: rmdir d2/dd/df/d23/d68/d88 39 2024-04-20T10:59:22.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.103+0000 7f8d34b04640 1 -- 172.21.15.27:0/3645656779 shutdown_connections 2024-04-20T10:59:22.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.103+0000 7f8d34b04640 1 --2- 172.21.15.27:0/3645656779 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 0x7f8d300ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.103+0000 7f8d34b04640 1 --2- 172.21.15.27:0/3645656779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 0x7f8d3015bfd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.103+0000 7f8d34b04640 1 -- 172.21.15.27:0/3645656779 >> 172.21.15.27:0/3645656779 conn(0x7f8d3006b930 msgr2=0x7f8d3006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:22.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.105+0000 7f8d34b04640 1 -- 172.21.15.27:0/3645656779 shutdown_connections 2024-04-20T10:59:22.107 INFO:tasks.workunit.client.0.smithi027.stdout:1/818: mknod d3/d7/d28/db0/c124 0 2024-04-20T10:59:22.108 INFO:tasks.workunit.client.0.smithi027.stdout:1/819: fsync d3/d7/f23 0 2024-04-20T10:59:22.108 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.105+0000 7f8d34b04640 1 -- 172.21.15.27:0/3645656779 wait complete. 2024-04-20T10:59:22.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.108+0000 7f8d34b04640 1 Processor -- start 2024-04-20T10:59:22.109 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.108+0000 7f8d34b04640 1 -- start start 2024-04-20T10:59:22.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.109+0000 7f8d34b04640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d3006e9a0 0x7f8d301e6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.109+0000 7f8d34b04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 0x7f8d3013be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.109+0000 7f8d34b04640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 0x7f8d3013c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.109+0000 7f8d34b04640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3006de90 con 0x7f8d300c7440 2024-04-20T10:59:22.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.109+0000 7f8d34b04640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30067a10 con 0x7f8d3006e9a0 2024-04-20T10:59:22.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.109+0000 7f8d34b04640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3006e170 con 0x7f8d300c9cc0 2024-04-20T10:59:22.112 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 0x7f8d3013be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 0x7f8d3013be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33416/0 (socket says 172.21.15.27:33416) 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2dd9b640 1 -- 172.21.15.27:0/3370922668 learned_addr learned my addr 172.21.15.27:0/3370922668 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2ed9d640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 0x7f8d3013c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2e59c640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d3006e9a0 0x7f8d301e6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2dd9b640 1 -- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 msgr2=0x7f8d3013c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2dd9b640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 0x7f8d3013c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2dd9b640 1 -- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d3006e9a0 msgr2=0x7f8d301e6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2dd9b640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d3006e9a0 0x7f8d301e6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2dd9b640 1 -- 172.21.15.27:0/3370922668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d30156bb0 con 0x7f8d300c7440 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.110+0000 7f8d2dd9b640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 0x7f8d3013be40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c500 tx=0x7f8d2400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.111+0000 7f8d1effd640 1 -- 172.21.15.27:0/3370922668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24018070 con 0x7f8d300c7440 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.111+0000 7f8d1effd640 1 -- 172.21.15.27:0/3370922668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d24007560 con 0x7f8d300c7440 2024-04-20T10:59:22.113 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.111+0000 7f8d1effd640 1 -- 172.21.15.27:0/3370922668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d240049e0 con 0x7f8d300c7440 2024-04-20T10:59:22.114 INFO:tasks.workunit.client.0.smithi027.stdout:7/646: dread d4/d1b/f63 [0,4194304] 0 2024-04-20T10:59:22.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.111+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3013b360 con 0x7f8d300c7440 2024-04-20T10:59:22.114 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.111+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d300c4d40 con 0x7f8d300c7440 2024-04-20T10:59:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:4/636: unlink d0/d14/d6b/da3/f1e 0 2024-04-20T10:59:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:3/649: rmdir d2/d80 39 2024-04-20T10:59:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:1/820: dread d3/d7/d28/f8b [0,4194304] 0 2024-04-20T10:59:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:8/685: dwrite d9/de/d1e/d4a/d6d/d79/fe4 [0,4194304] 0 2024-04-20T10:59:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:8/686: chown d9/de/da3/dab/fdb 545649129 1 2024-04-20T10:59:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:8/687: readlink d9/de/da3/l58 0 2024-04-20T10:59:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:8/688: dread - d9/de/d1e/dc1/fcc zero size 2024-04-20T10:59:22.115 INFO:tasks.workunit.client.0.smithi027.stdout:4/637: mkdir d0/d5e/d6c/de8 0 2024-04-20T10:59:22.116 INFO:tasks.workunit.client.0.smithi027.stdout:5/605: dread d19/d1b/d1c/d1e/d58/f5e [0,4194304] 0 2024-04-20T10:59:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.114+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d3004fcf0 con 0x7f8d300c7440 2024-04-20T10:59:22.116 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.115+0000 7f8d1effd640 1 -- 172.21.15.27:0/3370922668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d2401b070 con 0x7f8d300c7440 2024-04-20T10:59:22.116 INFO:tasks.workunit.client.0.smithi027.stdout:1/821: symlink d3/d7/d28/d35/l125 0 2024-04-20T10:59:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.115+0000 7f8d1effd640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d1006c940 0x7f8d1006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.116+0000 7f8d1effd640 1 -- 172.21.15.27:0/3370922668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f8d2408afc0 con 0x7f8d300c7440 2024-04-20T10:59:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.116+0000 7f8d2e59c640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d1006c940 0x7f8d1006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.117 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.116+0000 7f8d2e59c640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d1006c940 0x7f8d1006ee00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d20036000 tx=0x7f8d2003b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:22.117 INFO:tasks.workunit.client.0.smithi027.stdout:8/689: symlink d9/de/da3/dcd/lec 0 2024-04-20T10:59:22.118 INFO:tasks.workunit.client.0.smithi027.stdout:4/638: symlink d0/d5e/d6c/dd8/le9 0 2024-04-20T10:59:22.119 INFO:tasks.workunit.client.0.smithi027.stdout:5/606: mknod d19/d1b/d1c/d1e/d58/d9d/cc2 0 2024-04-20T10:59:22.119 INFO:tasks.workunit.client.0.smithi027.stdout:5/607: fdatasync d19/d1b/d1c/f45 0 2024-04-20T10:59:22.119 INFO:tasks.workunit.client.0.smithi027.stdout:2/568: dread d5/dc/d16/d32/f5a [0,4194304] 0 2024-04-20T10:59:22.120 INFO:tasks.workunit.client.0.smithi027.stdout:1/822: rename d3/d7/d28/fdd to d3/d20/dfb/f126 0 2024-04-20T10:59:22.121 INFO:tasks.workunit.client.0.smithi027.stdout:8/690: creat d9/de/d1e/d4a/d6d/d79/fed x:0 0 0 2024-04-20T10:59:22.121 INFO:tasks.workunit.client.0.smithi027.stdout:8/691: dread - d9/de/d1e/d4a/d6d/d79/feb zero size 2024-04-20T10:59:22.122 INFO:tasks.workunit.client.0.smithi027.stdout:8/692: dread - d9/de/d1e/d4a/d6d/d79/f92 zero size 2024-04-20T10:59:22.122 INFO:tasks.workunit.client.0.smithi027.stdout:8/693: fdatasync d9/de/d20/f73 0 2024-04-20T10:59:22.122 INFO:tasks.workunit.client.0.smithi027.stdout:5/608: mknod d19/d1b/d1c/d1e/d25/d40/d78/cc3 0 2024-04-20T10:59:22.123 INFO:tasks.workunit.client.0.smithi027.stdout:1/823: getdents d3/d7/d38/d62 0 2024-04-20T10:59:22.123 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.122+0000 7f8d1effd640 1 -- 172.21.15.27:0/3370922668 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8d24058cf0 con 0x7f8d300c7440 2024-04-20T10:59:22.124 INFO:tasks.workunit.client.0.smithi027.stdout:9/570: dwrite d4/d1b/d1f/d45/f71 [0,4194304] 0 2024-04-20T10:59:22.124 INFO:tasks.workunit.client.0.smithi027.stdout:6/697: dwrite da/d21/dab/d7e/d86/fb0 [0,4194304] 0 2024-04-20T10:59:22.124 INFO:tasks.workunit.client.0.smithi027.stdout:6/698: stat da/d21/dab/d19/f5d 0 2024-04-20T10:59:22.124 INFO:tasks.workunit.client.0.smithi027.stdout:9/571: fdatasync d4/d14/d43/d91/f96 0 2024-04-20T10:59:22.124 INFO:tasks.workunit.client.0.smithi027.stdout:6/699: write da/d21/fb2 [810872,121448] 0 2024-04-20T10:59:22.124 INFO:tasks.workunit.client.0.smithi027.stdout:0/764: dwrite d0/f1f [0,4194304] 0 2024-04-20T10:59:22.124 INFO:tasks.workunit.client.0.smithi027.stdout:5/609: rmdir d19/d1b/d1c/d1e/d58 39 2024-04-20T10:59:22.128 INFO:tasks.workunit.client.0.smithi027.stdout:1/824: creat d3/d7/d38/ddf/d73/f127 x:0 0 0 2024-04-20T10:59:22.132 INFO:tasks.workunit.client.0.smithi027.stdout:5/610: dread d19/d1b/d1c/f45 [0,4194304] 0 2024-04-20T10:59:22.132 INFO:tasks.workunit.client.0.smithi027.stdout:5/611: chown ce 0 1 2024-04-20T10:59:22.133 INFO:tasks.workunit.client.0.smithi027.stdout:9/572: mkdir d4/d1b/d1f/d57/db0 0 2024-04-20T10:59:22.133 INFO:tasks.workunit.client.0.smithi027.stdout:6/700: getdents da/d21/dab/d19/d2e 0 2024-04-20T10:59:22.133 INFO:tasks.workunit.client.0.smithi027.stdout:6/701: truncate da/dd/d3d/db9/fdd 483230 0 2024-04-20T10:59:22.136 INFO:tasks.workunit.client.0.smithi027.stdout:4/639: dwrite d0/d14/d6b/da3/f31 [0,4194304] 0 2024-04-20T10:59:22.138 INFO:tasks.workunit.client.0.smithi027.stdout:1/825: rmdir d3/d7/d28/d35 39 2024-04-20T10:59:22.138 INFO:tasks.workunit.client.0.smithi027.stdout:1/826: chown d3/d45/ca2 6095431 1 2024-04-20T10:59:22.138 INFO:tasks.workunit.client.0.smithi027.stdout:6/702: mkdir da/d21/dab/d19/d2e/d6b/de0 0 2024-04-20T10:59:22.138 INFO:tasks.workunit.client.0.smithi027.stdout:1/827: fsync d3/d7/d38/da4/dac/fb3 0 2024-04-20T10:59:22.138 INFO:tasks.workunit.client.0.smithi027.stdout:0/765: mkdir d0/d11/d2c/d2e/d2b/d106 0 2024-04-20T10:59:22.138 INFO:tasks.workunit.client.0.smithi027.stdout:1/828: chown d3/d7/d28/db0/ce2 134084 1 2024-04-20T10:59:22.138 INFO:tasks.workunit.client.0.smithi027.stdout:5/612: rename d19/d1b/d1c/d1e/d25/d40/d78/cc3 to d19/d1b/d1c/d1e/da3/db9/cc4 0 2024-04-20T10:59:22.139 INFO:tasks.workunit.client.0.smithi027.stdout:4/640: symlink d0/d14/d6b/da3/d1b/dc3/dc5/lea 0 2024-04-20T10:59:22.139 INFO:tasks.workunit.client.0.smithi027.stdout:6/703: mkdir da/d21/dab/d7e/d86/dcd/de1 0 2024-04-20T10:59:22.140 INFO:tasks.workunit.client.0.smithi027.stdout:0/766: mknod d0/d11/d5c/df0/c107 0 2024-04-20T10:59:22.141 INFO:tasks.workunit.client.0.smithi027.stdout:5/613: chown d19/d1b/d1c/d1e/d3c/f46 3462 1 2024-04-20T10:59:22.143 INFO:tasks.workunit.client.0.smithi027.stdout:6/704: mknod da/d21/d28/d2d/d53/d9a/ce2 0 2024-04-20T10:59:22.143 INFO:tasks.workunit.client.0.smithi027.stdout:4/641: link d0/d14/d6b/da3/d1a/d3b/fc8 d0/d14/d6b/da3/d1a/feb 0 2024-04-20T10:59:22.143 INFO:tasks.workunit.client.0.smithi027.stdout:5/614: rmdir d19/d1b/d1c/d1e/da3 39 2024-04-20T10:59:22.145 INFO:tasks.workunit.client.0.smithi027.stdout:5/615: link d19/d1b/f33 d19/d1b/d1c/d1e/d25/d40/d78/fc5 0 2024-04-20T10:59:22.146 INFO:tasks.workunit.client.0.smithi027.stdout:5/616: mknod d19/d1b/d1c/d1e/d22/cc6 0 2024-04-20T10:59:22.146 INFO:tasks.workunit.client.0.smithi027.stdout:3/650: dwrite d2/dd/d57/d8c/fb8 [0,4194304] 0 2024-04-20T10:59:22.148 INFO:tasks.workunit.client.0.smithi027.stdout:5/617: creat d19/d1b/d1c/d1e/d22/d68/fc7 x:0 0 0 2024-04-20T10:59:22.148 INFO:tasks.workunit.client.0.smithi027.stdout:5/618: fdatasync d19/d1b/d1c/f45 0 2024-04-20T10:59:22.148 INFO:tasks.workunit.client.0.smithi027.stdout:3/651: mknod d2/dd/df/d23/d68/d88/ccc 0 2024-04-20T10:59:22.149 INFO:tasks.workunit.client.0.smithi027.stdout:3/652: fsync d2/dd/f19 0 2024-04-20T10:59:22.149 INFO:tasks.workunit.client.0.smithi027.stdout:5/619: creat d19/d1b/d1c/d1e/d22/d68/fc8 x:0 0 0 2024-04-20T10:59:22.149 INFO:tasks.workunit.client.0.smithi027.stdout:5/620: stat d19/d1b/d1c/d1e/l54 0 2024-04-20T10:59:22.150 INFO:tasks.workunit.client.0.smithi027.stdout:5/621: creat d19/fc9 x:0 0 0 2024-04-20T10:59:22.152 INFO:tasks.workunit.client.0.smithi027.stdout:8/694: dwrite d9/de/d1e/fdd [0,4194304] 0 2024-04-20T10:59:22.154 INFO:tasks.workunit.client.0.smithi027.stdout:8/695: rmdir d9/de/d1e/d4a/d6d/db7/d97 39 2024-04-20T10:59:22.154 INFO:tasks.workunit.client.0.smithi027.stdout:8/696: stat d9/de/d1e/l5a 0 2024-04-20T10:59:22.157 INFO:tasks.workunit.client.0.smithi027.stdout:8/697: truncate d9/f15 522304 0 2024-04-20T10:59:22.158 INFO:tasks.workunit.client.0.smithi027.stdout:3/653: dread d2/d18/f29 [0,4194304] 0 2024-04-20T10:59:22.158 INFO:tasks.workunit.client.0.smithi027.stdout:3/654: readlink d2/dd/d57/l8b 0 2024-04-20T10:59:22.159 INFO:tasks.workunit.client.0.smithi027.stdout:8/698: creat d9/de/d1e/d4a/d6d/d75/fee x:0 0 0 2024-04-20T10:59:22.159 INFO:tasks.workunit.client.0.smithi027.stdout:5/622: read fc [1096202,16679] 0 2024-04-20T10:59:22.159 INFO:tasks.workunit.client.0.smithi027.stdout:5/623: truncate f18 5108712 0 2024-04-20T10:59:22.159 INFO:tasks.workunit.client.0.smithi027.stdout:5/624: read d19/d1b/d1c/d1e/d3c/f39 [111084,21696] 0 2024-04-20T10:59:22.160 INFO:tasks.workunit.client.0.smithi027.stdout:3/655: mknod d2/dd/df/d23/d68/ccd 0 2024-04-20T10:59:22.161 INFO:tasks.workunit.client.0.smithi027.stdout:8/699: creat d9/de/d1e/d4a/d6d/dea/d59/da0/fef x:0 0 0 2024-04-20T10:59:22.161 INFO:tasks.workunit.client.0.smithi027.stdout:8/700: chown d9/de/d1e/d49/c5f 47 1 2024-04-20T10:59:22.161 INFO:tasks.workunit.client.0.smithi027.stdout:5/625: mkdir d19/d1b/d1c/d1e/d22/dca 0 2024-04-20T10:59:22.162 INFO:tasks.workunit.client.0.smithi027.stdout:1/829: getdents d3/d7/d38/ddf/d73 0 2024-04-20T10:59:22.165 INFO:tasks.workunit.client.0.smithi027.stdout:9/573: dwrite d4/d14/d43/d91/f96 [0,4194304] 0 2024-04-20T10:59:22.173 INFO:tasks.workunit.client.0.smithi027.stdout:0/767: dwrite d0/d11/d2c/d2e/d2b/d7b/d34/f3b [4194304,4194304] 0 2024-04-20T10:59:22.176 INFO:tasks.workunit.client.0.smithi027.stdout:9/574: write d4/d1b/d31/f3c [1841947,76706] 0 2024-04-20T10:59:22.176 INFO:tasks.workunit.client.0.smithi027.stdout:4/642: dwrite d0/f3a [0,4194304] 0 2024-04-20T10:59:22.176 INFO:tasks.workunit.client.0.smithi027.stdout:4/643: read d0/d14/d6b/da3/d18/f4b [904740,28866] 0 2024-04-20T10:59:22.194 INFO:tasks.workunit.client.0.smithi027.stdout:9/575: dwrite d4/d2a/f77 [4194304,4194304] 0 2024-04-20T10:59:22.194 INFO:tasks.workunit.client.0.smithi027.stdout:9/576: getdents d4/d74/dae 0 2024-04-20T10:59:22.194 INFO:tasks.workunit.client.0.smithi027.stdout:9/577: fdatasync d4/d1b/d1f/f7c 0 2024-04-20T10:59:22.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.361+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d30143e00 con 0x7f8d300c7440 2024-04-20T10:59:22.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.361+0000 7f8d1effd640 1 -- 172.21.15.27:0/3370922668 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6933 (secure 0 0 0) 0x7f8d2405cff0 con 0x7f8d300c7440 2024-04-20T10:59:22.362 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:22.362 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5029,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:59:22.363 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-04-20T10:59:22.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d1006c940 msgr2=0x7f8d1006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d1006c940 0x7f8d1006ee00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d20036000 tx=0x7f8d2003b040 comp rx=0 tx=0).stop 2024-04-20T10:59:22.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 msgr2=0x7f8d3013be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 0x7f8d3013be40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c500 tx=0x7f8d2400c9c0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 shutdown_connections 2024-04-20T10:59:22.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8d1006c940 0x7f8d1006ee00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d300c9cc0 0x7f8d3013c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d300c7440 0x7f8d3013be40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 --2- 172.21.15.27:0/3370922668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d3006e9a0 0x7f8d301e6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 >> 172.21.15.27:0/3370922668 conn(0x7f8d3006b930 msgr2=0x7f8d300c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:22.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 shutdown_connections 2024-04-20T10:59:22.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.363+0000 7f8d34b04640 1 -- 172.21.15.27:0/3370922668 wait complete. 2024-04-20T10:59:22.375 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=3) 2024-04-20T10:59:22.375 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T10:59:22.375 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:59:22.385 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T10:59:22.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:22.419 INFO:tasks.workunit.client.0.smithi027.stdout:3/656: creat d2/d94/dc4/fce x:0 0 0 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/701: write d9/fd [3558723,84085] 0 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:1/830: creat d3/d45/d5d/d6c/f128 x:0 0 0 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/626: getdents d19/d1b/d1c/d1e/d25/d40/d82 0 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/702: write d9/de/d1e/d4a/d6d/d79/feb [754716,125021] 0 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/703: chown d9/de/d1e/d4a/d6d/db7/d5b/lae 17948 1 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/704: chown d9/de/d1e/dc1/fcc 5278 1 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/705: chown d9/de/d1e/l5a 7 1 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/627: write d19/d1b/d1c/d1e/d58/f61 [1710686,88054] 0 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/628: truncate d19/d1b/d1c/d1e/d25/d40/d82/fbc 715059 0 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:0/768: mknod d0/c108 0 2024-04-20T10:59:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:0/769: truncate d0/d11/d2c/d2e/d2b/d7b/d64/fe5 220174 0 2024-04-20T10:59:22.424 INFO:tasks.workunit.client.0.smithi027.stdout:4/644: rmdir d0/d14/d71 39 2024-04-20T10:59:22.425 INFO:tasks.workunit.client.0.smithi027.stdout:1/831: dread d3/d45/d5d/d6c/f90 [0,4194304] 0 2024-04-20T10:59:22.425 INFO:tasks.workunit.client.0.smithi027.stdout:4/645: dread - d0/d14/d6b/da3/d1b/d33/d5c/d6f/fa0 zero size 2024-04-20T10:59:22.425 INFO:tasks.workunit.client.0.smithi027.stdout:4/646: fsync d0/d14/d6b/da3/d1b/d33/d5c/d6f/fa9 0 2024-04-20T10:59:22.425 INFO:tasks.workunit.client.0.smithi027.stdout:4/647: write d0/d14/d6b/d92/fc7 [407325,74554] 0 2024-04-20T10:59:22.429 INFO:tasks.workunit.client.0.smithi027.stdout:5/629: symlink d19/d1b/d1c/d1e/d25/d40/d78/lcb 0 2024-04-20T10:59:22.431 INFO:tasks.workunit.client.0.smithi027.stdout:9/578: unlink d4/d14/d43/l60 0 2024-04-20T10:59:22.431 INFO:tasks.workunit.client.0.smithi027.stdout:9/579: fdatasync d4/d1b/d31/d23/d29/f35 0 2024-04-20T10:59:22.431 INFO:tasks.workunit.client.0.smithi027.stdout:9/580: write d4/d1b/f53 [717799,118005] 0 2024-04-20T10:59:22.431 INFO:tasks.workunit.client.0.smithi027.stdout:1/832: rmdir d3/d7/d28/db0 39 2024-04-20T10:59:22.431 INFO:tasks.workunit.client.0.smithi027.stdout:1/833: fsync d3/d7/d28/d92/dad/fe4 0 2024-04-20T10:59:22.432 INFO:tasks.workunit.client.0.smithi027.stdout:4/648: truncate d0/d14/d6b/da3/d18/fa8 1328657 0 2024-04-20T10:59:22.432 INFO:tasks.workunit.client.0.smithi027.stdout:4/649: dread - d0/d14/d6b/da3/d53/d6a/da1/db0/fcb zero size 2024-04-20T10:59:22.435 INFO:tasks.workunit.client.0.smithi027.stdout:9/581: creat d4/d1b/d50/d5d/fb1 x:0 0 0 2024-04-20T10:59:22.436 INFO:tasks.workunit.client.0.smithi027.stdout:1/834: unlink d3/d7/d38/d62/l76 0 2024-04-20T10:59:22.438 INFO:tasks.workunit.client.0.smithi027.stdout:4/650: creat d0/d5e/d6c/fec x:0 0 0 2024-04-20T10:59:22.438 INFO:tasks.workunit.client.0.smithi027.stdout:8/706: dwrite d9/de/f1d [0,4194304] 0 2024-04-20T10:59:22.439 INFO:tasks.workunit.client.0.smithi027.stdout:1/835: mknod d3/d20/dfb/c129 0 2024-04-20T10:59:22.439 INFO:tasks.workunit.client.0.smithi027.stdout:1/836: write d3/d20/ffc [742158,39451] 0 2024-04-20T10:59:22.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/651: dread d0/d14/d6b/da3/d1b/d33/d5c/fd2 [0,4194304] 0 2024-04-20T10:59:22.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/652: write d0/f3a [4916968,70419] 0 2024-04-20T10:59:22.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/653: readlink d0/d14/d6b/da3/d53/d79/l84 0 2024-04-20T10:59:22.441 INFO:tasks.workunit.client.0.smithi027.stdout:8/707: mknod d9/de/d1e/d4a/d6d/db7/d97/cf0 0 2024-04-20T10:59:22.441 INFO:tasks.workunit.client.0.smithi027.stdout:1/837: rename d3/d7/f1d to d3/d20/d85/f12a 0 2024-04-20T10:59:22.441 INFO:tasks.workunit.client.0.smithi027.stdout:7/647: sync 2024-04-20T10:59:22.441 INFO:tasks.workunit.client.0.smithi027.stdout:2/569: sync 2024-04-20T10:59:22.441 INFO:tasks.workunit.client.0.smithi027.stdout:6/705: sync 2024-04-20T10:59:22.442 INFO:tasks.workunit.client.0.smithi027.stdout:4/654: symlink d0/d14/d6b/d92/dcd/led 0 2024-04-20T10:59:22.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/838: symlink d3/d7/d28/l12b 0 2024-04-20T10:59:22.444 INFO:tasks.workunit.client.0.smithi027.stdout:2/570: creat d5/dc/d16/d32/fb7 x:0 0 0 2024-04-20T10:59:22.445 INFO:tasks.workunit.client.0.smithi027.stdout:7/648: mkdir d4/d5/d8/d21/d94/da7/dce 0 2024-04-20T10:59:22.446 INFO:tasks.workunit.client.0.smithi027.stdout:0/770: dwrite d0/d11/d2c/d2e/d2b/d7b/d12/d25/f31 [0,4194304] 0 2024-04-20T10:59:22.446 INFO:tasks.workunit.client.0.smithi027.stdout:0/771: chown d0/d11/d2c/d2e/d35/c89 31070 1 2024-04-20T10:59:22.446 INFO:tasks.workunit.client.0.smithi027.stdout:0/772: fsync d0/d11/d4b/fa3 0 2024-04-20T10:59:22.446 INFO:tasks.workunit.client.0.smithi027.stdout:2/571: dread d5/d26/f74 [0,4194304] 0 2024-04-20T10:59:22.447 INFO:tasks.workunit.client.0.smithi027.stdout:6/706: link da/d21/dab/fc7 da/d21/d96/fe3 0 2024-04-20T10:59:22.447 INFO:tasks.workunit.client.0.smithi027.stdout:6/707: chown da/d21/dab/d7e/d86/fb0 11201 1 2024-04-20T10:59:22.448 INFO:tasks.workunit.client.0.smithi027.stdout:4/655: symlink d0/d14/d6b/da3/d1a/d3b/lee 0 2024-04-20T10:59:22.448 INFO:tasks.workunit.client.0.smithi027.stdout:5/630: dwrite d19/d1b/d1c/d1e/d25/d40/f81 [0,4194304] 0 2024-04-20T10:59:22.449 INFO:tasks.workunit.client.0.smithi027.stdout:1/839: rmdir d3/d20/d107 39 2024-04-20T10:59:22.451 INFO:tasks.workunit.client.0.smithi027.stdout:7/649: rename d4/d5/d8/f43 to d4/d5/d8/d46/d77/fcf 0 2024-04-20T10:59:22.451 INFO:tasks.workunit.client.0.smithi027.stdout:7/650: chown d4/d5/d6f 26418483 1 2024-04-20T10:59:22.451 INFO:tasks.workunit.client.0.smithi027.stdout:7/651: stat d4/d5/d8/d46 0 2024-04-20T10:59:22.451 INFO:tasks.workunit.client.0.smithi027.stdout:7/652: write d4/d1b/d4a/fb3 [3677115,88225] 0 2024-04-20T10:59:22.452 INFO:tasks.workunit.client.0.smithi027.stdout:0/773: truncate d0/d11/d2c/d2e/d35/d8c/f98 1002026 0 2024-04-20T10:59:22.452 INFO:tasks.workunit.client.0.smithi027.stdout:8/708: dread d9/de/da3/f30 [0,4194304] 0 2024-04-20T10:59:22.453 INFO:tasks.workunit.client.0.smithi027.stdout:2/572: rmdir d5/dc/d16/d37 39 2024-04-20T10:59:22.454 INFO:tasks.workunit.client.0.smithi027.stdout:4/656: getdents d0/d14/d6b/da3/d18 0 2024-04-20T10:59:22.455 INFO:tasks.workunit.client.0.smithi027.stdout:0/774: dread d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f90 [0,4194304] 0 2024-04-20T10:59:22.455 INFO:tasks.workunit.client.0.smithi027.stdout:5/631: mkdir d19/d1b/d1c/d1e/d83/dcc 0 2024-04-20T10:59:22.457 INFO:tasks.workunit.client.0.smithi027.stdout:6/708: rename da/d21/dab/d1f/f3f to da/d21/dab/d19/d2e/d6b/fe4 0 2024-04-20T10:59:22.458 INFO:tasks.workunit.client.0.smithi027.stdout:6/709: fdatasync da/d21/dab/d19/f74 0 2024-04-20T10:59:22.458 INFO:tasks.workunit.client.0.smithi027.stdout:6/710: readlink da/dd/d3d/l40 0 2024-04-20T10:59:22.459 INFO:tasks.workunit.client.0.smithi027.stdout:7/653: creat d4/d5/d8/d46/d77/fd0 x:0 0 0 2024-04-20T10:59:22.461 INFO:tasks.workunit.client.0.smithi027.stdout:0/775: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f109 x:0 0 0 2024-04-20T10:59:22.461 INFO:tasks.workunit.client.0.smithi027.stdout:0/776: chown d0/d11/d2c/f52 2006065056 1 2024-04-20T10:59:22.463 INFO:tasks.workunit.client.0.smithi027.stdout:1/840: write d3/d7/d28/f48 [570824,33575] 0 2024-04-20T10:59:22.464 INFO:tasks.workunit.client.0.smithi027.stdout:0/777: mkdir d0/d11/d2c/d2e/d2b/d7b/d12/d25/d10a 0 2024-04-20T10:59:22.466 INFO:tasks.workunit.client.0.smithi027.stdout:5/632: link d19/d1b/d1c/d1e/d3c/f96 d19/d1b/d1c/d1e/d3c/fcd 0 2024-04-20T10:59:22.468 INFO:tasks.workunit.client.0.smithi027.stdout:1/841: stat d3/d7/d38/db8/l114 0 2024-04-20T10:59:22.474 INFO:tasks.workunit.client.0.smithi027.stdout:2/573: dwrite d5/dc/d16/d37/f3f [0,4194304] 0 2024-04-20T10:59:22.474 INFO:tasks.workunit.client.0.smithi027.stdout:2/574: fdatasync d5/d44/f90 0 2024-04-20T10:59:22.474 INFO:tasks.workunit.client.0.smithi027.stdout:2/575: fdatasync d5/dc/d16/d2a/f9f 0 2024-04-20T10:59:22.474 INFO:tasks.workunit.client.0.smithi027.stdout:2/576: fdatasync d5/d44/da7/f48 0 2024-04-20T10:59:22.475 INFO:tasks.workunit.client.0.smithi027.stdout:0/778: rename d0/d11/d62/fb8 to d0/d59/d9e/f10b 0 2024-04-20T10:59:22.479 INFO:tasks.workunit.client.0.smithi027.stdout:2/577: link d5/d44/da7/da0/l9e d5/d26/d78/lb8 0 2024-04-20T10:59:22.479 INFO:tasks.workunit.client.0.smithi027.stdout:2/578: write d5/dc/d16/d60/d8e/fb5 [448707,58742] 0 2024-04-20T10:59:22.481 INFO:tasks.workunit.client.0.smithi027.stdout:0/779: rename d0/d11/d2c/d2e/d2b/d7b/d12/dd7/ldb to d0/d11/d2c/d2e/d2b/d7b/d12/d1a/l10c 0 2024-04-20T10:59:22.481 INFO:tasks.workunit.client.0.smithi027.stdout:0/780: chown d0/d11/d2c/d2e/d61/cc4 66236890 1 2024-04-20T10:59:22.489 INFO:tasks.workunit.client.0.smithi027.stdout:2/579: dread d5/dc/d16/d37/f3f [0,4194304] 0 2024-04-20T10:59:22.492 INFO:tasks.workunit.client.0.smithi027.stdout:2/580: mkdir d5/dc/d16/d32/dab/db9 0 2024-04-20T10:59:22.493 INFO:tasks.workunit.client.0.smithi027.stdout:2/581: mkdir d5/dc/d16/d32/dab/dba 0 2024-04-20T10:59:22.493 INFO:tasks.workunit.client.0.smithi027.stdout:2/582: chown d5/dc/d16/d32 63 1 2024-04-20T10:59:22.493 INFO:tasks.workunit.client.0.smithi027.stdout:2/583: chown d5/dc/d16/d32/l92 0 1 2024-04-20T10:59:22.498 INFO:tasks.workunit.client.0.smithi027.stdout:8/709: dwrite d9/de/d1e/d4a/f45 [0,4194304] 0 2024-04-20T10:59:22.498 INFO:tasks.workunit.client.0.smithi027.stdout:7/654: dwrite d4/d5/d8/d21/d94/fa8 [0,4194304] 0 2024-04-20T10:59:22.498 INFO:tasks.workunit.client.0.smithi027.stdout:6/711: dwrite da/d21/dab/d19/f93 [0,4194304] 0 2024-04-20T10:59:22.499 INFO:tasks.workunit.client.0.smithi027.stdout:4/657: dwrite d0/d14/d6b/da3/d1b/d2f/f3f [0,4194304] 0 2024-04-20T10:59:22.505 INFO:tasks.workunit.client.0.smithi027.stdout:4/658: creat d0/d14/d6b/da3/d18/dbf/fef x:0 0 0 2024-04-20T10:59:22.505 INFO:tasks.workunit.client.0.smithi027.stdout:7/655: write d4/fc [1683359,15785] 0 2024-04-20T10:59:22.506 INFO:tasks.workunit.client.0.smithi027.stdout:4/659: getdents d0/d14/d6b/d92 0 2024-04-20T10:59:22.506 INFO:tasks.workunit.client.0.smithi027.stdout:4/660: write d0/d14/d6b/da3/d1b/f3d [897515,15863] 0 2024-04-20T10:59:22.509 INFO:tasks.workunit.client.0.smithi027.stdout:2/584: dread d5/d26/f3a [0,4194304] 0 2024-04-20T10:59:22.509 INFO:tasks.workunit.client.0.smithi027.stdout:7/656: dread d4/d1b/d4a/d62/f56 [0,4194304] 0 2024-04-20T10:59:22.510 INFO:tasks.workunit.client.0.smithi027.stdout:0/781: dwrite d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f90 [0,4194304] 0 2024-04-20T10:59:22.511 INFO:tasks.workunit.client.0.smithi027.stdout:0/782: fdatasync d0/d11/f32 0 2024-04-20T10:59:22.512 INFO:tasks.workunit.client.0.smithi027.stdout:2/585: rmdir d5/d26 39 2024-04-20T10:59:22.516 INFO:tasks.workunit.client.0.smithi027.stdout:2/586: mknod d5/d44/da7/da0/cbb 0 2024-04-20T10:59:22.517 INFO:tasks.workunit.client.0.smithi027.stdout:0/783: rename d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/fe2 to d0/d11/d4b/d5b/d6f/d82/dd1/deb/f10d 0 2024-04-20T10:59:22.518 INFO:tasks.workunit.client.0.smithi027.stdout:0/784: mkdir d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/d10e 0 2024-04-20T10:59:22.518 INFO:tasks.workunit.client.0.smithi027.stdout:0/785: fdatasync d0/d11/f32 0 2024-04-20T10:59:22.518 INFO:tasks.workunit.client.0.smithi027.stdout:0/786: dread - d0/d11/d62/f81 zero size 2024-04-20T10:59:22.519 INFO:tasks.workunit.client.0.smithi027.stdout:0/787: symlink d0/d11/d2c/d2e/d61/l10f 0 2024-04-20T10:59:22.519 INFO:tasks.workunit.client.0.smithi027.stdout:0/788: write d0/d11/d2c/d2e/f2a [933284,15539] 0 2024-04-20T10:59:22.521 INFO:tasks.workunit.client.0.smithi027.stdout:0/789: link d0/d11/d4b/c94 d0/d11/d5c/c110 0 2024-04-20T10:59:22.522 INFO:tasks.workunit.client.0.smithi027.stdout:0/790: link d0/d11/d2c/d2e/d2b/d7b/d12/dd7/fdf d0/d11/d2c/d2e/d2b/d7b/d12/f111 0 2024-04-20T10:59:22.523 INFO:tasks.workunit.client.0.smithi027.stdout:2/587: dread d5/dc/d16/d60/dac/f50 [0,4194304] 0 2024-04-20T10:59:22.524 INFO:tasks.workunit.client.0.smithi027.stdout:0/791: creat d0/d11/d2c/d2e/d2b/d7b/dcf/f112 x:0 0 0 2024-04-20T10:59:22.525 INFO:tasks.workunit.client.0.smithi027.stdout:0/792: creat d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/d10e/f113 x:0 0 0 2024-04-20T10:59:22.526 INFO:tasks.workunit.client.0.smithi027.stdout:0/793: read d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f86 [259590,78872] 0 2024-04-20T10:59:22.528 INFO:tasks.workunit.client.0.smithi027.stdout:0/794: link d0/d11/d2c/d2e/d2b/d7b/d34/d5a/f75 d0/d11/f114 0 2024-04-20T10:59:22.528 INFO:tasks.workunit.client.0.smithi027.stdout:6/712: dwrite da/d21/dab/d19/d2e/f38 [0,4194304] 0 2024-04-20T10:59:22.531 INFO:tasks.workunit.client.0.smithi027.stdout:4/661: dwrite d0/d14/d6b/da3/d18/dbf/fef [0,4194304] 0 2024-04-20T10:59:22.531 INFO:tasks.workunit.client.0.smithi027.stdout:2/588: read f1 [642556,68941] 0 2024-04-20T10:59:22.532 INFO:tasks.workunit.client.0.smithi027.stdout:2/589: truncate d5/f4b 4336100 0 2024-04-20T10:59:22.532 INFO:tasks.workunit.client.0.smithi027.stdout:7/657: dwrite d4/fb4 [0,4194304] 0 2024-04-20T10:59:22.532 INFO:tasks.workunit.client.0.smithi027.stdout:6/713: mknod da/dd/d3d/ce5 0 2024-04-20T10:59:22.532 INFO:tasks.workunit.client.0.smithi027.stdout:6/714: chown da/d21/dab/d19/d2e/d6b/d65/c77 0 1 2024-04-20T10:59:22.533 INFO:tasks.workunit.client.0.smithi027.stdout:6/715: write da/d21/d96/fa4 [134831,23807] 0 2024-04-20T10:59:22.534 INFO:tasks.workunit.client.0.smithi027.stdout:2/590: getdents d5 0 2024-04-20T10:59:22.535 INFO:tasks.workunit.client.0.smithi027.stdout:8/710: dwrite d9/de/d64/d6c/f8c [4194304,4194304] 0 2024-04-20T10:59:22.537 INFO:tasks.workunit.client.0.smithi027.stdout:8/711: dread d9/de/d1e/d4a/d6d/dea/d59/f99 [0,4194304] 0 2024-04-20T10:59:22.537 INFO:tasks.workunit.client.0.smithi027.stdout:4/662: link d0/d14/c5d d0/d14/d6b/da3/cf0 0 2024-04-20T10:59:22.539 INFO:tasks.workunit.client.0.smithi027.stdout:2/591: mkdir d5/dbc 0 2024-04-20T10:59:22.545 INFO:tasks.workunit.client.0.smithi027.stdout:4/663: mknod d0/d48/cf1 0 2024-04-20T10:59:22.545 INFO:tasks.workunit.client.0.smithi027.stdout:4/664: getdents d0/d14/d6b/d92 0 2024-04-20T10:59:22.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/592: dread d5/dc/d16/d37/f46 [0,4194304] 0 2024-04-20T10:59:22.547 INFO:tasks.workunit.client.0.smithi027.stdout:4/665: stat d0/d14/d6b/lce 0 2024-04-20T10:59:22.547 INFO:tasks.workunit.client.0.smithi027.stdout:2/593: read d5/f17 [4025575,52237] 0 2024-04-20T10:59:22.548 INFO:tasks.workunit.client.0.smithi027.stdout:2/594: dread d5/dc/d16/d37/f46 [0,4194304] 0 2024-04-20T10:59:22.553 INFO:tasks.workunit.client.0.smithi027.stdout:3/657: sync 2024-04-20T10:59:22.554 INFO:tasks.workunit.client.0.smithi027.stdout:3/658: dread d2/dd/f58 [0,4194304] 0 2024-04-20T10:59:22.560 INFO:tasks.workunit.client.0.smithi027.stdout:7/658: dwrite d4/d4d/f91 [0,4194304] 0 2024-04-20T10:59:22.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.561+0000 7f3c2b515640 1 Processor -- start 2024-04-20T10:59:22.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.561+0000 7f3c2b515640 1 -- start start 2024-04-20T10:59:22.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.562+0000 7f3c2b515640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24152320 0x7f3c24070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.562+0000 7f3c2b515640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3c2406fcd0 0x7f3c24071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.562+0000 7f3c2b515640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c240c9970 con 0x7f3c24151f40 2024-04-20T10:59:22.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.562+0000 7f3c2b515640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f3c24070210 con 0x7f3c2406fcd0 2024-04-20T10:59:22.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.562+0000 7f3c2b515640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2415b750 con 0x7f3c24152320 2024-04-20T10:59:22.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.562+0000 7f3c28ab0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24152320 0x7f3c24070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.563+0000 7f3c28ab0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24152320 0x7f3c24070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:53704/0 (socket says 172.21.15.27:53704) 2024-04-20T10:59:22.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.563+0000 7f3c28ab0640 1 -- 172.21.15.27:0/816716259 learned_addr learned my addr 172.21.15.27:0/816716259 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:22.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.563+0000 7f3c29ab2640 1 --2- 172.21.15.27:0/816716259 >> v2:172.21.15.137:3300/0 conn(0x7f3c2406fcd0 0x7f3c24071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.563+0000 7f3c28ab0640 1 -- 172.21.15.27:0/816716259 >> v2:172.21.15.137:3300/0 conn(0x7f3c2406fcd0 msgr2=0x7f3c24071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:22.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.563+0000 7f3c28ab0640 1 --2- 172.21.15.27:0/816716259 >> v2:172.21.15.137:3300/0 conn(0x7f3c2406fcd0 0x7f3c24071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.563+0000 7f3c28ab0640 1 -- 172.21.15.27:0/816716259 >> v1:172.21.15.137:6789/0 conn(0x7f3c24151f40 legacy=0x7f3c2406f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.563+0000 7f3c28ab0640 1 -- 172.21.15.27:0/816716259 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2415ba30 con 0x7f3c24152320 2024-04-20T10:59:22.565 INFO:tasks.workunit.client.0.smithi027.stdout:0/795: dwrite d0/d11/d2c/da1/fe3 [0,4194304] 0 2024-04-20T10:59:22.565 INFO:tasks.workunit.client.0.smithi027.stdout:4/666: dwrite d0/d14/d6b/da3/d1b/d33/f35 [4194304,4194304] 0 2024-04-20T10:59:22.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.565+0000 7f3c28ab0640 1 --2- 172.21.15.27:0/816716259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24152320 0x7f3c24070ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009870 tx=0x7f3c1402ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178479589663b6db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:22.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.565+0000 7f3c1bfff640 1 -- 172.21.15.27:0/816716259 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1402f840 con 0x7f3c24152320 2024-04-20T10:59:22.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.565+0000 7f3c1bfff640 1 -- 172.21.15.27:0/816716259 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1402f9e0 con 0x7f3c24152320 2024-04-20T10:59:22.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.565+0000 7f3c1bfff640 1 -- 172.21.15.27:0/816716259 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1402fdb0 con 0x7f3c24152320 2024-04-20T10:59:22.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.566+0000 7f3c2b515640 1 -- 172.21.15.27:0/816716259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24152320 msgr2=0x7f3c24070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.566+0000 7f3c2b515640 1 --2- 172.21.15.27:0/816716259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24152320 0x7f3c24070ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c14009870 tx=0x7f3c1402ba20 comp rx=0 tx=0).stop 2024-04-20T10:59:22.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.566+0000 7f3c2b515640 1 -- 172.21.15.27:0/816716259 shutdown_connections 2024-04-20T10:59:22.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.566+0000 7f3c2b515640 1 --2- 172.21.15.27:0/816716259 >> v2:172.21.15.137:3300/0 conn(0x7f3c2406fcd0 0x7f3c24071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.566+0000 7f3c2b515640 1 --2- 172.21.15.27:0/816716259 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24152320 0x7f3c24070ac0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.566+0000 7f3c2b515640 1 -- 172.21.15.27:0/816716259 >> 172.21.15.27:0/816716259 conn(0x7f3c240cee90 msgr2=0x7f3c240cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:22.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.567+0000 7f3c2b515640 1 -- 172.21.15.27:0/816716259 shutdown_connections 2024-04-20T10:59:22.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.567+0000 7f3c2b515640 1 -- 172.21.15.27:0/816716259 wait complete. 2024-04-20T10:59:22.568 INFO:tasks.workunit.client.0.smithi027.stdout:0/796: unlink d0/d11/fa9 0 2024-04-20T10:59:22.568 INFO:tasks.workunit.client.0.smithi027.stdout:4/667: rmdir d0/d14/d6b/da3/d1b 39 2024-04-20T10:59:22.568 INFO:tasks.workunit.client.0.smithi027.stdout:8/712: dwrite d9/de/d1e/d4a/d6d/d79/f92 [0,4194304] 0 2024-04-20T10:59:22.568 INFO:tasks.workunit.client.0.smithi027.stdout:8/713: stat d9/de/da3/f7f 0 2024-04-20T10:59:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.568+0000 7f3c2b515640 1 Processor -- start 2024-04-20T10:59:22.569 INFO:tasks.workunit.client.0.smithi027.stdout:0/797: symlink d0/d11/d2c/d2e/d2b/l115 0 2024-04-20T10:59:22.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.568+0000 7f3c2b515640 1 -- start start 2024-04-20T10:59:22.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c2b515640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2406fcd0 0x7f3c2414bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c2b515640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24151f40 0x7f3c240c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c2b515640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c24152320 0x7f3c240c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c2b515640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2415c2d0 con 0x7f3c2406fcd0 2024-04-20T10:59:22.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c2b515640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2415bff0 con 0x7f3c24152320 2024-04-20T10:59:22.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c2b515640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2406f0a0 con 0x7f3c24151f40 2024-04-20T10:59:22.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c292b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2406fcd0 0x7f3c2414bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c292b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2406fcd0 0x7f3c2414bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33432/0 (socket says 172.21.15.27:33432) 2024-04-20T10:59:22.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c292b1640 1 -- 172.21.15.27:0/955936668 learned_addr learned my addr 172.21.15.27:0/955936668 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:22.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c28ab0640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24151f40 0x7f3c240c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c292b1640 1 -- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24151f40 msgr2=0x7f3c240c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c292b1640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24151f40 0x7f3c240c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c292b1640 1 -- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c24152320 msgr2=0x7f3c240c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c292b1640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c24152320 0x7f3c240c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.572 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.570+0000 7f3c292b1640 1 -- 172.21.15.27:0/955936668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c24159740 con 0x7f3c2406fcd0 2024-04-20T10:59:22.572 INFO:tasks.workunit.client.0.smithi027.stdout:8/714: mkdir d9/de/d20/dbf/df1 0 2024-04-20T10:59:22.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.571+0000 7f3c292b1640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2406fcd0 0x7f3c2414bc30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c450 tx=0x7f3c2000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:22.573 INFO:tasks.workunit.client.0.smithi027.stdout:8/715: write d9/de/d1e/dc1/fcc [936212,77995] 0 2024-04-20T10:59:22.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.571+0000 7f3c19ffb640 1 -- 172.21.15.27:0/955936668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20017070 con 0x7f3c2406fcd0 2024-04-20T10:59:22.573 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.571+0000 7f3c19ffb640 1 -- 172.21.15.27:0/955936668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c200052d0 con 0x7f3c2406fcd0 2024-04-20T10:59:22.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.571+0000 7f3c19ffb640 1 -- 172.21.15.27:0/955936668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20003830 con 0x7f3c2406fcd0 2024-04-20T10:59:22.574 INFO:tasks.workunit.client.0.smithi027.stdout:7/659: write d4/d4d/f4f [2198320,119771] 0 2024-04-20T10:59:22.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.571+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2415b260 con 0x7f3c2406fcd0 2024-04-20T10:59:22.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.571+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c241f5410 con 0x7f3c2406fcd0 2024-04-20T10:59:22.575 INFO:tasks.workunit.client.0.smithi027.stdout:0/798: mknod d0/d11/d4b/d5b/d6f/d82/c116 0 2024-04-20T10:59:22.575 INFO:tasks.workunit.client.0.smithi027.stdout:0/799: dread - d0/d11/d5c/df0/f102 zero size 2024-04-20T10:59:22.575 INFO:tasks.workunit.client.0.smithi027.stdout:8/716: creat d9/de/d1e/d4a/d6d/db7/d97/ff2 x:0 0 0 2024-04-20T10:59:22.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.573+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2406e010 con 0x7f3c2406fcd0 2024-04-20T10:59:22.575 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.573+0000 7f3c19ffb640 1 -- 172.21.15.27:0/955936668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c2001a070 con 0x7f3c2406fcd0 2024-04-20T10:59:22.576 INFO:tasks.workunit.client.0.smithi027.stdout:7/660: mkdir d4/da2/dd1 0 2024-04-20T10:59:22.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.574+0000 7f3c19ffb640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3c0806c970 0x7f3c0806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:22.576 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.574+0000 7f3c19ffb640 1 -- 172.21.15.27:0/955936668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f3c2008adc0 con 0x7f3c2406fcd0 2024-04-20T10:59:22.576 INFO:tasks.workunit.client.0.smithi027.stdout:0/800: unlink d0/d59/dbf/ffa 0 2024-04-20T10:59:22.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.575+0000 7f3c28ab0640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3c0806c970 0x7f3c0806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:22.577 INFO:tasks.workunit.client.0.smithi027.stdout:4/668: dread d0/d14/d6b/da3/d1b/d2f/f67 [0,4194304] 0 2024-04-20T10:59:22.577 INFO:tasks.workunit.client.0.smithi027.stdout:8/717: creat d9/de/d1e/d4a/d6d/dea/d59/da0/dbd/ff3 x:0 0 0 2024-04-20T10:59:22.577 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.575+0000 7f3c28ab0640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3c0806c970 0x7f3c0806ee30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3c14036000 tx=0x7f3c14008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:22.578 INFO:tasks.workunit.client.0.smithi027.stdout:7/661: creat d4/d1b/d58/fd2 x:0 0 0 2024-04-20T10:59:22.578 INFO:tasks.workunit.client.0.smithi027.stdout:0/801: mkdir d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/d10e/d117 0 2024-04-20T10:59:22.578 INFO:tasks.workunit.client.0.smithi027.stdout:4/669: mkdir d0/d48/d68/d76/de2/df2 0 2024-04-20T10:59:22.578 INFO:tasks.workunit.client.0.smithi027.stdout:4/670: write d0/d14/d6b/da3/d18/db9/fd6 [547282,33048] 0 2024-04-20T10:59:22.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.579+0000 7f3c19ffb640 1 -- 172.21.15.27:0/955936668 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3c2406e010 con 0x7f3c2406fcd0 2024-04-20T10:59:22.582 INFO:tasks.workunit.client.0.smithi027.stdout:2/595: dwrite d5/d1e/d21/f33 [0,4194304] 0 2024-04-20T10:59:22.582 INFO:tasks.workunit.client.0.smithi027.stdout:3/659: dwrite d2/d18/d2c/d4c/d67/fab [0,4194304] 0 2024-04-20T10:59:22.584 INFO:tasks.workunit.client.0.smithi027.stdout:4/671: creat d0/d14/d6b/da3/d53/d6a/ff3 x:0 0 0 2024-04-20T10:59:22.584 INFO:tasks.workunit.client.0.smithi027.stdout:8/718: creat d9/de/d1e/d4a/ff4 x:0 0 0 2024-04-20T10:59:22.584 INFO:tasks.workunit.client.0.smithi027.stdout:4/672: fdatasync d0/d14/d6b/da3/d1b/d33/f35 0 2024-04-20T10:59:22.584 INFO:tasks.workunit.client.0.smithi027.stdout:0/802: dread d0/d11/d2c/fcd [0,4194304] 0 2024-04-20T10:59:22.587 INFO:tasks.workunit.client.0.smithi027.stdout:0/803: dread d0/d59/dbf/fd2 [0,4194304] 0 2024-04-20T10:59:22.587 INFO:tasks.workunit.client.0.smithi027.stdout:0/804: write d0/d11/d2c/d2e/d61/f78 [4422499,112827] 0 2024-04-20T10:59:22.594 INFO:tasks.workunit.client.0.smithi027.stdout:0/805: write d0/d59/dbf/ff5 [3050575,54657] 0 2024-04-20T10:59:22.595 INFO:tasks.workunit.client.0.smithi027.stdout:3/660: link d2/f16 d2/dd/d57/d8c/fcf 0 2024-04-20T10:59:22.595 INFO:tasks.workunit.client.0.smithi027.stdout:4/673: creat d0/d14/d6b/da3/d1a/ff4 x:0 0 0 2024-04-20T10:59:22.595 INFO:tasks.workunit.client.0.smithi027.stdout:7/662: dwrite d4/d5/d8/d46/f8f [4194304,4194304] 0 2024-04-20T10:59:22.597 INFO:tasks.workunit.client.0.smithi027.stdout:0/806: rmdir d0/d59/dbf 39 2024-04-20T10:59:22.597 INFO:tasks.workunit.client.0.smithi027.stdout:0/807: write d0/d11/f114 [930243,60151] 0 2024-04-20T10:59:22.597 INFO:tasks.workunit.client.0.smithi027.stdout:0/808: fsync d0/f42 0 2024-04-20T10:59:22.597 INFO:tasks.workunit.client.0.smithi027.stdout:4/674: creat d0/d48/ff5 x:0 0 0 2024-04-20T10:59:22.599 INFO:tasks.workunit.client.0.smithi027.stdout:7/663: creat d4/d5/fd3 x:0 0 0 2024-04-20T10:59:22.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/809: mkdir d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/d10e/d117/d118 0 2024-04-20T10:59:22.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/810: chown d0/d11/f114 6689 1 2024-04-20T10:59:22.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/811: write d0/d59/f73 [2264934,39290] 0 2024-04-20T10:59:22.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/812: fdatasync d0/d11/f65 0 2024-04-20T10:59:22.601 INFO:tasks.workunit.client.0.smithi027.stdout:0/813: chown d0/d11/d5c/c110 16770329 1 2024-04-20T10:59:22.602 INFO:tasks.workunit.client.0.smithi027.stdout:0/814: creat d0/d11/d2c/d2e/d2b/d7b/d12/f119 x:0 0 0 2024-04-20T10:59:22.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/815: mknod d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/c11a 0 2024-04-20T10:59:22.605 INFO:tasks.workunit.client.0.smithi027.stdout:0/816: rmdir d0/d11/d4b/d5b/d6f/d82/dd1/deb 39 2024-04-20T10:59:22.607 INFO:tasks.workunit.client.0.smithi027.stdout:3/661: write d2/dd/d57/d8c/fb8 [3223441,111656] 0 2024-04-20T10:59:22.607 INFO:tasks.workunit.client.0.smithi027.stdout:0/817: rename d0/d11/d2c/d2e/d2b/l115 to d0/d11/d2c/d2e/d2b/d7b/dcf/l11b 0 2024-04-20T10:59:22.608 INFO:tasks.workunit.client.0.smithi027.stdout:3/662: mkdir d2/d18/d2c/d4c/d67/d6c/dd0 0 2024-04-20T10:59:22.609 INFO:tasks.workunit.client.0.smithi027.stdout:3/663: symlink d2/dd/df/d33/ld1 0 2024-04-20T10:59:22.609 INFO:tasks.workunit.client.0.smithi027.stdout:0/818: dread d0/d11/d2c/d2e/d2b/d7b/d64/fb4 [0,4194304] 0 2024-04-20T10:59:22.610 INFO:tasks.workunit.client.0.smithi027.stdout:3/664: creat d2/dd/d57/d8d/fd2 x:0 0 0 2024-04-20T10:59:22.610 INFO:tasks.workunit.client.0.smithi027.stdout:3/665: truncate d2/fcb 759833 0 2024-04-20T10:59:22.610 INFO:tasks.workunit.client.0.smithi027.stdout:0/819: getdents d0/d11/d2c/d2e/d2b/d7b/d34/d5a 0 2024-04-20T10:59:22.611 INFO:tasks.workunit.client.0.smithi027.stdout:2/596: dwrite d5/d44/da7/da0/faa [0,4194304] 0 2024-04-20T10:59:22.614 INFO:tasks.workunit.client.0.smithi027.stdout:0/820: creat d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/d10e/d117/f11c x:0 0 0 2024-04-20T10:59:22.614 INFO:tasks.workunit.client.0.smithi027.stdout:7/664: read d4/d5/d8/d46/f8f [6936406,76683] 0 2024-04-20T10:59:22.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/675: dread d0/d14/d6b/da3/d18/f38 [0,4194304] 0 2024-04-20T10:59:22.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/676: fdatasync d0/d14/d6b/da3/d18/f54 0 2024-04-20T10:59:22.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/677: chown d0/d48/c64 1912965 1 2024-04-20T10:59:22.615 INFO:tasks.workunit.client.0.smithi027.stdout:4/678: write d0/d14/d6b/da3/d18/dbf/fef [4368764,43649] 0 2024-04-20T10:59:22.615 INFO:tasks.workunit.client.0.smithi027.stdout:2/597: mknod d5/dc/d16/d2a/d59/d66/cbd 0 2024-04-20T10:59:22.615 INFO:tasks.workunit.client.0.smithi027.stdout:0/821: symlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/l11d 0 2024-04-20T10:59:22.615 INFO:tasks.workunit.client.0.smithi027.stdout:2/598: write d5/dc/d16/d37/f6b [1853212,70486] 0 2024-04-20T10:59:22.619 INFO:tasks.workunit.client.0.smithi027.stdout:2/599: truncate d5/d26/f2c 737119 0 2024-04-20T10:59:22.621 INFO:tasks.workunit.client.0.smithi027.stdout:2/600: truncate d5/dc/fe 4603949 0 2024-04-20T10:59:22.625 INFO:tasks.workunit.client.0.smithi027.stdout:2/601: dread d5/dc/d16/d37/f3f [0,4194304] 0 2024-04-20T10:59:22.629 INFO:tasks.workunit.client.0.smithi027.stdout:4/679: dread d0/d14/d6b/da3/d1b/d2f/f3f [0,4194304] 0 2024-04-20T10:59:22.630 INFO:tasks.workunit.client.0.smithi027.stdout:2/602: creat d5/d26/d47/fbe x:0 0 0 2024-04-20T10:59:22.630 INFO:tasks.workunit.client.0.smithi027.stdout:2/603: readlink d5/d26/d78/lb6 0 2024-04-20T10:59:22.632 INFO:tasks.workunit.client.0.smithi027.stdout:4/680: creat d0/d14/d6b/da3/d1b/d2f/ff6 x:0 0 0 2024-04-20T10:59:22.632 INFO:tasks.workunit.client.0.smithi027.stdout:3/666: getdents d2/dd/d57/d8d 0 2024-04-20T10:59:22.635 INFO:tasks.workunit.client.0.smithi027.stdout:7/665: dwrite d4/d5/d6f/fca [0,4194304] 0 2024-04-20T10:59:22.636 INFO:tasks.workunit.client.0.smithi027.stdout:7/666: fdatasync d4/d5b/f9e 0 2024-04-20T10:59:22.636 INFO:tasks.workunit.client.0.smithi027.stdout:7/667: fsync d4/d1b/d4a/d62/faf 0 2024-04-20T10:59:22.636 INFO:tasks.workunit.client.0.smithi027.stdout:4/681: getdents d0/d14/d6b/da3/d1b/d33/d5c 0 2024-04-20T10:59:22.636 INFO:tasks.workunit.client.0.smithi027.stdout:3/667: mknod d2/dd/d22/cd3 0 2024-04-20T10:59:22.636 INFO:tasks.workunit.client.0.smithi027.stdout:3/668: chown d2/dd/df/c1b 18694 1 2024-04-20T10:59:22.638 INFO:tasks.workunit.client.0.smithi027.stdout:4/682: mknod d0/d14/d6b/da3/d53/d6a/cf7 0 2024-04-20T10:59:22.639 INFO:tasks.workunit.client.0.smithi027.stdout:4/683: rmdir d0/d14/d6b/da3/d1b/d2f 39 2024-04-20T10:59:22.641 INFO:tasks.workunit.client.0.smithi027.stdout:4/684: unlink d0/d5e/d6c/fbe 0 2024-04-20T10:59:22.642 INFO:tasks.workunit.client.0.smithi027.stdout:4/685: mkdir d0/d5e/d6c/dd8/df8 0 2024-04-20T10:59:22.644 INFO:tasks.workunit.client.0.smithi027.stdout:4/686: symlink d0/d14/d6b/da3/d53/d6a/lf9 0 2024-04-20T10:59:22.645 INFO:tasks.workunit.client.0.smithi027.stdout:2/604: dread d5/dc/d16/d60/dac/f94 [0,4194304] 0 2024-04-20T10:59:22.645 INFO:tasks.workunit.client.0.smithi027.stdout:2/605: write d5/dc/f55 [818099,50680] 0 2024-04-20T10:59:22.645 INFO:tasks.workunit.client.0.smithi027.stdout:2/606: fdatasync d5/dc/d16/d2a/d59/f8d 0 2024-04-20T10:59:22.646 INFO:tasks.workunit.client.0.smithi027.stdout:2/607: read d5/d44/fa5 [795909,10882] 0 2024-04-20T10:59:22.649 INFO:tasks.workunit.client.0.smithi027.stdout:4/687: unlink d0/d14/d6b/da3/f1c 0 2024-04-20T10:59:22.650 INFO:tasks.workunit.client.0.smithi027.stdout:2/608: rmdir d5/d44/da7 39 2024-04-20T10:59:22.651 INFO:tasks.workunit.client.0.smithi027.stdout:2/609: rename d5/dc/fe to d5/dc/d16/d60/fbf 0 2024-04-20T10:59:22.652 INFO:tasks.workunit.client.0.smithi027.stdout:2/610: readlink d5/dc/l1d 0 2024-04-20T10:59:22.652 INFO:tasks.workunit.client.0.smithi027.stdout:2/611: chown d5/dc/d16/d2a/d59/f81 3802 1 2024-04-20T10:59:22.652 INFO:tasks.workunit.client.0.smithi027.stdout:4/688: symlink d0/d14/d6b/da3/d1b/d33/d5c/lfa 0 2024-04-20T10:59:22.654 INFO:tasks.workunit.client.0.smithi027.stdout:2/612: getdents d5/dc/d16/d60/dac 0 2024-04-20T10:59:22.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/689: rename d0/fb to d0/d14/d6b/da3/d18/ffb 0 2024-04-20T10:59:22.657 INFO:tasks.workunit.client.0.smithi027.stdout:2/613: creat d5/d26/fc0 x:0 0 0 2024-04-20T10:59:22.658 INFO:tasks.workunit.client.0.smithi027.stdout:4/690: getdents d0/d48/d68/d76/de2 0 2024-04-20T10:59:22.660 INFO:tasks.workunit.client.0.smithi027.stdout:4/691: rmdir d0/d5e/d6c/dd8/df8 0 2024-04-20T10:59:22.660 INFO:tasks.workunit.client.0.smithi027.stdout:4/692: read - d0/d5e/d6c/fec zero size 2024-04-20T10:59:22.660 INFO:tasks.workunit.client.0.smithi027.stdout:7/668: dwrite d4/d5b/fae [0,4194304] 0 2024-04-20T10:59:22.661 INFO:tasks.workunit.client.0.smithi027.stdout:4/693: mkdir d0/d48/d68/d76/dfc 0 2024-04-20T10:59:22.667 INFO:tasks.workunit.client.0.smithi027.stdout:4/694: chown d0/d5e/d6c/dd8 17367845 1 2024-04-20T10:59:22.667 INFO:tasks.workunit.client.0.smithi027.stdout:7/669: rename d4/d5/l7f to d4/d1b/d4a/ld4 0 2024-04-20T10:59:22.667 INFO:tasks.workunit.client.0.smithi027.stdout:4/695: mkdir d0/d14/d71/dfd 0 2024-04-20T10:59:22.667 INFO:tasks.workunit.client.0.smithi027.stdout:4/696: chown d0/d14/d6b/da3/f9e 37849 1 2024-04-20T10:59:22.667 INFO:tasks.workunit.client.0.smithi027.stdout:7/670: mkdir d4/d1b/d4a/d62/dcd/dd5 0 2024-04-20T10:59:22.667 INFO:tasks.workunit.client.0.smithi027.stdout:7/671: write d4/d5/f1f [809561,16884] 0 2024-04-20T10:59:22.670 INFO:tasks.workunit.client.0.smithi027.stdout:3/669: dwrite d2/f16 [0,4194304] 0 2024-04-20T10:59:22.670 INFO:tasks.workunit.client.0.smithi027.stdout:3/670: dread - d2/dd/d57/d8d/fd2 zero size 2024-04-20T10:59:22.670 INFO:tasks.workunit.client.0.smithi027.stdout:3/671: fsync d2/dd/d57/d8d/fd2 0 2024-04-20T10:59:22.672 INFO:tasks.workunit.client.0.smithi027.stdout:3/672: creat d2/dd/d57/fd4 x:0 0 0 2024-04-20T10:59:22.672 INFO:tasks.workunit.client.0.smithi027.stdout:3/673: chown d2/dd/d57/d8d/d9d 8 1 2024-04-20T10:59:22.673 INFO:tasks.workunit.client.0.smithi027.stdout:3/674: mkdir d2/d18/d2c/d4c/d67/d6c/dd0/dd5 0 2024-04-20T10:59:22.673 INFO:tasks.workunit.client.0.smithi027.stdout:3/675: truncate d2/dd/d57/d8d/fd2 635685 0 2024-04-20T10:59:22.673 INFO:tasks.workunit.client.0.smithi027.stdout:7/672: write d4/d1b/d4a/f44 [832456,101617] 0 2024-04-20T10:59:22.673 INFO:tasks.workunit.client.0.smithi027.stdout:3/676: mknod d2/dd/d57/d8c/cd6 0 2024-04-20T10:59:22.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/673: creat d4/d4d/fd6 x:0 0 0 2024-04-20T10:59:22.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/674: fsync d4/d1b/d58/fb8 0 2024-04-20T10:59:22.676 INFO:tasks.workunit.client.0.smithi027.stdout:7/675: unlink d4/d4d/c9d 0 2024-04-20T10:59:22.677 INFO:tasks.workunit.client.0.smithi027.stdout:7/676: readlink d4/d5/d8/l36 0 2024-04-20T10:59:22.677 INFO:tasks.workunit.client.0.smithi027.stdout:7/677: rmdir d4/d5/d8/d46/d77/d7c 39 2024-04-20T10:59:22.677 INFO:tasks.workunit.client.0.smithi027.stdout:7/678: rmdir d4/d5/d81 39 2024-04-20T10:59:22.678 INFO:tasks.workunit.client.0.smithi027.stdout:5/633: sync 2024-04-20T10:59:22.678 INFO:tasks.workunit.client.0.smithi027.stdout:9/582: sync 2024-04-20T10:59:22.678 INFO:tasks.workunit.client.0.smithi027.stdout:1/842: sync 2024-04-20T10:59:22.678 INFO:tasks.workunit.client.0.smithi027.stdout:7/679: getdents d4/d1b/d4a/d7a 0 2024-04-20T10:59:22.679 INFO:tasks.workunit.client.0.smithi027.stdout:9/583: truncate d4/d14/f49 1852856 0 2024-04-20T10:59:22.679 INFO:tasks.workunit.client.0.smithi027.stdout:9/584: write d4/d1b/d50/d5d/fb1 [966172,68815] 0 2024-04-20T10:59:22.679 INFO:tasks.workunit.client.0.smithi027.stdout:9/585: fsync d4/d2a/f66 0 2024-04-20T10:59:22.680 INFO:tasks.workunit.client.0.smithi027.stdout:5/634: getdents d19/d1b/d1c/d1e/d22 0 2024-04-20T10:59:22.682 INFO:tasks.workunit.client.0.smithi027.stdout:5/635: link d19/d1b/d1c/d1e/d22/d68/f90 d19/d1b/d1c/d1e/d58/d9d/fce 0 2024-04-20T10:59:22.683 INFO:tasks.workunit.client.0.smithi027.stdout:9/586: dread d4/f6b [0,4194304] 0 2024-04-20T10:59:22.684 INFO:tasks.workunit.client.0.smithi027.stdout:3/677: write d2/d7f/fa4 [1861346,31043] 0 2024-04-20T10:59:22.684 INFO:tasks.workunit.client.0.smithi027.stdout:3/678: chown d2/dd/d57/d8c/fb1 533423761 1 2024-04-20T10:59:22.686 INFO:tasks.workunit.client.0.smithi027.stdout:5/636: rename d19/d1b/d1c/d1e/d25/d40/d78 to d19/d1b/d1c/d1e/d22/db1/dcf 0 2024-04-20T10:59:22.686 INFO:tasks.workunit.client.0.smithi027.stdout:5/637: chown d19/c85 813416 1 2024-04-20T10:59:22.686 INFO:tasks.workunit.client.0.smithi027.stdout:9/587: mkdir d4/d1b/d1f/d45/d8a/db2 0 2024-04-20T10:59:22.687 INFO:tasks.workunit.client.0.smithi027.stdout:9/588: mkdir d4/d1b/d1f/d57/db0/db3 0 2024-04-20T10:59:22.689 INFO:tasks.workunit.client.0.smithi027.stdout:4/697: dwrite d0/d48/f9d [4194304,4194304] 0 2024-04-20T10:59:22.701 INFO:tasks.workunit.client.0.smithi027.stdout:3/679: dread d2/d80/f5a [0,4194304] 0 2024-04-20T10:59:22.702 INFO:tasks.workunit.client.0.smithi027.stdout:3/680: mknod d2/d18/d2c/cd7 0 2024-04-20T10:59:22.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/843: dwrite d3/d7/d38/ddf/d73/f127 [0,4194304] 0 2024-04-20T10:59:22.704 INFO:tasks.workunit.client.0.smithi027.stdout:3/681: read d2/dd/d22/fa2 [526497,62660] 0 2024-04-20T10:59:22.704 INFO:tasks.workunit.client.0.smithi027.stdout:3/682: chown d2/dd/d22/l9c 8012143 1 2024-04-20T10:59:22.704 INFO:tasks.workunit.client.0.smithi027.stdout:3/683: readlink d2/dd/d57/l8b 0 2024-04-20T10:59:22.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/844: creat d3/d7/d38/db8/f12c x:0 0 0 2024-04-20T10:59:22.704 INFO:tasks.workunit.client.0.smithi027.stdout:2/614: dwrite d5/dc/d16/d60/dac/f50 [4194304,4194304] 0 2024-04-20T10:59:22.705 INFO:tasks.workunit.client.0.smithi027.stdout:3/684: mknod d2/d18/d2c/cd8 0 2024-04-20T10:59:22.706 INFO:tasks.workunit.client.0.smithi027.stdout:1/845: mknod d3/d7/d28/d35/c12d 0 2024-04-20T10:59:22.706 INFO:tasks.workunit.client.0.smithi027.stdout:2/615: link d5/d26/d30/l7a d5/dc/lc1 0 2024-04-20T10:59:22.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/589: write d4/d2a/f77 [6333738,14811] 0 2024-04-20T10:59:22.709 INFO:tasks.workunit.client.0.smithi027.stdout:1/846: mknod d3/d20/d85/c12e 0 2024-04-20T10:59:22.709 INFO:tasks.workunit.client.0.smithi027.stdout:1/847: chown d3/d7/d38/f56 28 1 2024-04-20T10:59:22.713 INFO:tasks.workunit.client.0.smithi027.stdout:2/616: creat d5/d44/da7/da0/fc2 x:0 0 0 2024-04-20T10:59:22.714 INFO:tasks.workunit.client.0.smithi027.stdout:5/638: dwrite d19/d1b/d1c/d1e/d58/f94 [0,4194304] 0 2024-04-20T10:59:22.714 INFO:tasks.workunit.client.0.smithi027.stdout:5/639: write d19/d1b/d1c/d1e/d25/faf [946131,20205] 0 2024-04-20T10:59:22.714 INFO:tasks.workunit.client.0.smithi027.stdout:5/640: readlink d19/d1b/d1c/d1e/d22/db1/lb5 0 2024-04-20T10:59:22.715 INFO:tasks.workunit.client.0.smithi027.stdout:1/848: read d3/d20/ffc [43436,74701] 0 2024-04-20T10:59:22.715 INFO:tasks.workunit.client.0.smithi027.stdout:1/849: write d3/d20/f46 [4818467,113965] 0 2024-04-20T10:59:22.715 INFO:tasks.workunit.client.0.smithi027.stdout:2/617: rename d5/dc/d16/d2a/d59/f81 to d5/d1e/d21/fc3 0 2024-04-20T10:59:22.715 INFO:tasks.workunit.client.0.smithi027.stdout:2/618: write d5/d26/f2c [1484410,2836] 0 2024-04-20T10:59:22.719 INFO:tasks.workunit.client.0.smithi027.stdout:1/850: unlink d3/d45/d5d/lf3 0 2024-04-20T10:59:22.719 INFO:tasks.workunit.client.0.smithi027.stdout:1/851: fsync d3/d7/f34 0 2024-04-20T10:59:22.720 INFO:tasks.workunit.client.0.smithi027.stdout:2/619: unlink d5/d1e/l9b 0 2024-04-20T10:59:22.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/852: creat d3/d7/d38/ddf/d73/f12f x:0 0 0 2024-04-20T10:59:22.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/853: chown d3/d7/d28/l31 3845223 1 2024-04-20T10:59:22.722 INFO:tasks.workunit.client.0.smithi027.stdout:2/620: mknod d5/d26/d30/cc4 0 2024-04-20T10:59:22.723 INFO:tasks.workunit.client.0.smithi027.stdout:1/854: creat d3/d7e/f130 x:0 0 0 2024-04-20T10:59:22.724 INFO:tasks.workunit.client.0.smithi027.stdout:1/855: creat d3/d45/dba/f131 x:0 0 0 2024-04-20T10:59:22.724 INFO:tasks.workunit.client.0.smithi027.stdout:1/856: fdatasync d3/d20/f4d 0 2024-04-20T10:59:22.725 INFO:tasks.workunit.client.0.smithi027.stdout:2/621: rename d5/f28 to d5/d44/d7c/fc5 0 2024-04-20T10:59:22.726 INFO:tasks.workunit.client.0.smithi027.stdout:5/641: dread f0 [0,4194304] 0 2024-04-20T10:59:22.726 INFO:tasks.workunit.client.0.smithi027.stdout:1/857: mkdir d3/d20/d107/d132 0 2024-04-20T10:59:22.727 INFO:tasks.workunit.client.0.smithi027.stdout:1/858: truncate d3/d7/d38/ddf/f3f 4504891 0 2024-04-20T10:59:22.727 INFO:tasks.workunit.client.0.smithi027.stdout:1/859: chown d3/d7/d38/d62 232 1 2024-04-20T10:59:22.728 INFO:tasks.workunit.client.0.smithi027.stdout:4/698: dwrite d0/d14/d6b/da3/d18/db9/fd6 [0,4194304] 0 2024-04-20T10:59:22.728 INFO:tasks.workunit.client.0.smithi027.stdout:4/699: write d0/d14/d6b/da3/fe5 [792704,37701] 0 2024-04-20T10:59:22.728 INFO:tasks.workunit.client.0.smithi027.stdout:1/860: symlink d3/d7/d28/d92/dad/d118/l133 0 2024-04-20T10:59:22.729 INFO:tasks.workunit.client.0.smithi027.stdout:5/642: symlink d19/d1b/d1c/d1e/d3c/ld0 0 2024-04-20T10:59:22.729 INFO:tasks.workunit.client.0.smithi027.stdout:5/643: write d19/d1b/d1c/d1e/d3c/f24 [3331699,96125] 0 2024-04-20T10:59:22.729 INFO:tasks.workunit.client.0.smithi027.stdout:5/644: chown l10 1916 1 2024-04-20T10:59:22.729 INFO:tasks.workunit.client.0.smithi027.stdout:5/645: read d19/d1b/d1c/d1e/f88 [22614,59472] 0 2024-04-20T10:59:22.729 INFO:tasks.workunit.client.0.smithi027.stdout:5/646: fsync d19/d1b/d1c/d1e/d25/f2f 0 2024-04-20T10:59:22.729 INFO:tasks.workunit.client.0.smithi027.stdout:9/590: dwrite d4/d1b/d31/d23/f47 [0,4194304] 0 2024-04-20T10:59:22.733 INFO:tasks.workunit.client.0.smithi027.stdout:5/647: symlink d19/d1b/d1c/d1e/d22/d68/ld1 0 2024-04-20T10:59:22.733 INFO:tasks.workunit.client.0.smithi027.stdout:5/648: truncate d19/d1b/d1c/fad 979410 0 2024-04-20T10:59:22.739 INFO:tasks.workunit.client.0.smithi027.stdout:1/861: dread d3/d7/d38/ddf/d73/f7d [0,4194304] 0 2024-04-20T10:59:22.740 INFO:tasks.workunit.client.0.smithi027.stdout:3/685: dwrite d2/dd/d22/f2f [0,4194304] 0 2024-04-20T10:59:22.740 INFO:tasks.workunit.client.0.smithi027.stdout:4/700: dread d0/d14/d6b/da3/d1b/d2f/f73 [0,4194304] 0 2024-04-20T10:59:22.740 INFO:tasks.workunit.client.0.smithi027.stdout:4/701: stat d0 0 2024-04-20T10:59:22.740 INFO:tasks.workunit.client.0.smithi027.stdout:3/686: fdatasync d2/d18/d2e/f35 0 2024-04-20T10:59:22.741 INFO:tasks.workunit.client.0.smithi027.stdout:1/862: getdents d3/d45/d5d/d6c 0 2024-04-20T10:59:22.744 INFO:tasks.workunit.client.0.smithi027.stdout:3/687: getdents d2/dd/d57/d8d/dc6 0 2024-04-20T10:59:22.744 INFO:tasks.workunit.client.0.smithi027.stdout:2/622: dread d5/d1e/d21/f33 [0,4194304] 0 2024-04-20T10:59:22.746 INFO:tasks.workunit.client.0.smithi027.stdout:4/702: rename d0/d14/d6b/da3/d1b/d33/fc2 to d0/d14/d6b/da3/d1b/dc3/dc5/ffe 0 2024-04-20T10:59:22.746 INFO:tasks.workunit.client.0.smithi027.stdout:2/623: dread d5/d44/d7c/f7e [0,4194304] 0 2024-04-20T10:59:22.746 INFO:tasks.workunit.client.0.smithi027.stdout:2/624: write d5/d26/f3a [694265,26299] 0 2024-04-20T10:59:22.747 INFO:tasks.workunit.client.0.smithi027.stdout:1/863: rmdir d3/d7/d38/db8 39 2024-04-20T10:59:22.749 INFO:tasks.workunit.client.0.smithi027.stdout:9/591: dwrite d4/d1b/d31/f9a [0,4194304] 0 2024-04-20T10:59:22.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.883+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c2404fcf0 con 0x7f3c2406fcd0 2024-04-20T10:59:22.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.884+0000 7f3c19ffb640 1 -- 172.21.15.27:0/955936668 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6933 (secure 0 0 0) 0x7f3c2415b260 con 0x7f3c2406fcd0 2024-04-20T10:59:22.884 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:22.884 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5029,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:59:22.885 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3c0806c970 msgr2=0x7f3c0806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3c0806c970 0x7f3c0806ee30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3c14036000 tx=0x7f3c14008030 comp rx=0 tx=0).stop 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2406fcd0 msgr2=0x7f3c2414bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2406fcd0 0x7f3c2414bc30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c450 tx=0x7f3c2000c910 comp rx=0 tx=0).stop 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 shutdown_connections 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f3c0806c970 0x7f3c0806ee30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c24152320 0x7f3c240c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f3c24151f40 0x7f3c240c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 --2- 172.21.15.27:0/955936668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c2406fcd0 0x7f3c2414bc30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 >> 172.21.15.27:0/955936668 conn(0x7f3c240cee90 msgr2=0x7f3c2406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 shutdown_connections 2024-04-20T10:59:22.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:22.886+0000 7f3c2b515640 1 -- 172.21.15.27:0/955936668 wait complete. 2024-04-20T10:59:22.897 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T10:59:23.001 INFO:tasks.workunit.client.0.smithi027.stdout:3/688: creat d2/fd9 x:0 0 0 2024-04-20T10:59:23.001 INFO:tasks.workunit.client.0.smithi027.stdout:4/703: creat d0/d14/d6b/da3/d1a/dae/fff x:0 0 0 2024-04-20T10:59:23.001 INFO:tasks.workunit.client.0.smithi027.stdout:1/864: getdents d3/d7/d38 0 2024-04-20T10:59:23.002 INFO:tasks.workunit.client.0.smithi027.stdout:6/716: sync 2024-04-20T10:59:23.003 INFO:tasks.workunit.client.0.smithi027.stdout:3/689: creat d2/d18/d2c/d4c/d67/d6c/dd0/fda x:0 0 0 2024-04-20T10:59:23.003 INFO:tasks.workunit.client.0.smithi027.stdout:3/690: write d2/dd/f77 [698369,62055] 0 2024-04-20T10:59:23.004 INFO:tasks.workunit.client.0.smithi027.stdout:4/704: unlink d0/d14/d6b/da3/d1a/fa4 0 2024-04-20T10:59:23.005 INFO:tasks.workunit.client.0.smithi027.stdout:1/865: rename d3/d7/d28/db0/ld7 to d3/d7e/l134 0 2024-04-20T10:59:23.006 INFO:tasks.workunit.client.0.smithi027.stdout:6/717: mknod da/d21/dab/d19/ce6 0 2024-04-20T10:59:23.006 INFO:tasks.workunit.client.0.smithi027.stdout:4/705: dread d0/d14/d6b/da3/d1b/f9c [0,4194304] 0 2024-04-20T10:59:23.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/691: creat d2/dd/df/d23/fdb x:0 0 0 2024-04-20T10:59:23.007 INFO:tasks.workunit.client.0.smithi027.stdout:1/866: mkdir d3/d7/d38/d135 0 2024-04-20T10:59:23.007 INFO:tasks.workunit.client.0.smithi027.stdout:4/706: symlink d0/d14/d6b/d92/l100 0 2024-04-20T10:59:23.008 INFO:tasks.workunit.client.0.smithi027.stdout:6/718: dread da/f84 [0,4194304] 0 2024-04-20T10:59:23.008 INFO:tasks.workunit.client.0.smithi027.stdout:3/692: creat d2/d18/d2c/d4c/d67/d6c/fdc x:0 0 0 2024-04-20T10:59:23.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/693: write d2/dd/df/d23/d68/fbb [408441,35508] 0 2024-04-20T10:59:23.011 INFO:tasks.workunit.client.0.smithi027.stdout:2/625: dread d5/f4b [0,4194304] 0 2024-04-20T10:59:23.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/719: rename c9 to da/d21/d28/d2d/ce7 0 2024-04-20T10:59:23.011 INFO:tasks.workunit.client.0.smithi027.stdout:1/867: dread d3/d7/d38/ddf/f4a [0,4194304] 0 2024-04-20T10:59:23.011 INFO:tasks.workunit.client.0.smithi027.stdout:4/707: mknod d0/d14/c101 0 2024-04-20T10:59:23.013 INFO:tasks.workunit.client.0.smithi027.stdout:3/694: unlink d2/d18/d2e/fc3 0 2024-04-20T10:59:23.014 INFO:tasks.workunit.client.0.smithi027.stdout:2/626: rmdir d5/d26/d30/db4 0 2024-04-20T10:59:23.015 INFO:tasks.workunit.client.0.smithi027.stdout:6/720: mkdir da/d21/dab/d19/d2e/d6b/d65/de8 0 2024-04-20T10:59:23.016 INFO:tasks.workunit.client.0.smithi027.stdout:1/868: symlink d3/d7/d28/d92/dad/l136 0 2024-04-20T10:59:23.017 INFO:tasks.workunit.client.0.smithi027.stdout:4/708: truncate d0/d14/d6b/da3/d1b/f61 1328132 0 2024-04-20T10:59:23.017 INFO:tasks.workunit.client.0.smithi027.stdout:4/709: fsync d0/d14/d6b/da3/d1b/d33/f9a 0 2024-04-20T10:59:23.018 INFO:tasks.workunit.client.0.smithi027.stdout:3/695: symlink d2/d18/d2c/ldd 0 2024-04-20T10:59:23.018 INFO:tasks.workunit.client.0.smithi027.stdout:2/627: truncate d5/f17 3902841 0 2024-04-20T10:59:23.019 INFO:tasks.workunit.client.0.smithi027.stdout:2/628: write d5/dc/d16/d60/d8e/fb5 [695347,52238] 0 2024-04-20T10:59:23.019 INFO:tasks.workunit.client.0.smithi027.stdout:2/629: readlink d5/dc/d16/d32/l92 0 2024-04-20T10:59:23.019 INFO:tasks.workunit.client.0.smithi027.stdout:6/721: mknod da/dd/d3d/ce9 0 2024-04-20T10:59:23.020 INFO:tasks.workunit.client.0.smithi027.stdout:1/869: mkdir d3/d7/d38/db8/d137 0 2024-04-20T10:59:23.020 INFO:tasks.workunit.client.0.smithi027.stdout:4/710: chown d0/d14/d6b/da3/d1a/l24 8396 1 2024-04-20T10:59:23.021 INFO:tasks.workunit.client.0.smithi027.stdout:3/696: symlink d2/d94/dc4/lde 0 2024-04-20T10:59:23.021 INFO:tasks.workunit.client.0.smithi027.stdout:6/722: rmdir da/d21/dab 39 2024-04-20T10:59:23.022 INFO:tasks.workunit.client.0.smithi027.stdout:1/870: creat d3/d7/d28/db0/f138 x:0 0 0 2024-04-20T10:59:23.022 INFO:tasks.workunit.client.0.smithi027.stdout:1/871: dread - d3/d45/f116 zero size 2024-04-20T10:59:23.023 INFO:tasks.workunit.client.0.smithi027.stdout:4/711: link d0/d14/d6b/da3/d18/db9/fc9 d0/d14/d6b/da3/d1b/d33/d5c/d6f/dd1/f102 0 2024-04-20T10:59:23.024 INFO:tasks.workunit.client.0.smithi027.stdout:4/712: chown d0/d14/d6b/da3/d53/d6a/faa 3332 1 2024-04-20T10:59:23.024 INFO:tasks.workunit.client.0.smithi027.stdout:3/697: creat d2/d80/fdf x:0 0 0 2024-04-20T10:59:23.025 INFO:tasks.workunit.client.0.smithi027.stdout:1/872: creat d3/d45/f139 x:0 0 0 2024-04-20T10:59:23.027 INFO:tasks.workunit.client.0.smithi027.stdout:4/713: rename d0/d14/d6b/da3/d18/l81 to d0/d48/d68/d76/l103 0 2024-04-20T10:59:23.029 INFO:tasks.workunit.client.0.smithi027.stdout:3/698: dread d2/dd/f1a [0,4194304] 0 2024-04-20T10:59:23.030 INFO:tasks.workunit.client.0.smithi027.stdout:3/699: getdents d2/dd/d22 0 2024-04-20T10:59:23.031 INFO:tasks.workunit.client.0.smithi027.stdout:3/700: rmdir d2/dd/df 39 2024-04-20T10:59:23.031 INFO:tasks.workunit.client.0.smithi027.stdout:6/723: dread da/d21/dab/d19/f6a [0,4194304] 0 2024-04-20T10:59:23.031 INFO:tasks.workunit.client.0.smithi027.stdout:6/724: fsync da/d21/fd9 0 2024-04-20T10:59:23.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/701: creat d2/d18/d2c/d4c/d67/fe0 x:0 0 0 2024-04-20T10:59:23.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/702: truncate d2/dd/d22/fbf 107745 0 2024-04-20T10:59:23.033 INFO:tasks.workunit.client.0.smithi027.stdout:3/703: mkdir d2/dd/d57/d8d/dc6/de1 0 2024-04-20T10:59:23.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/704: mkdir d2/dd/d22/de2 0 2024-04-20T10:59:23.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/705: symlink d2/dd/d57/le3 0 2024-04-20T10:59:23.036 INFO:tasks.workunit.client.0.smithi027.stdout:3/706: unlink d2/d18/f5b 0 2024-04-20T10:59:23.043 INFO:tasks.workunit.client.0.smithi027.stdout:6/725: write da/dd/f69 [1031690,109176] 0 2024-04-20T10:59:23.043 INFO:tasks.workunit.client.0.smithi027.stdout:2/630: dwrite d5/d1e/f70 [0,4194304] 0 2024-04-20T10:59:23.044 INFO:tasks.workunit.client.0.smithi027.stdout:6/726: creat da/d21/dab/d19/fea x:0 0 0 2024-04-20T10:59:23.045 INFO:tasks.workunit.client.0.smithi027.stdout:2/631: truncate d5/d26/d30/f3c 2514548 0 2024-04-20T10:59:23.048 INFO:tasks.workunit.client.0.smithi027.stdout:4/714: dwrite d0/d14/d6b/da3/d18/f38 [0,4194304] 0 2024-04-20T10:59:23.048 INFO:tasks.workunit.client.0.smithi027.stdout:8/719: sync 2024-04-20T10:59:23.048 INFO:tasks.workunit.client.0.smithi027.stdout:9/592: sync 2024-04-20T10:59:23.049 INFO:tasks.workunit.client.0.smithi027.stdout:5/649: sync 2024-04-20T10:59:23.049 INFO:tasks.workunit.client.0.smithi027.stdout:7/680: sync 2024-04-20T10:59:23.049 INFO:tasks.workunit.client.0.smithi027.stdout:0/822: sync 2024-04-20T10:59:23.050 INFO:tasks.workunit.client.0.smithi027.stdout:8/720: rmdir d9/de/d1e/d4a/d6d/dea/d59/de6 0 2024-04-20T10:59:23.050 INFO:tasks.workunit.client.0.smithi027.stdout:9/593: getdents d4/d74 0 2024-04-20T10:59:23.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/594: fdatasync d4/d1b/d31/d23/f47 0 2024-04-20T10:59:23.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/595: write d4/d1b/f22 [1002788,113584] 0 2024-04-20T10:59:23.051 INFO:tasks.workunit.client.0.smithi027.stdout:6/727: dread da/d21/dab/d19/f6a [0,4194304] 0 2024-04-20T10:59:23.051 INFO:tasks.workunit.client.0.smithi027.stdout:8/721: symlink d9/de/d1e/d49/lf5 0 2024-04-20T10:59:23.052 INFO:tasks.workunit.client.0.smithi027.stdout:2/632: dread f1 [0,4194304] 0 2024-04-20T10:59:23.052 INFO:tasks.workunit.client.0.smithi027.stdout:2/633: chown d5/dc/d16/d60/dac/f67 98989886 1 2024-04-20T10:59:23.053 INFO:tasks.workunit.client.0.smithi027.stdout:8/722: rmdir d9/de/d1e/d4a/d6d/dea/d59/da0 39 2024-04-20T10:59:23.054 INFO:tasks.workunit.client.0.smithi027.stdout:2/634: creat d5/dc/d16/d60/d8e/fc6 x:0 0 0 2024-04-20T10:59:23.054 INFO:tasks.workunit.client.0.smithi027.stdout:3/707: dwrite d2/dd/fc0 [0,4194304] 0 2024-04-20T10:59:23.055 INFO:tasks.workunit.client.0.smithi027.stdout:3/708: read - d2/fad zero size 2024-04-20T10:59:23.056 INFO:tasks.workunit.client.0.smithi027.stdout:8/723: unlink d9/de/d1e/d4a/d6d/db7/fe3 0 2024-04-20T10:59:23.056 INFO:tasks.workunit.client.0.smithi027.stdout:8/724: fsync d9/de/d20/dbf/fc3 0 2024-04-20T10:59:23.057 INFO:tasks.workunit.client.0.smithi027.stdout:2/635: symlink d5/d26/lc7 0 2024-04-20T10:59:23.058 INFO:tasks.workunit.client.0.smithi027.stdout:0/823: dread d0/d11/d2c/d2e/d2b/d7b/d34/f3b [0,4194304] 0 2024-04-20T10:59:23.060 INFO:tasks.workunit.client.0.smithi027.stdout:4/715: dread d0/d14/d6b/da3/d18/dbf/fef [0,4194304] 0 2024-04-20T10:59:23.061 INFO:tasks.workunit.client.0.smithi027.stdout:4/716: readlink d0/d14/d6b/da3/d1b/d33/d5c/d6f/l8c 0 2024-04-20T10:59:23.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/681: dread d4/d1b/d58/f5f [0,4194304] 0 2024-04-20T10:59:23.061 INFO:tasks.workunit.client.0.smithi027.stdout:3/709: unlink d2/dd/f77 0 2024-04-20T10:59:23.062 INFO:tasks.workunit.client.0.smithi027.stdout:8/725: symlink d9/de/da3/lf6 0 2024-04-20T10:59:23.064 INFO:tasks.workunit.client.0.smithi027.stdout:2/636: rmdir d5/dc/d16/d2a 39 2024-04-20T10:59:23.065 INFO:tasks.workunit.client.0.smithi027.stdout:0/824: unlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/l11d 0 2024-04-20T10:59:23.067 INFO:tasks.workunit.client.0.smithi027.stdout:4/717: rename d0/d14/d6b/da3/d1b/d33/fe7 to d0/d14/d6b/da3/d1b/d33/d5c/d6f/d7b/f104 0 2024-04-20T10:59:23.068 INFO:tasks.workunit.client.0.smithi027.stdout:7/682: truncate d4/d5/d8/f82 2674602 0 2024-04-20T10:59:23.068 INFO:tasks.workunit.client.0.smithi027.stdout:7/683: fsync d4/d1b/d4a/d62/f9f 0 2024-04-20T10:59:23.068 INFO:tasks.workunit.client.0.smithi027.stdout:7/684: stat d4/d5/d8/f82 0 2024-04-20T10:59:23.069 INFO:tasks.workunit.client.0.smithi027.stdout:7/685: truncate d4/d5/d6f/fc6 12010 0 2024-04-20T10:59:23.069 INFO:tasks.workunit.client.0.smithi027.stdout:0/825: creat d0/d11/d2c/d2e/d2b/d7b/d12/d25/f11e x:0 0 0 2024-04-20T10:59:23.070 INFO:tasks.workunit.client.0.smithi027.stdout:0/826: mkdir d0/d11/d4b/d5b/d6f/d82/dd1/d11f 0 2024-04-20T10:59:23.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/827: dread - d0/d11/d2c/d2e/d2b/d7b/d34/f43 zero size 2024-04-20T10:59:23.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/828: fsync d0/d11/d2c/d2e/d2b/d7b/d34/f43 0 2024-04-20T10:59:23.071 INFO:tasks.workunit.client.0.smithi027.stdout:0/829: mknod d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/c120 0 2024-04-20T10:59:23.071 INFO:tasks.workunit.client.0.smithi027.stdout:6/728: write da/d21/dab/f4f [169606,102674] 0 2024-04-20T10:59:23.072 INFO:tasks.workunit.client.0.smithi027.stdout:8/726: dread d9/ff [0,4194304] 0 2024-04-20T10:59:23.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/830: truncate d0/d11/d2c/d2e/d55/f56 2568485 0 2024-04-20T10:59:23.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/831: read d0/f42 [571372,109503] 0 2024-04-20T10:59:23.075 INFO:tasks.workunit.client.0.smithi027.stdout:6/729: symlink da/d21/dab/d7e/daf/leb 0 2024-04-20T10:59:23.075 INFO:tasks.workunit.client.0.smithi027.stdout:6/730: chown da/d21/dab/d19/d2e/l5c 1 1 2024-04-20T10:59:23.075 INFO:tasks.workunit.client.0.smithi027.stdout:6/731: readlink da/d21/d28/d54/lb6 0 2024-04-20T10:59:23.076 INFO:tasks.workunit.client.0.smithi027.stdout:8/727: link f3 d9/de/d1e/d4a/d6d/db7/d5b/ff7 0 2024-04-20T10:59:23.078 INFO:tasks.workunit.client.0.smithi027.stdout:0/832: mknod d0/d11/d4b/d5b/d6f/d82/dd3/c121 0 2024-04-20T10:59:23.079 INFO:tasks.workunit.client.0.smithi027.stdout:5/650: dwrite d19/d1b/d1c/dbd/dbe/fbf [0,4194304] 0 2024-04-20T10:59:23.082 INFO:tasks.workunit.client.0.smithi027.stdout:0/833: mknod d0/d11/d2c/d2e/d2b/d106/c122 0 2024-04-20T10:59:23.085 INFO:tasks.workunit.client.0.smithi027.stdout:5/651: creat d19/d1b/d1c/d1e/d3c/fd2 x:0 0 0 2024-04-20T10:59:23.086 INFO:tasks.workunit.client.0.smithi027.stdout:6/732: truncate da/d21/dab/f11 891456 0 2024-04-20T10:59:23.087 INFO:tasks.workunit.client.0.smithi027.stdout:8/728: dread d9/de/d1e/d4a/f8f [0,4194304] 0 2024-04-20T10:59:23.089 INFO:tasks.workunit.client.0.smithi027.stdout:6/733: dread da/d21/dab/d19/d2e/f41 [0,4194304] 0 2024-04-20T10:59:23.090 INFO:tasks.workunit.client.0.smithi027.stdout:5/652: link d19/c85 d19/d1b/d1c/dbd/dbe/cd3 0 2024-04-20T10:59:23.091 INFO:tasks.workunit.client.0.smithi027.stdout:5/653: dread d19/d1b/d1c/d1e/d3c/fcd [0,4194304] 0 2024-04-20T10:59:23.091 INFO:tasks.workunit.client.0.smithi027.stdout:8/729: truncate d9/de/da3/d3c/f9c 1816274 0 2024-04-20T10:59:23.093 INFO:tasks.workunit.client.0.smithi027.stdout:6/734: symlink da/d21/dab/d7e/d86/dcd/lec 0 2024-04-20T10:59:23.093 INFO:tasks.workunit.client.0.smithi027.stdout:6/735: stat da/d21/dab/d19/d2e/d32/f44 0 2024-04-20T10:59:23.093 INFO:tasks.workunit.client.0.smithi027.stdout:6/736: write da/dd/d79/fb7 [879039,59047] 0 2024-04-20T10:59:23.095 INFO:tasks.workunit.client.0.smithi027.stdout:5/654: mknod d19/d1b/d1c/d1e/d58/d9d/cd4 0 2024-04-20T10:59:23.095 INFO:tasks.workunit.client.0.smithi027.stdout:5/655: write d19/d1b/d1c/d1e/d58/d9d/fce [872583,84354] 0 2024-04-20T10:59:23.095 INFO:tasks.workunit.client.0.smithi027.stdout:8/730: truncate d9/de/f1d 756073 0 2024-04-20T10:59:23.096 INFO:tasks.workunit.client.0.smithi027.stdout:8/731: write d9/f54 [800496,8451] 0 2024-04-20T10:59:23.096 INFO:tasks.workunit.client.0.smithi027.stdout:8/732: readlink l8 0 2024-04-20T10:59:23.096 INFO:tasks.workunit.client.0.smithi027.stdout:6/737: mknod da/d21/dab/d7e/d86/da3/ced 0 2024-04-20T10:59:23.097 INFO:tasks.workunit.client.0.smithi027.stdout:7/686: write d4/d1b/fb0 [3366121,3149] 0 2024-04-20T10:59:23.098 INFO:tasks.workunit.client.0.smithi027.stdout:6/738: dread da/f84 [0,4194304] 0 2024-04-20T10:59:23.098 INFO:tasks.workunit.client.0.smithi027.stdout:2/637: dwrite d5/d26/f3a [0,4194304] 0 2024-04-20T10:59:23.099 INFO:tasks.workunit.client.0.smithi027.stdout:5/656: getdents d19/d1b/d1c/d1e/d25/d40 0 2024-04-20T10:59:23.100 INFO:tasks.workunit.client.0.smithi027.stdout:8/733: creat d9/de/d1e/d4a/d6d/ff8 x:0 0 0 2024-04-20T10:59:23.100 INFO:tasks.workunit.client.0.smithi027.stdout:8/734: chown d9/de/da3/lf6 1 1 2024-04-20T10:59:23.100 INFO:tasks.workunit.client.0.smithi027.stdout:8/735: chown d9/de/d1e/dc1 231982 1 2024-04-20T10:59:23.101 INFO:tasks.workunit.client.0.smithi027.stdout:7/687: truncate d4/d1b/f53 1090051 0 2024-04-20T10:59:23.102 INFO:tasks.workunit.client.0.smithi027.stdout:6/739: symlink da/d21/d28/d2d/d53/lee 0 2024-04-20T10:59:23.104 INFO:tasks.workunit.client.0.smithi027.stdout:5/657: truncate d19/d1b/f86 228910 0 2024-04-20T10:59:23.104 INFO:tasks.workunit.client.0.smithi027.stdout:5/658: fdatasync d19/d1b/d1c/fad 0 2024-04-20T10:59:23.105 INFO:tasks.workunit.client.0.smithi027.stdout:8/736: creat d9/de/d1e/d4a/d6d/dea/d62/ff9 x:0 0 0 2024-04-20T10:59:23.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/737: chown d9/de/da3/l58 306 1 2024-04-20T10:59:23.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/738: dread d9/de/d1e/d4a/d6d/dea/d59/d74/fb9 [0,4194304] 0 2024-04-20T10:59:23.107 INFO:tasks.workunit.client.0.smithi027.stdout:8/739: stat d9/de/d1e/d4a/d6d/d79/fe4 0 2024-04-20T10:59:23.109 INFO:tasks.workunit.client.0.smithi027.stdout:6/740: creat da/d21/d28/d2d/d53/dae/fef x:0 0 0 2024-04-20T10:59:23.111 INFO:tasks.workunit.client.0.smithi027.stdout:5/659: creat d19/d1b/d1c/d1e/d25/d40/fd5 x:0 0 0 2024-04-20T10:59:23.118 INFO:tasks.workunit.client.0.smithi027.stdout:6/741: rename da/d21/d28/d2d/d53/d9a/lc9 to da/d21/dab/d19/d2e/d6b/de0/lf0 0 2024-04-20T10:59:23.124 INFO:tasks.workunit.client.0.smithi027.stdout:6/742: write da/d21/dab/d7e/d86/fb0 [333408,77907] 0 2024-04-20T10:59:23.132 INFO:tasks.workunit.client.0.smithi027.stdout:6/743: mkdir da/d21/dab/d19/d2e/df1 0 2024-04-20T10:59:23.133 INFO:tasks.workunit.client.0.smithi027.stdout:2/638: dwrite d5/f17 [4194304,4194304] 0 2024-04-20T10:59:23.134 INFO:tasks.workunit.client.0.smithi027.stdout:7/688: dwrite d4/d5b/fae [0,4194304] 0 2024-04-20T10:59:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:7/689: chown d4/d5b 109304576 1 2024-04-20T10:59:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:7/690: truncate d4/d1b/d4a/f9c 438225 0 2024-04-20T10:59:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:6/744: symlink da/d21/dab/d19/d2e/df1/lf2 0 2024-04-20T10:59:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/740: dwrite d9/de/faf [4194304,4194304] 0 2024-04-20T10:59:23.135 INFO:tasks.workunit.client.0.smithi027.stdout:6/745: dread da/d21/dab/f4d [0,4194304] 0 2024-04-20T10:59:23.140 INFO:tasks.workunit.client.0.smithi027.stdout:7/691: mknod d4/d5/d8/d46/d77/d7c/cd7 0 2024-04-20T10:59:23.140 INFO:tasks.workunit.client.0.smithi027.stdout:7/692: write d4/d5/d8/fac [1037879,41538] 0 2024-04-20T10:59:23.140 INFO:tasks.workunit.client.0.smithi027.stdout:7/693: write d4/d1b/d58/fd2 [346783,53710] 0 2024-04-20T10:59:23.145 INFO:tasks.workunit.client.0.smithi027.stdout:8/741: dread d9/f34 [0,4194304] 0 2024-04-20T10:59:23.146 INFO:tasks.workunit.client.0.smithi027.stdout:8/742: chown d9/de/d1e/d4a/d6d/dea/d59/da0/dbd/ff3 264 1 2024-04-20T10:59:23.148 INFO:tasks.workunit.client.0.smithi027.stdout:7/694: symlink d4/d1b/d4a/d7a/ld8 0 2024-04-20T10:59:23.148 INFO:tasks.workunit.client.0.smithi027.stdout:7/695: fsync d4/d5/f14 0 2024-04-20T10:59:23.148 INFO:tasks.workunit.client.0.smithi027.stdout:8/743: getdents d9/de/d1e 0 2024-04-20T10:59:23.149 INFO:tasks.workunit.client.0.smithi027.stdout:7/696: creat d4/d1b/d4a/d62/dcd/dd5/fd9 x:0 0 0 2024-04-20T10:59:23.153 INFO:tasks.workunit.client.0.smithi027.stdout:2/639: dwrite d5/f4b [0,4194304] 0 2024-04-20T10:59:23.153 INFO:tasks.workunit.client.0.smithi027.stdout:2/640: write d5/dc/d16/d37/f6b [2415771,54342] 0 2024-04-20T10:59:23.153 INFO:tasks.workunit.client.0.smithi027.stdout:7/697: mknod d4/d5/d8/cda 0 2024-04-20T10:59:23.159 INFO:tasks.workunit.client.0.smithi027.stdout:6/746: dwrite da/d21/d28/fbb [0,4194304] 0 2024-04-20T10:59:23.159 INFO:tasks.workunit.client.0.smithi027.stdout:2/641: creat d5/dc/d16/d37/fc8 x:0 0 0 2024-04-20T10:59:23.161 INFO:tasks.workunit.client.0.smithi027.stdout:6/747: dread da/d21/dab/d19/f5d [0,4194304] 0 2024-04-20T10:59:23.165 INFO:tasks.workunit.client.0.smithi027.stdout:7/698: write d4/d5/f14 [3906972,41558] 0 2024-04-20T10:59:23.170 INFO:tasks.workunit.client.0.smithi027.stdout:6/748: dread da/d21/dab/d19/f93 [0,4194304] 0 2024-04-20T10:59:23.170 INFO:tasks.workunit.client.0.smithi027.stdout:6/749: write da/d21/d28/f3a [4719516,114091] 0 2024-04-20T10:59:23.171 INFO:tasks.workunit.client.0.smithi027.stdout:6/750: symlink da/dd/d3d/db9/lf3 0 2024-04-20T10:59:23.176 INFO:tasks.workunit.client.0.smithi027.stdout:6/751: mknod da/d21/dab/d7e/d86/da3/cf4 0 2024-04-20T10:59:23.178 INFO:tasks.workunit.client.0.smithi027.stdout:6/752: truncate da/d21/dab/d19/f93 1696334 0 2024-04-20T10:59:23.178 INFO:tasks.workunit.client.0.smithi027.stdout:6/753: write da/d21/f5b [4286902,30846] 0 2024-04-20T10:59:23.178 INFO:tasks.workunit.client.0.smithi027.stdout:6/754: chown da/d21/dab/d19/d2e/d6b/d65/fbc 182 1 2024-04-20T10:59:23.179 INFO:tasks.workunit.client.0.smithi027.stdout:6/755: write da/d21/dab/f11 [70698,34258] 0 2024-04-20T10:59:23.179 INFO:tasks.workunit.client.0.smithi027.stdout:2/642: dwrite d5/d1e/f70 [0,4194304] 0 2024-04-20T10:59:23.197 INFO:tasks.workunit.client.0.smithi027.stdout:1/873: sync 2024-04-20T10:59:23.198 INFO:tasks.workunit.client.0.smithi027.stdout:2/643: creat d5/dc/d16/d60/d8e/fc9 x:0 0 0 2024-04-20T10:59:23.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/874: mkdir d3/d7/d38/da4/dac/d13a 0 2024-04-20T10:59:23.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/875: write d3/d7/ffd [511947,74048] 0 2024-04-20T10:59:23.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/876: write d3/d7/d38/f78 [1087407,6007] 0 2024-04-20T10:59:23.200 INFO:tasks.workunit.client.0.smithi027.stdout:6/756: dwrite da/d21/d28/d2d/d53/dae/fef [0,4194304] 0 2024-04-20T10:59:23.200 INFO:tasks.workunit.client.0.smithi027.stdout:7/699: dwrite d4/d5/d8/d21/f48 [0,4194304] 0 2024-04-20T10:59:23.200 INFO:tasks.workunit.client.0.smithi027.stdout:2/644: mkdir d5/d44/d7c/dca 0 2024-04-20T10:59:23.203 INFO:tasks.workunit.client.0.smithi027.stdout:1/877: creat d3/d7e/d123/f13b x:0 0 0 2024-04-20T10:59:23.203 INFO:tasks.workunit.client.0.smithi027.stdout:6/757: mknod da/d21/dab/d7e/d86/dcd/cf5 0 2024-04-20T10:59:23.206 INFO:tasks.workunit.client.0.smithi027.stdout:7/700: mknod d4/d5/d8/d21/d94/da7/dce/cdb 0 2024-04-20T10:59:23.206 INFO:tasks.workunit.client.0.smithi027.stdout:7/701: fsync d4/d5/d8/d21/d94/fa8 0 2024-04-20T10:59:23.207 INFO:tasks.workunit.client.0.smithi027.stdout:2/645: creat d5/dbc/fcb x:0 0 0 2024-04-20T10:59:23.207 INFO:tasks.workunit.client.0.smithi027.stdout:2/646: chown d5/dc/d16/d60/dac/c4c 10771365 1 2024-04-20T10:59:23.207 INFO:tasks.workunit.client.0.smithi027.stdout:1/878: creat d3/d45/dba/f13c x:0 0 0 2024-04-20T10:59:23.207 INFO:tasks.workunit.client.0.smithi027.stdout:1/879: dread - d3/d45/fed zero size 2024-04-20T10:59:23.208 INFO:tasks.workunit.client.0.smithi027.stdout:6/758: getdents da/d21/d28/d2d/d53/dae 0 2024-04-20T10:59:23.209 INFO:tasks.workunit.client.0.smithi027.stdout:7/702: creat d4/d5/d8/d46/d77/fdc x:0 0 0 2024-04-20T10:59:23.209 INFO:tasks.workunit.client.0.smithi027.stdout:2/647: rmdir d5 39 2024-04-20T10:59:23.209 INFO:tasks.workunit.client.0.smithi027.stdout:2/648: dread - d5/dbc/fcb zero size 2024-04-20T10:59:23.209 INFO:tasks.workunit.client.0.smithi027.stdout:2/649: dread - d5/dc/d16/d60/d8e/fc9 zero size 2024-04-20T10:59:23.209 INFO:tasks.workunit.client.0.smithi027.stdout:1/880: stat d3/d20/c5b 0 2024-04-20T10:59:23.210 INFO:tasks.workunit.client.0.smithi027.stdout:6/759: creat da/d21/dab/d7e/d86/ff6 x:0 0 0 2024-04-20T10:59:23.212 INFO:tasks.workunit.client.0.smithi027.stdout:7/703: creat d4/d1b/d4a/d62/fdd x:0 0 0 2024-04-20T10:59:23.213 INFO:tasks.workunit.client.0.smithi027.stdout:7/704: fsync d4/d1b/d4a/d7a/fcb 0 2024-04-20T10:59:23.213 INFO:tasks.workunit.client.0.smithi027.stdout:7/705: dread - d4/f98 zero size 2024-04-20T10:59:23.213 INFO:tasks.workunit.client.0.smithi027.stdout:8/744: write d9/de/f5e [639568,90050] 0 2024-04-20T10:59:23.213 INFO:tasks.workunit.client.0.smithi027.stdout:8/745: stat d9/de/d1e/f65 0 2024-04-20T10:59:23.213 INFO:tasks.workunit.client.0.smithi027.stdout:2/650: rename d5/d44/da7/da0 to d5/d44/da7/dcc 0 2024-04-20T10:59:23.214 INFO:tasks.workunit.client.0.smithi027.stdout:1/881: getdents d3/d7e/d123 0 2024-04-20T10:59:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:7/706: dread d4/d5/d8/d21/f48 [0,4194304] 0 2024-04-20T10:59:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:9/596: sync 2024-04-20T10:59:23.223 INFO:tasks.workunit.client.0.smithi027.stdout:9/597: read d4/d1b/d1f/d45/f83 [313587,74029] 0 2024-04-20T10:59:23.225 INFO:tasks.workunit.client.0.smithi027.stdout:8/746: link d9/de/d1e/d4a/d6d/d7d/dd6/cd9 d9/de/d1e/ddc/cfa 0 2024-04-20T10:59:23.225 INFO:tasks.workunit.client.0.smithi027.stdout:6/760: read da/d21/d28/d2d/d53/f5e [523191,3266] 0 2024-04-20T10:59:23.225 INFO:tasks.workunit.client.0.smithi027.stdout:6/761: chown da/dd/d3d/f7f 19 1 2024-04-20T10:59:23.226 INFO:tasks.workunit.client.0.smithi027.stdout:6/762: dread da/d21/dab/f29 [0,4194304] 0 2024-04-20T10:59:23.227 INFO:tasks.workunit.client.0.smithi027.stdout:1/882: getdents d3/d7/d28/d92 0 2024-04-20T10:59:23.228 INFO:tasks.workunit.client.0.smithi027.stdout:9/598: truncate d4/d14/f44 2284438 0 2024-04-20T10:59:23.241 INFO:tasks.workunit.client.0.smithi027.stdout:9/599: dread d4/d1b/d1f/d45/f71 [0,4194304] 0 2024-04-20T10:59:23.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/763: unlink da/cda 0 2024-04-20T10:59:23.243 INFO:tasks.workunit.client.0.smithi027.stdout:1/883: link d3/d7/d28/l6b d3/d7/d28/d92/dad/d118/l13d 0 2024-04-20T10:59:23.244 INFO:tasks.workunit.client.0.smithi027.stdout:9/600: unlink d4/d1b/d1f/f7c 0 2024-04-20T10:59:23.244 INFO:tasks.workunit.client.0.smithi027.stdout:9/601: fdatasync d4/d14/f27 0 2024-04-20T10:59:23.244 INFO:tasks.workunit.client.0.smithi027.stdout:9/602: write d4/d1b/d31/f67 [1932647,41039] 0 2024-04-20T10:59:23.244 INFO:tasks.workunit.client.0.smithi027.stdout:2/651: dwrite d5/d26/f3a [0,4194304] 0 2024-04-20T10:59:23.245 INFO:tasks.workunit.client.0.smithi027.stdout:6/764: truncate da/dd/f59 3124341 0 2024-04-20T10:59:23.245 INFO:tasks.workunit.client.0.smithi027.stdout:8/747: dwrite d9/de/f71 [0,4194304] 0 2024-04-20T10:59:23.245 INFO:tasks.workunit.client.0.smithi027.stdout:8/748: write d9/de/d1e/d4a/d6d/f9b [161646,90116] 0 2024-04-20T10:59:23.246 INFO:tasks.workunit.client.0.smithi027.stdout:1/884: mknod d3/d7/d38/da4/dac/d13a/c13e 0 2024-04-20T10:59:23.247 INFO:tasks.workunit.client.0.smithi027.stdout:2/652: truncate d5/d44/da7/dcc/faa 3819752 0 2024-04-20T10:59:23.247 INFO:tasks.workunit.client.0.smithi027.stdout:6/765: rmdir da/db8 39 2024-04-20T10:59:23.249 INFO:tasks.workunit.client.0.smithi027.stdout:8/749: rmdir d9/de/d20 39 2024-04-20T10:59:23.249 INFO:tasks.workunit.client.0.smithi027.stdout:8/750: chown d9/de/d1e/d4a/d6d/d79/f92 0 1 2024-04-20T10:59:23.251 INFO:tasks.workunit.client.0.smithi027.stdout:1/885: link d3/d7/d38/ddf/d73/f120 d3/d41/dc6/f13f 0 2024-04-20T10:59:23.254 INFO:tasks.workunit.client.0.smithi027.stdout:2/653: rename d5/dc/d16/d60/d8e/fc9 to d5/d44/da7/dcc/fcd 0 2024-04-20T10:59:23.254 INFO:tasks.workunit.client.0.smithi027.stdout:2/654: stat d5/d44/da7/dcc/fc2 0 2024-04-20T10:59:23.260 INFO:tasks.workunit.client.0.smithi027.stdout:2/655: write d5/dc/d16/d60/dac/f94 [2683443,51213] 0 2024-04-20T10:59:23.262 INFO:tasks.workunit.client.0.smithi027.stdout:6/766: link da/dd/d3c/d9e/ca1 da/d21/dab/d19/d2e/d6b/d65/cf7 0 2024-04-20T10:59:23.262 INFO:tasks.workunit.client.0.smithi027.stdout:6/767: chown da/d21/dab/fb5 4377 1 2024-04-20T10:59:23.262 INFO:tasks.workunit.client.0.smithi027.stdout:6/768: truncate da/d21/d28/d2d/d53/dae/fef 4356318 0 2024-04-20T10:59:23.263 INFO:tasks.workunit.client.0.smithi027.stdout:8/751: dread d9/de/da3/d3c/fd3 [0,4194304] 0 2024-04-20T10:59:23.263 INFO:tasks.workunit.client.0.smithi027.stdout:8/752: stat d9/de/d1e/d49/f5c 0 2024-04-20T10:59:23.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/886: unlink d3/d7/d28/db0/ce2 0 2024-04-20T10:59:23.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/887: chown d3/d7/ff 0 1 2024-04-20T10:59:23.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/888: read d3/d7/fc [2873492,64122] 0 2024-04-20T10:59:23.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/889: truncate d3/d45/fe7 2854822 0 2024-04-20T10:59:23.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/890: chown d3/d7/d38/d62 78122713 1 2024-04-20T10:59:23.279 INFO:tasks.workunit.client.0.smithi027.stdout:8/753: dwrite f2 [0,4194304] 0 2024-04-20T10:59:23.279 INFO:tasks.workunit.client.0.smithi027.stdout:8/754: fdatasync d9/de/da3/d3c/f6f 0 2024-04-20T10:59:23.280 INFO:tasks.workunit.client.0.smithi027.stdout:2/656: mknod d5/d44/da7/cce 0 2024-04-20T10:59:23.280 INFO:tasks.workunit.client.0.smithi027.stdout:2/657: chown d5/dc/d16/d60/dac/d8f 30837 1 2024-04-20T10:59:23.281 INFO:tasks.workunit.client.0.smithi027.stdout:6/769: rename da/dd/d79/l82 to da/d21/d28/d2d/d53/dae/ddf/lf8 0 2024-04-20T10:59:23.281 INFO:tasks.workunit.client.0.smithi027.stdout:6/770: fsync da/d21/dab/d19/d2e/f38 0 2024-04-20T10:59:23.281 INFO:tasks.workunit.client.0.smithi027.stdout:6/771: dread - da/d21/dab/d19/d2e/d6b/d65/fbc zero size 2024-04-20T10:59:23.281 INFO:tasks.workunit.client.0.smithi027.stdout:6/772: stat da/d21/dab/d19 0 2024-04-20T10:59:23.283 INFO:tasks.workunit.client.0.smithi027.stdout:8/755: symlink d9/de/d1e/d4a/d6d/dea/d59/d74/de8/lfb 0 2024-04-20T10:59:23.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/756: fsync d9/de/d1e/d4a/fde 0 2024-04-20T10:59:23.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/757: chown d9 117946 1 2024-04-20T10:59:23.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/758: fdatasync d9/de/da3/d3c/f6f 0 2024-04-20T10:59:23.284 INFO:tasks.workunit.client.0.smithi027.stdout:2/658: unlink d5/dc/d16/d60/f69 0 2024-04-20T10:59:23.285 INFO:tasks.workunit.client.0.smithi027.stdout:1/891: rename d3/d41/dc6/de6 to d3/d7/d38/da4/d140 0 2024-04-20T10:59:23.286 INFO:tasks.workunit.client.0.smithi027.stdout:6/773: unlink f3 0 2024-04-20T10:59:23.288 INFO:tasks.workunit.client.0.smithi027.stdout:8/759: creat d9/de/d1e/d4a/d6d/dea/d59/ffc x:0 0 0 2024-04-20T10:59:23.288 INFO:tasks.workunit.client.0.smithi027.stdout:2/659: mknod d5/dc/d16/d60/d8e/ccf 0 2024-04-20T10:59:23.290 INFO:tasks.workunit.client.0.smithi027.stdout:4/718: sync 2024-04-20T10:59:23.290 INFO:tasks.workunit.client.0.smithi027.stdout:3/710: sync 2024-04-20T10:59:23.290 INFO:tasks.workunit.client.0.smithi027.stdout:5/660: sync 2024-04-20T10:59:23.290 INFO:tasks.workunit.client.0.smithi027.stdout:0/834: sync 2024-04-20T10:59:23.291 INFO:tasks.workunit.client.0.smithi027.stdout:5/661: read - d19/d1b/d1c/d1e/f79 zero size 2024-04-20T10:59:23.291 INFO:tasks.workunit.client.0.smithi027.stdout:5/662: fdatasync d19/d1b/d1c/d1e/d3c/f24 0 2024-04-20T10:59:23.291 INFO:tasks.workunit.client.0.smithi027.stdout:8/760: dread d9/de/f1d [0,4194304] 0 2024-04-20T10:59:23.291 INFO:tasks.workunit.client.0.smithi027.stdout:2/660: dread d5/dc/d16/d60/dac/f67 [0,4194304] 0 2024-04-20T10:59:23.291 INFO:tasks.workunit.client.0.smithi027.stdout:1/892: rename d3/d7/d38/da4/cb2 to d3/d7/d38/da4/dac/de3/c141 0 2024-04-20T10:59:23.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/774: creat da/dd/ff9 x:0 0 0 2024-04-20T10:59:23.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/775: chown da/d21/dab/d19/d2e/d6b/c8e 3 1 2024-04-20T10:59:23.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/776: write da/dd/d79/fb7 [1734433,88248] 0 2024-04-20T10:59:23.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/777: chown da/d21/dab/f17 29334 1 2024-04-20T10:59:23.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/778: chown da/db8 2216703 1 2024-04-20T10:59:23.293 INFO:tasks.workunit.client.0.smithi027.stdout:3/711: getdents d2/dd/df 0 2024-04-20T10:59:23.293 INFO:tasks.workunit.client.0.smithi027.stdout:4/719: mknod d0/d14/d6b/da3/d1b/d2f/c105 0 2024-04-20T10:59:23.294 INFO:tasks.workunit.client.0.smithi027.stdout:4/720: write d0/d48/ff5 [937011,73945] 0 2024-04-20T10:59:23.294 INFO:tasks.workunit.client.0.smithi027.stdout:4/721: fdatasync d0/d48/f9d 0 2024-04-20T10:59:23.294 INFO:tasks.workunit.client.0.smithi027.stdout:5/663: creat d19/d1b/d1c/d1e/d22/fd6 x:0 0 0 2024-04-20T10:59:23.295 INFO:tasks.workunit.client.0.smithi027.stdout:8/761: mkdir d9/de/d20/dfd 0 2024-04-20T10:59:23.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/661: mkdir d5/dc/d16/dd0 0 2024-04-20T10:59:23.296 INFO:tasks.workunit.client.0.smithi027.stdout:6/779: creat da/ffa x:0 0 0 2024-04-20T10:59:23.297 INFO:tasks.workunit.client.0.smithi027.stdout:0/835: write d0/d11/d2c/d2e/d2b/d7b/d12/f3e [2573821,25214] 0 2024-04-20T10:59:23.299 INFO:tasks.workunit.client.0.smithi027.stdout:3/712: mknod d2/dd/d22/ce4 0 2024-04-20T10:59:23.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/722: unlink d0/d14/d6b/da3/d1b/d33/d5c/d6f/d7b/fe3 0 2024-04-20T10:59:23.300 INFO:tasks.workunit.client.0.smithi027.stdout:4/723: chown d0/d14/d6b/da3/d1b/d33/cb8 433816 1 2024-04-20T10:59:23.301 INFO:tasks.workunit.client.0.smithi027.stdout:5/664: mknod d19/d1b/d1c/dbd/cd7 0 2024-04-20T10:59:23.301 INFO:tasks.workunit.client.0.smithi027.stdout:5/665: readlink d19/d1b/d1c/d1e/d25/la6 0 2024-04-20T10:59:23.301 INFO:tasks.workunit.client.0.smithi027.stdout:5/666: chown d19/d1b/d1c/d1e/d22/d68/d75/l92 108257 1 2024-04-20T10:59:23.302 INFO:tasks.workunit.client.0.smithi027.stdout:8/762: mkdir d9/de/d64/dfe 0 2024-04-20T10:59:23.302 INFO:tasks.workunit.client.0.smithi027.stdout:9/603: dread d4/d14/f49 [0,4194304] 0 2024-04-20T10:59:23.302 INFO:tasks.workunit.client.0.smithi027.stdout:3/713: rmdir d2/d7f 39 2024-04-20T10:59:23.302 INFO:tasks.workunit.client.0.smithi027.stdout:5/667: dread d19/d1b/d1c/d1e/d3c/f96 [0,4194304] 0 2024-04-20T10:59:23.302 INFO:tasks.workunit.client.0.smithi027.stdout:4/724: creat d0/d14/d6b/da3/d1b/d33/d5c/d6f/f106 x:0 0 0 2024-04-20T10:59:23.303 INFO:tasks.workunit.client.0.smithi027.stdout:4/725: write d0/d14/d6b/da3/f9e [64793,60377] 0 2024-04-20T10:59:23.303 INFO:tasks.workunit.client.0.smithi027.stdout:2/662: mknod d5/cd1 0 2024-04-20T10:59:23.305 INFO:tasks.workunit.client.0.smithi027.stdout:8/763: mkdir d9/de/d18/dff 0 2024-04-20T10:59:23.305 INFO:tasks.workunit.client.0.smithi027.stdout:3/714: symlink d2/d18/d2c/d4c/d67/le5 0 2024-04-20T10:59:23.306 INFO:tasks.workunit.client.0.smithi027.stdout:5/668: creat d19/d1b/d1c/d1e/d83/dcc/fd8 x:0 0 0 2024-04-20T10:59:23.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/604: creat d4/d1b/d1f/d5a/fb4 x:0 0 0 2024-04-20T10:59:23.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/605: chown d4/d14/d43/d91/f96 192 1 2024-04-20T10:59:23.310 INFO:tasks.workunit.client.0.smithi027.stdout:4/726: link d0/d14/d6b/da3/d18/c2a d0/d14/d6b/da3/d1b/dc3/dc5/c107 0 2024-04-20T10:59:23.310 INFO:tasks.workunit.client.0.smithi027.stdout:8/764: stat d9/de/d1e/d49/c5f 0 2024-04-20T10:59:23.312 INFO:tasks.workunit.client.0.smithi027.stdout:3/715: dread d2/f66 [0,4194304] 0 2024-04-20T10:59:23.312 INFO:tasks.workunit.client.0.smithi027.stdout:3/716: dread - d2/dd/f39 zero size 2024-04-20T10:59:23.312 INFO:tasks.workunit.client.0.smithi027.stdout:5/669: creat d19/d1b/d1c/dbd/dbe/fd9 x:0 0 0 2024-04-20T10:59:23.313 INFO:tasks.workunit.client.0.smithi027.stdout:8/765: mknod d9/de/d64/d6c/dc4/c100 0 2024-04-20T10:59:23.315 INFO:tasks.workunit.client.0.smithi027.stdout:9/606: rename d4/d1b/d31/d23/c4e to d4/d14/d43/cb5 0 2024-04-20T10:59:23.316 INFO:tasks.workunit.client.0.smithi027.stdout:9/607: chown d4/d1b/d1f/d45 46162 1 2024-04-20T10:59:23.316 INFO:tasks.workunit.client.0.smithi027.stdout:9/608: truncate d4/d14/d43/f86 1018410 0 2024-04-20T10:59:23.316 INFO:tasks.workunit.client.0.smithi027.stdout:0/836: dwrite d0/d11/d4b/fa3 [0,4194304] 0 2024-04-20T10:59:23.316 INFO:tasks.workunit.client.0.smithi027.stdout:1/893: dwrite d3/d7/d28/ff1 [0,4194304] 0 2024-04-20T10:59:23.317 INFO:tasks.workunit.client.0.smithi027.stdout:8/766: creat d9/de/d1e/d4a/d6d/dea/db3/f101 x:0 0 0 2024-04-20T10:59:23.318 INFO:tasks.workunit.client.0.smithi027.stdout:5/670: symlink d19/d1b/d1c/d1e/d3c/lda 0 2024-04-20T10:59:23.320 INFO:tasks.workunit.client.0.smithi027.stdout:0/837: symlink d0/d11/d2c/d2e/d2b/d7b/dcf/l123 0 2024-04-20T10:59:23.320 INFO:tasks.workunit.client.0.smithi027.stdout:0/838: fsync d0/d11/d2c/d2e/d2b/d7b/d64/fb4 0 2024-04-20T10:59:23.321 INFO:tasks.workunit.client.0.smithi027.stdout:0/839: chown d0/d59/d9e 2542 1 2024-04-20T10:59:23.322 INFO:tasks.workunit.client.0.smithi027.stdout:8/767: creat d9/de/d1e/d4a/d6d/db7/f102 x:0 0 0 2024-04-20T10:59:23.322 INFO:tasks.workunit.client.0.smithi027.stdout:8/768: fdatasync d9/de/d1e/d4a/d6d/d79/feb 0 2024-04-20T10:59:23.322 INFO:tasks.workunit.client.0.smithi027.stdout:8/769: chown d9/de/d64/d6c/f8c 516126 1 2024-04-20T10:59:23.323 INFO:tasks.workunit.client.0.smithi027.stdout:5/671: creat d19/d1b/d1c/dbd/fdb x:0 0 0 2024-04-20T10:59:23.328 INFO:tasks.workunit.client.0.smithi027.stdout:2/663: dwrite d5/d44/da7/f48 [0,4194304] 0 2024-04-20T10:59:23.328 INFO:tasks.workunit.client.0.smithi027.stdout:2/664: write d5/dc/f55 [976475,13579] 0 2024-04-20T10:59:23.330 INFO:tasks.workunit.client.0.smithi027.stdout:9/609: link d4/d1b/la4 d4/d2a/lb6 0 2024-04-20T10:59:23.330 INFO:tasks.workunit.client.0.smithi027.stdout:9/610: fsync d4/d1b/d31/f3c 0 2024-04-20T10:59:23.332 INFO:tasks.workunit.client.0.smithi027.stdout:3/717: dwrite d2/fc5 [0,4194304] 0 2024-04-20T10:59:23.332 INFO:tasks.workunit.client.0.smithi027.stdout:0/840: unlink d0/d11/d2c/d2e/d2b/d7b/d12/c4f 0 2024-04-20T10:59:23.332 INFO:tasks.workunit.client.0.smithi027.stdout:2/665: dread - d5/f6c zero size 2024-04-20T10:59:23.338 INFO:tasks.workunit.client.0.smithi027.stdout:9/611: mkdir d4/d1b/d1f/d5a/db7 0 2024-04-20T10:59:23.347 INFO:tasks.workunit.client.0.smithi027.stdout:7/707: sync 2024-04-20T10:59:23.348 INFO:tasks.workunit.client.0.smithi027.stdout:3/718: truncate d2/d7f/fa4 100823 0 2024-04-20T10:59:23.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/672: write d19/d1b/d1c/d1e/f56 [2529824,46420] 0 2024-04-20T10:59:23.351 INFO:tasks.workunit.client.0.smithi027.stdout:8/770: dwrite d9/de/d1e/d4a/f45 [0,4194304] 0 2024-04-20T10:59:23.351 INFO:tasks.workunit.client.0.smithi027.stdout:1/894: dwrite d3/d7/f1f [0,4194304] 0 2024-04-20T10:59:23.351 INFO:tasks.workunit.client.0.smithi027.stdout:1/895: chown d3/d20/dfb 103069644 1 2024-04-20T10:59:23.353 INFO:tasks.workunit.client.0.smithi027.stdout:2/666: creat d5/dbc/fd2 x:0 0 0 2024-04-20T10:59:23.356 INFO:tasks.workunit.client.0.smithi027.stdout:7/708: rename d4/d5/d8/d21/d94/da7/fb9 to d4/d1b/d4a/d7a/d93/fde 0 2024-04-20T10:59:23.356 INFO:tasks.workunit.client.0.smithi027.stdout:7/709: dread - d4/d5/f6c zero size 2024-04-20T10:59:23.357 INFO:tasks.workunit.client.0.smithi027.stdout:1/896: mkdir d3/d41/dc6/d142 0 2024-04-20T10:59:23.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/612: dwrite d4/d1b/d50/d5d/fb1 [0,4194304] 0 2024-04-20T10:59:23.359 INFO:tasks.workunit.client.0.smithi027.stdout:5/673: symlink d19/d1b/d1c/d1e/d83/dcc/ldc 0 2024-04-20T10:59:23.360 INFO:tasks.workunit.client.0.smithi027.stdout:5/674: read d19/d1b/d1c/fad [674161,90779] 0 2024-04-20T10:59:23.360 INFO:tasks.workunit.client.0.smithi027.stdout:5/675: fdatasync d19/d1b/d1c/d1e/d58/f7b 0 2024-04-20T10:59:23.360 INFO:tasks.workunit.client.0.smithi027.stdout:2/667: mknod d5/dc/d16/d60/cd3 0 2024-04-20T10:59:23.360 INFO:tasks.workunit.client.0.smithi027.stdout:2/668: stat d5/d44/f65 0 2024-04-20T10:59:23.361 INFO:tasks.workunit.client.0.smithi027.stdout:2/669: chown d5/d44/da7/dcc/l9e 548898513 1 2024-04-20T10:59:23.361 INFO:tasks.workunit.client.0.smithi027.stdout:3/719: read d2/dd/fc0 [3794700,106177] 0 2024-04-20T10:59:23.373 INFO:tasks.workunit.client.0.smithi027.stdout:3/720: write d2/dd/d22/f2f [2942640,51599] 0 2024-04-20T10:59:23.373 INFO:tasks.workunit.client.0.smithi027.stdout:3/721: stat d2/f25 0 2024-04-20T10:59:23.373 INFO:tasks.workunit.client.0.smithi027.stdout:3/722: chown d2/d18/d2c/f30 9299 1 2024-04-20T10:59:23.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/897: dwrite d3/d7/d38/ddf/d73/f7d [0,4194304] 0 2024-04-20T10:59:23.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/898: dread - d3/d7/d38/ddf/d73/f12f zero size 2024-04-20T10:59:23.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/899: readlink d3/d20/dfb/l101 0 2024-04-20T10:59:23.378 INFO:tasks.workunit.client.0.smithi027.stdout:3/723: dread d2/d18/d2c/d4c/f7e [0,4194304] 0 2024-04-20T10:59:23.617 INFO:tasks.workunit.client.0.smithi027.stdout:9/613: truncate d4/d1b/d31/f3d 20989 0 2024-04-20T10:59:23.617 INFO:tasks.workunit.client.0.smithi027.stdout:5/676: rmdir d19/d1b/d1c/d1e/d22/db1/dcf 39 2024-04-20T10:59:23.617 INFO:tasks.workunit.client.0.smithi027.stdout:2/670: rename d5/d44/fa5 to d5/dc/d16/db1/fd4 0 2024-04-20T10:59:23.618 INFO:tasks.workunit.client.0.smithi027.stdout:1/900: mkdir d3/d143 0 2024-04-20T10:59:23.620 INFO:tasks.workunit.client.0.smithi027.stdout:9/614: creat d4/fb8 x:0 0 0 2024-04-20T10:59:23.621 INFO:tasks.workunit.client.0.smithi027.stdout:5/677: mknod d19/d1b/d1c/d1e/d22/d68/d75/db7/cdd 0 2024-04-20T10:59:23.622 INFO:tasks.workunit.client.0.smithi027.stdout:2/671: mkdir d5/dc/d16/dd0/dd5 0 2024-04-20T10:59:23.623 INFO:tasks.workunit.client.0.smithi027.stdout:3/724: truncate d2/dd/d57/d8d/fc2 306386 0 2024-04-20T10:59:23.623 INFO:tasks.workunit.client.0.smithi027.stdout:3/725: stat d2/d94/dc4/lde 0 2024-04-20T10:59:23.623 INFO:tasks.workunit.client.0.smithi027.stdout:3/726: fsync d2/d18/d2c/d4c/d67/d6c/f87 0 2024-04-20T10:59:23.625 INFO:tasks.workunit.client.0.smithi027.stdout:5/678: rename d19/d1b/d1c/d1e/d3c/lda to d19/d1b/d1c/d1e/d22/dca/lde 0 2024-04-20T10:59:23.627 INFO:tasks.workunit.client.0.smithi027.stdout:9/615: link d4/d14/l37 d4/d2a/lb9 0 2024-04-20T10:59:23.628 INFO:tasks.workunit.client.0.smithi027.stdout:4/727: sync 2024-04-20T10:59:23.628 INFO:tasks.workunit.client.0.smithi027.stdout:6/780: sync 2024-04-20T10:59:23.628 INFO:tasks.workunit.client.0.smithi027.stdout:8/771: sync 2024-04-20T10:59:23.628 INFO:tasks.workunit.client.0.smithi027.stdout:0/841: sync 2024-04-20T10:59:23.629 INFO:tasks.workunit.client.0.smithi027.stdout:0/842: truncate d0/d11/f21 872770 0 2024-04-20T10:59:23.629 INFO:tasks.workunit.client.0.smithi027.stdout:0/843: write d0/d59/d9e/fb7 [976505,83122] 0 2024-04-20T10:59:23.630 INFO:tasks.workunit.client.0.smithi027.stdout:8/772: mkdir d9/de/d1e/d4a/d6d/d103 0 2024-04-20T10:59:23.633 INFO:tasks.workunit.client.0.smithi027.stdout:8/773: rename d9/de/da3/d3c/d4b to d9/de/d1e/d4a/d6d/db7/d5b/d104 0 2024-04-20T10:59:23.635 INFO:tasks.workunit.client.0.smithi027.stdout:8/774: link d9/de/d1e/d4a/d6d/d75/c94 d9/de/d1e/d4a/d6d/d7d/c105 0 2024-04-20T10:59:23.635 INFO:tasks.workunit.client.0.smithi027.stdout:8/775: write d9/de/d64/d6c/f8c [1088544,4607] 0 2024-04-20T10:59:23.636 INFO:tasks.workunit.client.0.smithi027.stdout:8/776: creat d9/de/da3/f106 x:0 0 0 2024-04-20T10:59:23.637 INFO:tasks.workunit.client.0.smithi027.stdout:8/777: mkdir d9/de/d1e/d107 0 2024-04-20T10:59:23.637 INFO:tasks.workunit.client.0.smithi027.stdout:6/781: read da/d21/dab/d19/d2e/d6b/f62 [365871,6656] 0 2024-04-20T10:59:23.639 INFO:tasks.workunit.client.0.smithi027.stdout:6/782: truncate da/d21/dab/d19/d2e/f41 602544 0 2024-04-20T10:59:23.640 INFO:tasks.workunit.client.0.smithi027.stdout:6/783: mknod da/d21/d28/cfb 0 2024-04-20T10:59:23.640 INFO:tasks.workunit.client.0.smithi027.stdout:3/727: dread d2/d18/f53 [0,4194304] 0 2024-04-20T10:59:23.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/784: rename da/d21/d28/fad to da/d21/dab/d19/ffc 0 2024-04-20T10:59:23.642 INFO:tasks.workunit.client.0.smithi027.stdout:8/778: write d9/de/d1e/d4a/d6d/db7/d97/fa4 [863902,60703] 0 2024-04-20T10:59:23.642 INFO:tasks.workunit.client.0.smithi027.stdout:6/785: rename da/d21/dab/d7e/daf/cb1 to da/db8/cfd 0 2024-04-20T10:59:23.642 INFO:tasks.workunit.client.0.smithi027.stdout:6/786: truncate da/d21/dab/fc7 825013 0 2024-04-20T10:59:23.643 INFO:tasks.workunit.client.0.smithi027.stdout:9/616: write d4/d74/d72/f98 [1555127,34183] 0 2024-04-20T10:59:23.644 INFO:tasks.workunit.client.0.smithi027.stdout:8/779: rmdir d9/de/d18 39 2024-04-20T10:59:23.644 INFO:tasks.workunit.client.0.smithi027.stdout:2/672: dwrite d5/dc/d16/d60/fbf [0,4194304] 0 2024-04-20T10:59:23.645 INFO:tasks.workunit.client.0.smithi027.stdout:9/617: symlink d4/d1b/d31/d51/lba 0 2024-04-20T10:59:23.646 INFO:tasks.workunit.client.0.smithi027.stdout:8/780: mknod d9/de/d1e/d4a/d6d/db7/d5b/c108 0 2024-04-20T10:59:23.648 INFO:tasks.workunit.client.0.smithi027.stdout:2/673: creat d5/dc/d16/d2a/d59/fd6 x:0 0 0 2024-04-20T10:59:23.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/618: dread d4/d1b/f53 [0,4194304] 0 2024-04-20T10:59:23.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/619: truncate d4/d1b/d31/f3d 717788 0 2024-04-20T10:59:23.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/620: read d4/f7 [3565429,119701] 0 2024-04-20T10:59:23.649 INFO:tasks.workunit.client.0.smithi027.stdout:8/781: creat d9/de/d1e/d4a/d6d/d75/d7c/f109 x:0 0 0 2024-04-20T10:59:23.649 INFO:tasks.workunit.client.0.smithi027.stdout:8/782: stat d9/de/d1e/d4a/d6d/db7/d97/fa1 0 2024-04-20T10:59:23.649 INFO:tasks.workunit.client.0.smithi027.stdout:8/783: fdatasync d9/de/f2b 0 2024-04-20T10:59:23.650 INFO:tasks.workunit.client.0.smithi027.stdout:8/784: dread - d9/de/d1e/d4a/d6d/dea/d59/da0/fe0 zero size 2024-04-20T10:59:23.650 INFO:tasks.workunit.client.0.smithi027.stdout:2/674: rename d5/d1e/f70 to d5/dc/fd7 0 2024-04-20T10:59:23.653 INFO:tasks.workunit.client.0.smithi027.stdout:9/621: rename d4/d1b/d1f/d5a/f79 to d4/d1b/d1f/d45/d8a/db2/fbb 0 2024-04-20T10:59:23.654 INFO:tasks.workunit.client.0.smithi027.stdout:6/787: dread da/d21/dab/d19/d2e/d32/fb3 [0,4194304] 0 2024-04-20T10:59:23.654 INFO:tasks.workunit.client.0.smithi027.stdout:2/675: link d5/dc/d16/d60/dac/f67 d5/dc/d16/d32/dab/dba/fd8 0 2024-04-20T10:59:23.656 INFO:tasks.workunit.client.0.smithi027.stdout:8/785: rename d9/de/d1e/d4a/d6d/fbb to d9/de/d1e/d4a/d6d/db7/f10a 0 2024-04-20T10:59:23.656 INFO:tasks.workunit.client.0.smithi027.stdout:9/622: dread d4/d1b/d1f/d45/d8a/f6e [0,4194304] 0 2024-04-20T10:59:23.657 INFO:tasks.workunit.client.0.smithi027.stdout:6/788: mknod da/d21/dab/d7e/cfe 0 2024-04-20T10:59:23.657 INFO:tasks.workunit.client.0.smithi027.stdout:6/789: stat da/d21/d28/d2d/d53 0 2024-04-20T10:59:23.657 INFO:tasks.workunit.client.0.smithi027.stdout:6/790: readlink da/d21/dab/d19/d2e/d6b/ld8 0 2024-04-20T10:59:23.658 INFO:tasks.workunit.client.0.smithi027.stdout:6/791: readlink da/d21/dab/d7e/d86/dcd/lec 0 2024-04-20T10:59:23.658 INFO:tasks.workunit.client.0.smithi027.stdout:2/676: truncate d5/dc/d16/d60/d8e/fb5 387350 0 2024-04-20T10:59:23.658 INFO:tasks.workunit.client.0.smithi027.stdout:9/623: creat d4/d1b/d1f/d5a/fbc x:0 0 0 2024-04-20T10:59:23.659 INFO:tasks.workunit.client.0.smithi027.stdout:3/728: dwrite d2/d7f/fa4 [0,4194304] 0 2024-04-20T10:59:23.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/792: mkdir da/dcc/dff 0 2024-04-20T10:59:23.660 INFO:tasks.workunit.client.0.smithi027.stdout:2/677: truncate d5/dc/d16/d60/dac/f50 7729917 0 2024-04-20T10:59:23.660 INFO:tasks.workunit.client.0.smithi027.stdout:2/678: write d5/dc/f1c [5361932,116635] 0 2024-04-20T10:59:23.660 INFO:tasks.workunit.client.0.smithi027.stdout:2/679: fsync d5/f6c 0 2024-04-20T10:59:23.660 INFO:tasks.workunit.client.0.smithi027.stdout:2/680: truncate d5/dbc/fd2 932829 0 2024-04-20T10:59:23.661 INFO:tasks.workunit.client.0.smithi027.stdout:3/729: creat d2/dd/df/d23/fe6 x:0 0 0 2024-04-20T10:59:23.661 INFO:tasks.workunit.client.0.smithi027.stdout:3/730: truncate d2/d18/f26 4988241 0 2024-04-20T10:59:23.661 INFO:tasks.workunit.client.0.smithi027.stdout:3/731: chown d2/d7f 0 1 2024-04-20T10:59:23.662 INFO:tasks.workunit.client.0.smithi027.stdout:6/793: stat da/d21/dab/lbd 0 2024-04-20T10:59:23.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/681: unlink d5/d44/da7/dcc/cbb 0 2024-04-20T10:59:23.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/682: truncate d5/d44/da7/dcc/fcd 1020603 0 2024-04-20T10:59:23.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/683: chown d5/d44/da7/dcc/l9e 1 1 2024-04-20T10:59:23.665 INFO:tasks.workunit.client.0.smithi027.stdout:2/684: rename d5/dbc/fcb to d5/d26/fd9 0 2024-04-20T10:59:23.666 INFO:tasks.workunit.client.0.smithi027.stdout:2/685: read d5/dbc/fd2 [155099,99793] 0 2024-04-20T10:59:23.667 INFO:tasks.workunit.client.0.smithi027.stdout:2/686: mknod d5/d26/d47/cda 0 2024-04-20T10:59:23.669 INFO:tasks.workunit.client.0.smithi027.stdout:2/687: symlink d5/dc/d16/db1/ldb 0 2024-04-20T10:59:23.669 INFO:tasks.workunit.client.0.smithi027.stdout:2/688: dread - d5/d26/fd9 zero size 2024-04-20T10:59:23.671 INFO:tasks.workunit.client.0.smithi027.stdout:9/624: write d4/d1b/d31/f75 [3876184,121864] 0 2024-04-20T10:59:23.671 INFO:tasks.workunit.client.0.smithi027.stdout:9/625: chown d4/d1b/f22 11019 1 2024-04-20T10:59:23.671 INFO:tasks.workunit.client.0.smithi027.stdout:2/689: rename d5/d1e/d21/fc3 to d5/d26/d47/fdc 0 2024-04-20T10:59:23.672 INFO:tasks.workunit.client.0.smithi027.stdout:6/794: dread da/d21/dab/fa2 [0,4194304] 0 2024-04-20T10:59:23.675 INFO:tasks.workunit.client.0.smithi027.stdout:8/786: dwrite d9/de/d1e/d4a/d6d/db7/d97/fa4 [0,4194304] 0 2024-04-20T10:59:23.677 INFO:tasks.workunit.client.0.smithi027.stdout:8/787: getdents d9/de/d1e/d4a/d6d/dea/d59/d74/de8 0 2024-04-20T10:59:23.677 INFO:tasks.workunit.client.0.smithi027.stdout:8/788: write d9/de/d64/d6c/f93 [606874,32731] 0 2024-04-20T10:59:23.680 INFO:tasks.workunit.client.0.smithi027.stdout:9/626: dread d4/d1b/d31/d23/f99 [0,4194304] 0 2024-04-20T10:59:23.686 INFO:tasks.workunit.client.0.smithi027.stdout:9/627: symlink d4/lbd 0 2024-04-20T10:59:23.686 INFO:tasks.workunit.client.0.smithi027.stdout:9/628: getdents d4/d1b/d50/d5d 0 2024-04-20T10:59:23.688 INFO:tasks.workunit.client.0.smithi027.stdout:9/629: creat d4/d2a/fbe x:0 0 0 2024-04-20T10:59:23.688 INFO:tasks.workunit.client.0.smithi027.stdout:9/630: read d4/f1c [1639238,99800] 0 2024-04-20T10:59:23.688 INFO:tasks.workunit.client.0.smithi027.stdout:9/631: write d4/d1b/d1f/d45/f83 [882625,580] 0 2024-04-20T10:59:23.688 INFO:tasks.workunit.client.0.smithi027.stdout:9/632: chown d4/d74/d72 454658651 1 2024-04-20T10:59:23.690 INFO:tasks.workunit.client.0.smithi027.stdout:9/633: link d4/d14/l37 d4/d1b/d1f/d45/lbf 0 2024-04-20T10:59:23.690 INFO:tasks.workunit.client.0.smithi027.stdout:9/634: readlink d4/d1b/d1f/d45/d8a/l5e 0 2024-04-20T10:59:23.690 INFO:tasks.workunit.client.0.smithi027.stdout:9/635: fsync d4/d1b/d1f/d45/d8a/db2/fbb 0 2024-04-20T10:59:23.690 INFO:tasks.workunit.client.0.smithi027.stdout:9/636: write d4/f1e [2624917,28807] 0 2024-04-20T10:59:23.692 INFO:tasks.workunit.client.0.smithi027.stdout:9/637: unlink d4/d1b/caa 0 2024-04-20T10:59:23.694 INFO:tasks.workunit.client.0.smithi027.stdout:8/789: dwrite d9/de/d1e/d4a/d6d/d75/f83 [0,4194304] 0 2024-04-20T10:59:23.694 INFO:tasks.workunit.client.0.smithi027.stdout:8/790: stat d9/f15 0 2024-04-20T10:59:23.694 INFO:tasks.workunit.client.0.smithi027.stdout:8/791: readlink d9/le2 0 2024-04-20T10:59:23.696 INFO:tasks.workunit.client.0.smithi027.stdout:7/710: dread d4/d5/f33 [0,4194304] 0 2024-04-20T10:59:23.696 INFO:tasks.workunit.client.0.smithi027.stdout:9/638: dread d4/d1b/d31/d23/d29/f35 [0,4194304] 0 2024-04-20T10:59:23.696 INFO:tasks.workunit.client.0.smithi027.stdout:9/639: fsync d4/d1b/d1f/d5a/fbc 0 2024-04-20T10:59:23.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/711: truncate d4/d5/fa 5225734 0 2024-04-20T10:59:23.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/712: readlink d4/d5/d81/lc3 0 2024-04-20T10:59:23.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/713: write d4/d5b/f9e [57461,1747] 0 2024-04-20T10:59:23.700 INFO:tasks.workunit.client.0.smithi027.stdout:2/690: dwrite d5/dc/d16/d32/dab/dba/fd8 [0,4194304] 0 2024-04-20T10:59:23.701 INFO:tasks.workunit.client.0.smithi027.stdout:2/691: symlink d5/d26/d47/d8a/ldd 0 2024-04-20T10:59:23.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/795: dwrite da/d21/dab/f4f [0,4194304] 0 2024-04-20T10:59:23.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/692: creat d5/dc/d16/d2a/fde x:0 0 0 2024-04-20T10:59:23.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/693: stat d5/dc/d16/d32/c4f 0 2024-04-20T10:59:23.705 INFO:tasks.workunit.client.0.smithi027.stdout:2/694: mknod d5/d26/d78/cdf 0 2024-04-20T10:59:23.705 INFO:tasks.workunit.client.0.smithi027.stdout:9/640: dread d4/d1b/f89 [0,4194304] 0 2024-04-20T10:59:23.705 INFO:tasks.workunit.client.0.smithi027.stdout:9/641: chown d4/d1b/d31/f9a 7 1 2024-04-20T10:59:23.706 INFO:tasks.workunit.client.0.smithi027.stdout:6/796: write da/d21/dab/d19/d2e/d6b/fd2 [180412,62561] 0 2024-04-20T10:59:23.706 INFO:tasks.workunit.client.0.smithi027.stdout:2/695: truncate d5/d1e/f42 1933386 0 2024-04-20T10:59:23.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/642: mkdir d4/d1b/d1f/d5a/db7/dc0 0 2024-04-20T10:59:23.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/643: chown d4/d1b/d50/d5d 2412 1 2024-04-20T10:59:23.708 INFO:tasks.workunit.client.0.smithi027.stdout:6/797: symlink da/d21/d28/d54/l100 0 2024-04-20T10:59:23.708 INFO:tasks.workunit.client.0.smithi027.stdout:6/798: chown da/d21/dab/d7e/c94 433253 1 2024-04-20T10:59:23.708 INFO:tasks.workunit.client.0.smithi027.stdout:2/696: symlink d5/dc/d16/d32/dab/le0 0 2024-04-20T10:59:23.710 INFO:tasks.workunit.client.0.smithi027.stdout:9/644: truncate d4/d1b/d8f/f90 6494101 0 2024-04-20T10:59:23.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/714: dwrite d4/d5/f5c [0,4194304] 0 2024-04-20T10:59:23.730 INFO:tasks.workunit.client.0.smithi027.stdout:3/732: dwrite d2/f66 [0,4194304] 0 2024-04-20T10:59:23.730 INFO:tasks.workunit.client.0.smithi027.stdout:3/733: chown d2/dd/df/d23/d68/l9b 8 1 2024-04-20T10:59:23.734 INFO:tasks.workunit.client.0.smithi027.stdout:9/645: dwrite d4/d14/f44 [0,4194304] 0 2024-04-20T10:59:23.749 INFO:tasks.workunit.client.0.smithi027.stdout:2/697: dwrite d5/dc/d16/d60/fbf [0,4194304] 0 2024-04-20T10:59:23.763 INFO:tasks.workunit.client.0.smithi027.stdout:2/698: dwrite f1 [0,4194304] 0 2024-04-20T10:59:23.763 INFO:tasks.workunit.client.0.smithi027.stdout:2/699: write d5/dc/f41 [2205246,125876] 0 2024-04-20T10:59:23.772 INFO:tasks.workunit.client.0.smithi027.stdout:2/700: write d5/dc/fd7 [1378873,5485] 0 2024-04-20T10:59:23.772 INFO:tasks.workunit.client.0.smithi027.stdout:2/701: fsync d5/d44/d7c/fc5 0 2024-04-20T10:59:23.977 INFO:tasks.workunit.client.0.smithi027.stdout:7/715: truncate d4/d1b/d4a/f68 2535664 0 2024-04-20T10:59:23.979 INFO:tasks.workunit.client.0.smithi027.stdout:3/734: unlink d2/d7f/fa4 0 2024-04-20T10:59:23.981 INFO:tasks.workunit.client.0.smithi027.stdout:9/646: unlink d4/lbd 0 2024-04-20T10:59:23.986 INFO:tasks.workunit.client.0.smithi027.stdout:2/702: rename d5/d44/da7 to d5/dc/d16/d2a/d59/da4/de1 0 2024-04-20T10:59:23.988 INFO:tasks.workunit.client.0.smithi027.stdout:7/716: creat d4/d5/d8/d21/d94/da7/dce/fdf x:0 0 0 2024-04-20T10:59:23.988 INFO:tasks.workunit.client.0.smithi027.stdout:3/735: read d2/f66 [1277779,27868] 0 2024-04-20T10:59:23.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/647: creat d4/d1b/d1f/fc1 x:0 0 0 2024-04-20T10:59:23.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/648: dread - d4/fb8 zero size 2024-04-20T10:59:23.989 INFO:tasks.workunit.client.0.smithi027.stdout:9/649: fdatasync d4/d1b/d1f/d45/f71 0 2024-04-20T10:59:23.990 INFO:tasks.workunit.client.0.smithi027.stdout:9/650: chown d4/d14/d43 175194 1 2024-04-20T10:59:23.991 INFO:tasks.workunit.client.0.smithi027.stdout:2/703: rename d5/dc/d16/d60 to d5/dc/d16/d37/de2 0 2024-04-20T10:59:23.992 INFO:tasks.workunit.client.0.smithi027.stdout:9/651: mknod d4/d1b/d31/d23/d29/d7f/cc2 0 2024-04-20T10:59:23.994 INFO:tasks.workunit.client.0.smithi027.stdout:2/704: link d5/d26/d78/lb6 d5/d26/d30/le3 0 2024-04-20T10:59:23.994 INFO:tasks.workunit.client.0.smithi027.stdout:3/736: dread d2/dd/df/d23/f50 [0,4194304] 0 2024-04-20T10:59:23.994 INFO:tasks.workunit.client.0.smithi027.stdout:3/737: write d2/dd/d57/fa5 [865983,53406] 0 2024-04-20T10:59:23.994 INFO:tasks.workunit.client.0.smithi027.stdout:3/738: readlink d2/dd/df/d23/d68/l9b 0 2024-04-20T10:59:23.995 INFO:tasks.workunit.client.0.smithi027.stdout:9/652: mknod d4/d74/d72/cc3 0 2024-04-20T10:59:23.995 INFO:tasks.workunit.client.0.smithi027.stdout:2/705: mknod d5/d44/d7c/ce4 0 2024-04-20T10:59:23.996 INFO:tasks.workunit.client.0.smithi027.stdout:9/653: chown d4/l59 5 1 2024-04-20T10:59:23.996 INFO:tasks.workunit.client.0.smithi027.stdout:9/654: readlink d4/d1b/d31/d51/lba 0 2024-04-20T10:59:24.005 INFO:tasks.workunit.client.0.smithi027.stdout:7/717: dwrite d4/d1b/f72 [4194304,4194304] 0 2024-04-20T10:59:24.006 INFO:tasks.workunit.client.0.smithi027.stdout:7/718: fsync d4/d1b/d4a/d62/fa5 0 2024-04-20T10:59:24.008 INFO:tasks.workunit.client.0.smithi027.stdout:7/719: mknod d4/d5/d8/d46/ce0 0 2024-04-20T10:59:24.008 INFO:tasks.workunit.client.0.smithi027.stdout:6/799: sync 2024-04-20T10:59:24.009 INFO:tasks.workunit.client.0.smithi027.stdout:4/728: sync 2024-04-20T10:59:24.009 INFO:tasks.workunit.client.0.smithi027.stdout:5/679: sync 2024-04-20T10:59:24.009 INFO:tasks.workunit.client.0.smithi027.stdout:1/901: sync 2024-04-20T10:59:24.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/844: sync 2024-04-20T10:59:24.009 INFO:tasks.workunit.client.0.smithi027.stdout:1/902: chown d3/d7/d38/da4/dac/de3 0 1 2024-04-20T10:59:24.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/845: write d0/d11/d2c/d2e/d2b/d7b/d34/d5a/fcc [758735,44158] 0 2024-04-20T10:59:24.009 INFO:tasks.workunit.client.0.smithi027.stdout:7/720: creat d4/d5b/fe1 x:0 0 0 2024-04-20T10:59:24.010 INFO:tasks.workunit.client.0.smithi027.stdout:5/680: creat d19/d1b/d1c/d1e/d3c/fdf x:0 0 0 2024-04-20T10:59:24.011 INFO:tasks.workunit.client.0.smithi027.stdout:9/655: read d4/d1b/f89 [756006,92399] 0 2024-04-20T10:59:24.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/800: symlink da/d21/dab/d19/d2e/df1/l101 0 2024-04-20T10:59:24.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/801: chown da/d21/dab/d7e/d86/da3/cf4 29288 1 2024-04-20T10:59:24.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/802: chown da/f2a 16194 1 2024-04-20T10:59:24.014 INFO:tasks.workunit.client.0.smithi027.stdout:7/721: creat d4/d1b/d58/fe2 x:0 0 0 2024-04-20T10:59:24.015 INFO:tasks.workunit.client.0.smithi027.stdout:6/803: mkdir da/d21/dab/d19/d2e/df1/d102 0 2024-04-20T10:59:24.017 INFO:tasks.workunit.client.0.smithi027.stdout:4/729: rename d0/d14/d6b/da3/d1b/d33/d5c/d6f/fa0 to d0/f108 0 2024-04-20T10:59:24.020 INFO:tasks.workunit.client.0.smithi027.stdout:7/722: rename d4/d1b/d4a/d62/f56 to d4/d5/d8/d46/fe3 0 2024-04-20T10:59:24.021 INFO:tasks.workunit.client.0.smithi027.stdout:0/846: read d0/d11/d2c/d2e/d2b/d3d/f84 [1730632,68497] 0 2024-04-20T10:59:24.021 INFO:tasks.workunit.client.0.smithi027.stdout:3/739: dwrite d2/d18/d2c/f78 [0,4194304] 0 2024-04-20T10:59:24.021 INFO:tasks.workunit.client.0.smithi027.stdout:4/730: mknod d0/d14/d6b/da3/c109 0 2024-04-20T10:59:24.022 INFO:tasks.workunit.client.0.smithi027.stdout:2/706: dwrite d5/d44/d7c/fc5 [4194304,4194304] 0 2024-04-20T10:59:24.022 INFO:tasks.workunit.client.0.smithi027.stdout:5/681: dread d19/d1b/d1c/d1e/d3c/f74 [0,4194304] 0 2024-04-20T10:59:24.023 INFO:tasks.workunit.client.0.smithi027.stdout:6/804: rename da/dd/d3d/f90 to da/dd/d3d/db9/f103 0 2024-04-20T10:59:24.024 INFO:tasks.workunit.client.0.smithi027.stdout:1/903: dread d3/d7/d38/ddf/dcc/fd0 [0,4194304] 0 2024-04-20T10:59:24.024 INFO:tasks.workunit.client.0.smithi027.stdout:3/740: mkdir d2/dd/d57/de7 0 2024-04-20T10:59:24.024 INFO:tasks.workunit.client.0.smithi027.stdout:3/741: chown d2/dd/d57/f59 85 1 2024-04-20T10:59:24.024 INFO:tasks.workunit.client.0.smithi027.stdout:3/742: write d2/d94/dc4/fce [462186,108830] 0 2024-04-20T10:59:24.024 INFO:tasks.workunit.client.0.smithi027.stdout:7/723: dread d4/f70 [0,4194304] 0 2024-04-20T10:59:24.024 INFO:tasks.workunit.client.0.smithi027.stdout:2/707: truncate d5/d44/d7c/f7e 112074 0 2024-04-20T10:59:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:4/731: rename d0/d14/d6b/da3/d53/d79 to d0/d14/d6b/da3/d18/d10a 0 2024-04-20T10:59:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:4/732: truncate d0/d14/d6b/da3/d53/d6a/f9b 1042191 0 2024-04-20T10:59:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:6/805: creat da/d21/dab/d7e/d86/f104 x:0 0 0 2024-04-20T10:59:24.026 INFO:tasks.workunit.client.0.smithi027.stdout:6/806: write da/d21/dab/d7e/d86/ff6 [429421,75882] 0 2024-04-20T10:59:24.028 INFO:tasks.workunit.client.0.smithi027.stdout:7/724: rename d4/d1b/d58/f5f to d4/d5/d6f/fe4 0 2024-04-20T10:59:24.031 INFO:tasks.workunit.client.0.smithi027.stdout:8/792: write d9/de/d18/f22 [1070502,127190] 0 2024-04-20T10:59:24.032 INFO:tasks.workunit.client.0.smithi027.stdout:9/656: write d4/f1c [4246758,8504] 0 2024-04-20T10:59:24.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/743: rename d2/dd/d57/d8d/fa3 to d2/dd/d57/d8d/dc6/de1/fe8 0 2024-04-20T10:59:24.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/682: dread d19/d1b/d1c/d1e/d22/db1/f9f [0,4194304] 0 2024-04-20T10:59:24.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/683: fdatasync d19/d1b/f33 0 2024-04-20T10:59:24.036 INFO:tasks.workunit.client.0.smithi027.stdout:5/684: dread d19/d1b/d1c/d1e/d3c/f96 [0,4194304] 0 2024-04-20T10:59:24.038 INFO:tasks.workunit.client.0.smithi027.stdout:7/725: rename d4/d5/d8/d46/fe3 to d4/d1b/d58/fe5 0 2024-04-20T10:59:24.041 INFO:tasks.workunit.client.0.smithi027.stdout:3/744: unlink d2/dd/d57/d8c/fcf 0 2024-04-20T10:59:24.043 INFO:tasks.workunit.client.0.smithi027.stdout:7/726: link d4/d5/d8/d21/d94/da7/dc2/lc8 d4/d1b/le6 0 2024-04-20T10:59:24.045 INFO:tasks.workunit.client.0.smithi027.stdout:8/793: rename d9/de/d64/l91 to d9/de/d1e/d4a/d6d/dea/d59/da0/dbd/l10b 0 2024-04-20T10:59:24.046 INFO:tasks.workunit.client.0.smithi027.stdout:3/745: creat d2/dd/d57/d8d/fe9 x:0 0 0 2024-04-20T10:59:24.046 INFO:tasks.workunit.client.0.smithi027.stdout:3/746: chown d2/dd/d22/l9c 282882945 1 2024-04-20T10:59:24.046 INFO:tasks.workunit.client.0.smithi027.stdout:3/747: truncate d2/fc5 4380795 0 2024-04-20T10:59:24.048 INFO:tasks.workunit.client.0.smithi027.stdout:7/727: rename d4/d5/d8/d21/f31 to d4/d5/d8/d45/fe7 0 2024-04-20T10:59:24.049 INFO:tasks.workunit.client.0.smithi027.stdout:8/794: creat d9/de/d1e/d4a/d6d/d79/f10c x:0 0 0 2024-04-20T10:59:24.052 INFO:tasks.workunit.client.0.smithi027.stdout:3/748: link d2/d80/f45 d2/dd/df/d23/fea 0 2024-04-20T10:59:24.052 INFO:tasks.workunit.client.0.smithi027.stdout:1/904: dwrite d3/d7/d38/f56 [0,4194304] 0 2024-04-20T10:59:24.052 INFO:tasks.workunit.client.0.smithi027.stdout:1/905: chown d3 1072 1 2024-04-20T10:59:24.055 INFO:tasks.workunit.client.0.smithi027.stdout:1/906: write d3/d20/f4d [1546561,46607] 0 2024-04-20T10:59:24.055 INFO:tasks.workunit.client.0.smithi027.stdout:1/907: write d3/d7e/f83 [276446,1077] 0 2024-04-20T10:59:24.055 INFO:tasks.workunit.client.0.smithi027.stdout:1/908: dread - d3/d7/d38/ddf/de9/f10d zero size 2024-04-20T10:59:24.055 INFO:tasks.workunit.client.0.smithi027.stdout:7/728: creat d4/d5/d81/fe8 x:0 0 0 2024-04-20T10:59:24.056 INFO:tasks.workunit.client.0.smithi027.stdout:1/909: creat d3/d20/f144 x:0 0 0 2024-04-20T10:59:24.060 INFO:tasks.workunit.client.0.smithi027.stdout:1/910: dread d3/d7/d38/da4/d140/fea [0,4194304] 0 2024-04-20T10:59:24.062 INFO:tasks.workunit.client.0.smithi027.stdout:1/911: creat d3/d7/d38/da4/dbc/f145 x:0 0 0 2024-04-20T10:59:24.062 INFO:tasks.workunit.client.0.smithi027.stdout:1/912: read - d3/d7/d38/ddf/fde zero size 2024-04-20T10:59:24.062 INFO:tasks.workunit.client.0.smithi027.stdout:1/913: truncate d3/d20/f104 333291 0 2024-04-20T10:59:24.063 INFO:tasks.workunit.client.0.smithi027.stdout:1/914: symlink d3/d7/d28/db0/d11c/l146 0 2024-04-20T10:59:24.064 INFO:tasks.workunit.client.0.smithi027.stdout:1/915: write d3/d20/f4d [205294,34412] 0 2024-04-20T10:59:24.065 INFO:tasks.workunit.client.0.smithi027.stdout:1/916: creat d3/d7/d38/f147 x:0 0 0 2024-04-20T10:59:24.074 INFO:tasks.workunit.client.0.smithi027.stdout:1/917: dread d3/d7/d28/d92/f109 [4194304,4194304] 0 2024-04-20T10:59:24.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/685: dwrite d19/d1b/d1c/d1e/d25/faf [0,4194304] 0 2024-04-20T10:59:24.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/686: readlink d19/d1b/d1c/d1e/l54 0 2024-04-20T10:59:24.076 INFO:tasks.workunit.client.0.smithi027.stdout:0/847: dwrite d0/d11/d2c/d2e/d2b/d7b/dcf/f112 [0,4194304] 0 2024-04-20T10:59:24.076 INFO:tasks.workunit.client.0.smithi027.stdout:0/848: readlink d0/d11/d2c/d2e/d35/d8c/lbc 0 2024-04-20T10:59:24.076 INFO:tasks.workunit.client.0.smithi027.stdout:2/708: dwrite f1 [0,4194304] 0 2024-04-20T10:59:24.077 INFO:tasks.workunit.client.0.smithi027.stdout:2/709: fsync d5/d44/f90 0 2024-04-20T10:59:24.077 INFO:tasks.workunit.client.0.smithi027.stdout:1/918: mknod d3/d7/d28/d92/c148 0 2024-04-20T10:59:24.080 INFO:tasks.workunit.client.0.smithi027.stdout:8/795: dwrite d9/de/d1e/f65 [0,4194304] 0 2024-04-20T10:59:24.080 INFO:tasks.workunit.client.0.smithi027.stdout:9/657: dwrite d4/f6b [0,4194304] 0 2024-04-20T10:59:24.080 INFO:tasks.workunit.client.0.smithi027.stdout:9/658: fsync d4/d1b/d50/fa2 0 2024-04-20T10:59:24.080 INFO:tasks.workunit.client.0.smithi027.stdout:9/659: write d4/d1b/d31/d23/d29/d7f/fad [257522,65588] 0 2024-04-20T10:59:24.081 INFO:tasks.workunit.client.0.smithi027.stdout:4/733: dwrite d0/d14/d6b/da3/d18/db9/fc9 [4194304,4194304] 0 2024-04-20T10:59:24.081 INFO:tasks.workunit.client.0.smithi027.stdout:4/734: stat d0/d14/d6b/da3/d1b/dc3/dc5 0 2024-04-20T10:59:24.082 INFO:tasks.workunit.client.0.smithi027.stdout:0/849: creat d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/d10e/f124 x:0 0 0 2024-04-20T10:59:24.083 INFO:tasks.workunit.client.0.smithi027.stdout:2/710: creat d5/dc/d16/db1/fe5 x:0 0 0 2024-04-20T10:59:24.083 INFO:tasks.workunit.client.0.smithi027.stdout:2/711: read - d5/dc/d16/d2a/d59/fd6 zero size 2024-04-20T10:59:24.083 INFO:tasks.workunit.client.0.smithi027.stdout:2/712: dread - d5/dc/d16/d2a/fde zero size 2024-04-20T10:59:24.086 INFO:tasks.workunit.client.0.smithi027.stdout:7/729: rename d4/d1b/d58/fe5 to d4/d5/d8/fe9 0 2024-04-20T10:59:24.086 INFO:tasks.workunit.client.0.smithi027.stdout:7/730: write d4/d1b/d4a/d62/dcd/dd5/fd9 [823400,84869] 0 2024-04-20T10:59:24.088 INFO:tasks.workunit.client.0.smithi027.stdout:9/660: truncate d4/d1b/d31/f67 1780582 0 2024-04-20T10:59:24.091 INFO:tasks.workunit.client.0.smithi027.stdout:2/713: creat d5/dc/d16/d37/de2/dac/fe6 x:0 0 0 2024-04-20T10:59:24.091 INFO:tasks.workunit.client.0.smithi027.stdout:1/919: dread d3/d7/d38/ddf/d73/fc4 [0,4194304] 0 2024-04-20T10:59:24.093 INFO:tasks.workunit.client.0.smithi027.stdout:8/796: rename d9/de/d1e/dc1/fcc to d9/de/d1e/dc1/dd1/f10d 0 2024-04-20T10:59:24.094 INFO:tasks.workunit.client.0.smithi027.stdout:7/731: creat d4/da2/fea x:0 0 0 2024-04-20T10:59:24.096 INFO:tasks.workunit.client.0.smithi027.stdout:9/661: symlink d4/d1b/d31/d51/lc4 0 2024-04-20T10:59:24.096 INFO:tasks.workunit.client.0.smithi027.stdout:2/714: getdents d5/d1e/d21 0 2024-04-20T10:59:24.100 INFO:tasks.workunit.client.0.smithi027.stdout:1/920: link d3/d7/d28/db0/d11c/l146 d3/d7/d28/d92/dad/l149 0 2024-04-20T10:59:24.101 INFO:tasks.workunit.client.0.smithi027.stdout:8/797: unlink d9/de/d20/dbf/fc3 0 2024-04-20T10:59:24.101 INFO:tasks.workunit.client.0.smithi027.stdout:8/798: write d9/de/d1e/d4a/d6d/dea/d59/da0/fe0 [412007,35676] 0 2024-04-20T10:59:24.101 INFO:tasks.workunit.client.0.smithi027.stdout:8/799: fdatasync d9/de/da3/d3c/f6f 0 2024-04-20T10:59:24.103 INFO:tasks.workunit.client.0.smithi027.stdout:7/732: mkdir d4/d5/d8/d46/d77/deb 0 2024-04-20T10:59:24.103 INFO:tasks.workunit.client.0.smithi027.stdout:9/662: creat d4/d1b/d1f/d5a/db7/fc5 x:0 0 0 2024-04-20T10:59:24.103 INFO:tasks.workunit.client.0.smithi027.stdout:1/921: symlink d3/d45/l14a 0 2024-04-20T10:59:24.104 INFO:tasks.workunit.client.0.smithi027.stdout:9/663: write d4/d14/d43/f86 [876169,12233] 0 2024-04-20T10:59:24.105 INFO:tasks.workunit.client.0.smithi027.stdout:9/664: chown d4/d1b/d8f 189803 1 2024-04-20T10:59:24.105 INFO:tasks.workunit.client.0.smithi027.stdout:9/665: write d4/d14/d43/f86 [174480,101642] 0 2024-04-20T10:59:24.105 INFO:tasks.workunit.client.0.smithi027.stdout:8/800: rename d9/de/d64/c6e to d9/de/d1e/d4a/d6d/d75/c10e 0 2024-04-20T10:59:24.105 INFO:tasks.workunit.client.0.smithi027.stdout:4/735: dwrite d0/d14/d6b/da3/d1b/f7f [0,4194304] 0 2024-04-20T10:59:24.105 INFO:tasks.workunit.client.0.smithi027.stdout:1/922: symlink d3/d45/dba/l14b 0 2024-04-20T10:59:24.106 INFO:tasks.workunit.client.0.smithi027.stdout:5/687: dwrite d19/d1b/d1c/d1e/f88 [0,4194304] 0 2024-04-20T10:59:24.107 INFO:tasks.workunit.client.0.smithi027.stdout:1/923: dread d3/d7/d38/d62/f6e [0,4194304] 0 2024-04-20T10:59:24.108 INFO:tasks.workunit.client.0.smithi027.stdout:8/801: link d9/de/d1e/d4a/c36 d9/de/d1e/d4a/d6d/dea/d59/d74/de8/c10f 0 2024-04-20T10:59:24.112 INFO:tasks.workunit.client.0.smithi027.stdout:4/736: link d0/d14/d6b/da3/d18/dbf/ld9 d0/d14/d6b/da3/d1b/d33/d5c/d6f/dd1/l10b 0 2024-04-20T10:59:24.112 INFO:tasks.workunit.client.0.smithi027.stdout:0/850: dwrite d0/d11/d2c/d2e/d2b/d3d/f84 [4194304,4194304] 0 2024-04-20T10:59:24.112 INFO:tasks.workunit.client.0.smithi027.stdout:0/851: rename d0 to d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71/d125 22 2024-04-20T10:59:24.118 INFO:tasks.workunit.client.0.smithi027.stdout:1/924: symlink d3/d7/d28/l14c 0 2024-04-20T10:59:24.118 INFO:tasks.workunit.client.0.smithi027.stdout:0/852: creat d0/d11/d2c/d2e/d2b/d7b/dcf/f126 x:0 0 0 2024-04-20T10:59:24.120 INFO:tasks.workunit.client.0.smithi027.stdout:0/853: unlink d0/d59/f68 0 2024-04-20T10:59:24.123 INFO:tasks.workunit.client.0.smithi027.stdout:4/737: dread d0/f3a [4194304,4194304] 0 2024-04-20T10:59:24.124 INFO:tasks.workunit.client.0.smithi027.stdout:2/715: dwrite d5/dc/f1c [4194304,4194304] 0 2024-04-20T10:59:24.124 INFO:tasks.workunit.client.0.smithi027.stdout:7/733: dwrite d4/d4d/fc5 [0,4194304] 0 2024-04-20T10:59:24.128 INFO:tasks.workunit.client.0.smithi027.stdout:5/688: dwrite fb [0,4194304] 0 2024-04-20T10:59:24.131 INFO:tasks.workunit.client.0.smithi027.stdout:1/925: read d3/d7/d38/ddf/d73/f127 [2676660,39541] 0 2024-04-20T10:59:24.131 INFO:tasks.workunit.client.0.smithi027.stdout:7/734: symlink d4/d5/d8/d21/d94/da7/dc2/lec 0 2024-04-20T10:59:24.134 INFO:tasks.workunit.client.0.smithi027.stdout:1/926: creat d3/d20/d107/f14d x:0 0 0 2024-04-20T10:59:24.134 INFO:tasks.workunit.client.0.smithi027.stdout:7/735: creat d4/d5/d6f/fed x:0 0 0 2024-04-20T10:59:24.134 INFO:tasks.workunit.client.0.smithi027.stdout:7/736: write d4/d1b/d58/fd2 [856702,24526] 0 2024-04-20T10:59:24.135 INFO:tasks.workunit.client.0.smithi027.stdout:0/854: write d0/d11/d2c/d2e/d2b/d7b/d12/d25/f31 [2845714,30838] 0 2024-04-20T10:59:24.135 INFO:tasks.workunit.client.0.smithi027.stdout:1/927: creat d3/d7/f14e x:0 0 0 2024-04-20T10:59:24.136 INFO:tasks.workunit.client.0.smithi027.stdout:5/689: dread d19/d1b/d1c/d1e/d22/db1/dcf/fc5 [0,4194304] 0 2024-04-20T10:59:24.136 INFO:tasks.workunit.client.0.smithi027.stdout:7/737: getdents d4/d5/d8/d46 0 2024-04-20T10:59:24.136 INFO:tasks.workunit.client.0.smithi027.stdout:7/738: chown d4/d1b/d4a/d7a/cb7 1454 1 2024-04-20T10:59:24.137 INFO:tasks.workunit.client.0.smithi027.stdout:1/928: mknod d3/d45/d5d/c14f 0 2024-04-20T10:59:24.139 INFO:tasks.workunit.client.0.smithi027.stdout:5/690: creat d19/d1b/d1c/d1e/d58/d9d/fe0 x:0 0 0 2024-04-20T10:59:24.139 INFO:tasks.workunit.client.0.smithi027.stdout:5/691: fsync d19/d1b/d1c/d1e/d3c/f24 0 2024-04-20T10:59:24.139 INFO:tasks.workunit.client.0.smithi027.stdout:5/692: symlink d19/d1b/d1c/d1e/d22/db1/le1 0 2024-04-20T10:59:24.140 INFO:tasks.workunit.client.0.smithi027.stdout:8/802: dread d9/de/da3/f30 [4194304,4194304] 0 2024-04-20T10:59:24.142 INFO:tasks.workunit.client.0.smithi027.stdout:4/738: dwrite d0/d14/d6b/da3/d18/db9/fc9 [0,4194304] 0 2024-04-20T10:59:24.142 INFO:tasks.workunit.client.0.smithi027.stdout:4/739: fdatasync d0/d14/d6b/da3/d1b/f7f 0 2024-04-20T10:59:24.142 INFO:tasks.workunit.client.0.smithi027.stdout:4/740: chown d0/d5e/c95 124549989 1 2024-04-20T10:59:24.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/803: mknod d9/de/d1e/d4a/d6d/dea/db3/c110 0 2024-04-20T10:59:24.147 INFO:tasks.workunit.client.0.smithi027.stdout:1/929: dread d3/d7/f34 [0,4194304] 0 2024-04-20T10:59:24.151 INFO:tasks.workunit.client.0.smithi027.stdout:8/804: symlink d9/de/d64/l111 0 2024-04-20T10:59:24.151 INFO:tasks.workunit.client.0.smithi027.stdout:8/805: write d9/de/d1e/d4a/d6d/dea/db3/f101 [876386,106799] 0 2024-04-20T10:59:24.153 INFO:tasks.workunit.client.0.smithi027.stdout:4/741: link d0/d14/d6b/da3/d1a/d3b/lee d0/d14/l10c 0 2024-04-20T10:59:24.155 INFO:tasks.workunit.client.0.smithi027.stdout:7/739: dwrite d4/d1b/f53 [0,4194304] 0 2024-04-20T10:59:24.157 INFO:tasks.workunit.client.0.smithi027.stdout:7/740: getdents d4/d5/d8/d46/d77/d7c 0 2024-04-20T10:59:24.158 INFO:tasks.workunit.client.0.smithi027.stdout:7/741: write d4/f70 [201058,126467] 0 2024-04-20T10:59:24.159 INFO:tasks.workunit.client.0.smithi027.stdout:7/742: mknod d4/d1b/d4a/d7a/cee 0 2024-04-20T10:59:24.161 INFO:tasks.workunit.client.0.smithi027.stdout:7/743: rename d4/d5/d8/d45/c57 to d4/d4d/cef 0 2024-04-20T10:59:24.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/855: dwrite d0/d11/d2c/d2e/d2b/d3d/f4c [0,4194304] 0 2024-04-20T10:59:24.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/856: write d0/d11/d2c/d2e/d2b/d7b/d12/fc1 [1426060,51683] 0 2024-04-20T10:59:24.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/857: truncate d0/d11/d2c/d2e/d2b/d7b/d12/fad 549552 0 2024-04-20T10:59:24.165 INFO:tasks.workunit.client.0.smithi027.stdout:0/858: unlink d0/d11/d2c/c6d 0 2024-04-20T10:59:24.167 INFO:tasks.workunit.client.0.smithi027.stdout:0/859: symlink d0/d11/d2c/d2e/d2b/l127 0 2024-04-20T10:59:24.168 INFO:tasks.workunit.client.0.smithi027.stdout:0/860: getdents d0/d59/d9e 0 2024-04-20T10:59:24.168 INFO:tasks.workunit.client.0.smithi027.stdout:0/861: chown d0/d11/d2c/d2e/d61/f78 15 1 2024-04-20T10:59:24.169 INFO:tasks.workunit.client.0.smithi027.stdout:7/744: dread d4/f2e [0,4194304] 0 2024-04-20T10:59:24.171 INFO:tasks.workunit.client.0.smithi027.stdout:7/745: symlink d4/d1b/d4a/lf0 0 2024-04-20T10:59:24.171 INFO:tasks.workunit.client.0.smithi027.stdout:7/746: write d4/d5/d8/d45/f71 [929859,39800] 0 2024-04-20T10:59:24.171 INFO:tasks.workunit.client.0.smithi027.stdout:0/862: dread d0/d59/d9e/fb7 [0,4194304] 0 2024-04-20T10:59:24.172 INFO:tasks.workunit.client.0.smithi027.stdout:7/747: symlink d4/d5/lf1 0 2024-04-20T10:59:24.173 INFO:tasks.workunit.client.0.smithi027.stdout:0/863: creat d0/d11/d2c/d2e/d55/f128 x:0 0 0 2024-04-20T10:59:24.173 INFO:tasks.workunit.client.0.smithi027.stdout:0/864: write d0/d59/f73 [900229,90653] 0 2024-04-20T10:59:24.173 INFO:tasks.workunit.client.0.smithi027.stdout:0/865: truncate d0/d11/d4b/d5b/d6f/d82/dd1/deb/f100 875014 0 2024-04-20T10:59:24.174 INFO:tasks.workunit.client.0.smithi027.stdout:0/866: dread d0/d11/d2c/d2e/d2b/d7b/d64/fb4 [0,4194304] 0 2024-04-20T10:59:24.174 INFO:tasks.workunit.client.0.smithi027.stdout:7/748: mknod d4/d1b/d4a/cf2 0 2024-04-20T10:59:24.176 INFO:tasks.workunit.client.0.smithi027.stdout:0/867: unlink d0/d11/d2c/d2e/d2b/d3d/f7e 0 2024-04-20T10:59:24.176 INFO:tasks.workunit.client.0.smithi027.stdout:6/807: sync 2024-04-20T10:59:24.181 INFO:tasks.workunit.client.0.smithi027.stdout:5/693: dwrite d19/d1b/d1c/d1e/d58/f5e [0,4194304] 0 2024-04-20T10:59:24.182 INFO:tasks.workunit.client.0.smithi027.stdout:4/742: dwrite d0/d14/d6b/da3/d1a/ff4 [0,4194304] 0 2024-04-20T10:59:24.182 INFO:tasks.workunit.client.0.smithi027.stdout:1/930: dwrite d3/f1b [4194304,4194304] 0 2024-04-20T10:59:24.182 INFO:tasks.workunit.client.0.smithi027.stdout:0/868: creat d0/d11/d2c/d2e/d2b/d7b/d64/f129 x:0 0 0 2024-04-20T10:59:24.183 INFO:tasks.workunit.client.0.smithi027.stdout:5/694: mknod d19/d1b/d1c/d1e/d22/d68/ce2 0 2024-04-20T10:59:24.185 INFO:tasks.workunit.client.0.smithi027.stdout:4/743: creat d0/d5e/d6c/f10d x:0 0 0 2024-04-20T10:59:24.186 INFO:tasks.workunit.client.0.smithi027.stdout:8/806: dwrite d9/de/da3/d3c/f61 [0,4194304] 0 2024-04-20T10:59:24.186 INFO:tasks.workunit.client.0.smithi027.stdout:8/807: truncate d9/de/da3/d3c/f9c 2234898 0 2024-04-20T10:59:24.189 INFO:tasks.workunit.client.0.smithi027.stdout:5/695: write d19/d1b/d1c/d1e/d22/db1/f9f [1810838,16039] 0 2024-04-20T10:59:24.192 INFO:tasks.workunit.client.0.smithi027.stdout:6/808: write da/d21/d28/fbb [3908184,103457] 0 2024-04-20T10:59:24.193 INFO:tasks.workunit.client.0.smithi027.stdout:1/931: symlink d3/d7/d38/ddf/de9/l150 0 2024-04-20T10:59:24.193 INFO:tasks.workunit.client.0.smithi027.stdout:1/932: stat d3/d20/d107/f14d 0 2024-04-20T10:59:24.193 INFO:tasks.workunit.client.0.smithi027.stdout:1/933: truncate d3/d41/f80 549241 0 2024-04-20T10:59:24.194 INFO:tasks.workunit.client.0.smithi027.stdout:0/869: creat d0/d11/f12a x:0 0 0 2024-04-20T10:59:24.194 INFO:tasks.workunit.client.0.smithi027.stdout:0/870: write d0/d11/d2c/d2e/d2b/d7b/d12/f44 [2986283,17356] 0 2024-04-20T10:59:24.194 INFO:tasks.workunit.client.0.smithi027.stdout:7/749: getdents d4/d1b/d4a 0 2024-04-20T10:59:24.195 INFO:tasks.workunit.client.0.smithi027.stdout:4/744: rename d0/d14/d6b/da3/d1a/d3b/l6d to d0/d5e/d6c/l10e 0 2024-04-20T10:59:24.200 INFO:tasks.workunit.client.0.smithi027.stdout:6/809: symlink da/d21/dab/l105 0 2024-04-20T10:59:24.200 INFO:tasks.workunit.client.0.smithi027.stdout:5/696: read d19/d1b/d1c/d1e/f88 [3951411,4067] 0 2024-04-20T10:59:24.200 INFO:tasks.workunit.client.0.smithi027.stdout:0/871: link d0/d11/d2c/d2e/d2b/d3d/f84 d0/d11/d2c/d2e/d2b/d7b/d34/d5a/f12b 0 2024-04-20T10:59:24.200 INFO:tasks.workunit.client.0.smithi027.stdout:0/872: stat d0/d11/d2c/d2e/d61/lec 0 2024-04-20T10:59:24.200 INFO:tasks.workunit.client.0.smithi027.stdout:0/873: write d0/d11/d2c/ff7 [82285,22940] 0 2024-04-20T10:59:24.200 INFO:tasks.workunit.client.0.smithi027.stdout:7/750: mknod d4/d5/d8/d46/cf3 0 2024-04-20T10:59:24.202 INFO:tasks.workunit.client.0.smithi027.stdout:6/810: rename da/d21/d28/d2d/d53/f76 to da/d21/d96/f106 0 2024-04-20T10:59:24.202 INFO:tasks.workunit.client.0.smithi027.stdout:6/811: chown c5 12 1 2024-04-20T10:59:24.203 INFO:tasks.workunit.client.0.smithi027.stdout:0/874: mknod d0/d11/d4b/d5b/d6f/d82/c12c 0 2024-04-20T10:59:24.205 INFO:tasks.workunit.client.0.smithi027.stdout:6/812: unlink da/d21/dab/d7e/d86/dcd/lec 0 2024-04-20T10:59:24.206 INFO:tasks.workunit.client.0.smithi027.stdout:0/875: mknod d0/d11/d2c/d2e/d2b/d7b/d12/c12d 0 2024-04-20T10:59:24.208 INFO:tasks.workunit.client.0.smithi027.stdout:6/813: rename da/d21/dab/d19/fea to da/d21/dab/d7e/d86/da3/f107 0 2024-04-20T10:59:24.208 INFO:tasks.workunit.client.0.smithi027.stdout:6/814: chown da/d21/dab/d19/cc0 5876409 1 2024-04-20T10:59:24.210 INFO:tasks.workunit.client.0.smithi027.stdout:0/876: truncate d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/f8a 185919 0 2024-04-20T10:59:24.210 INFO:tasks.workunit.client.0.smithi027.stdout:0/877: fdatasync d0/d11/d4b/d5b/d6f/fe7 0 2024-04-20T10:59:24.211 INFO:tasks.workunit.client.0.smithi027.stdout:6/815: getdents da/d21 0 2024-04-20T10:59:24.211 INFO:tasks.workunit.client.0.smithi027.stdout:6/816: dread - da/d21/dab/d19/ffc zero size 2024-04-20T10:59:24.212 INFO:tasks.workunit.client.0.smithi027.stdout:5/697: dread fb [0,4194304] 0 2024-04-20T10:59:24.214 INFO:tasks.workunit.client.0.smithi027.stdout:0/878: creat d0/d11/d2c/d2e/d35/d8c/f12e x:0 0 0 2024-04-20T10:59:24.214 INFO:tasks.workunit.client.0.smithi027.stdout:0/879: dread - d0/d11/d2c/d2e/d2b/d7b/d12/f111 zero size 2024-04-20T10:59:24.214 INFO:tasks.workunit.client.0.smithi027.stdout:0/880: dread - d0/d11/d2c/d2e/d2b/d7b/d12/d25/ff4 zero size 2024-04-20T10:59:24.216 INFO:tasks.workunit.client.0.smithi027.stdout:7/751: write d4/fb4 [3541541,5875] 0 2024-04-20T10:59:24.216 INFO:tasks.workunit.client.0.smithi027.stdout:7/752: write d4/d5/d8/d21/fbb [83438,44546] 0 2024-04-20T10:59:24.217 INFO:tasks.workunit.client.0.smithi027.stdout:5/698: creat d19/d1b/d1c/d1e/d58/db4/fe3 x:0 0 0 2024-04-20T10:59:24.219 INFO:tasks.workunit.client.0.smithi027.stdout:8/808: dwrite d9/de/d1e/d4a/d6d/db7/d97/fa4 [0,4194304] 0 2024-04-20T10:59:24.220 INFO:tasks.workunit.client.0.smithi027.stdout:1/934: dwrite d3/d7/ffd [0,4194304] 0 2024-04-20T10:59:24.220 INFO:tasks.workunit.client.0.smithi027.stdout:1/935: dread - d3/d7/d28/d35/f11e zero size 2024-04-20T10:59:24.222 INFO:tasks.workunit.client.0.smithi027.stdout:7/753: symlink d4/d5/d8/d21/d94/da7/dce/lf4 0 2024-04-20T10:59:24.222 INFO:tasks.workunit.client.0.smithi027.stdout:7/754: dread - d4/d5/d8/d21/d94/fbd zero size 2024-04-20T10:59:24.224 INFO:tasks.workunit.client.0.smithi027.stdout:4/745: dwrite d0/d14/d6b/da3/d18/f54 [0,4194304] 0 2024-04-20T10:59:24.226 INFO:tasks.workunit.client.0.smithi027.stdout:1/936: truncate d3/d45/fb6 626777 0 2024-04-20T10:59:24.227 INFO:tasks.workunit.client.0.smithi027.stdout:8/809: write d9/fd [7819802,69615] 0 2024-04-20T10:59:24.228 INFO:tasks.workunit.client.0.smithi027.stdout:8/810: truncate d9/de/d1e/d4a/d6d/d79/feb 1830185 0 2024-04-20T10:59:24.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/755: truncate d4/d5/d8/f38 380147 0 2024-04-20T10:59:24.232 INFO:tasks.workunit.client.0.smithi027.stdout:4/746: rename d0/f37 to d0/d14/d71/dfd/f10f 0 2024-04-20T10:59:24.232 INFO:tasks.workunit.client.0.smithi027.stdout:7/756: dread d4/d1b/d58/fd2 [0,4194304] 0 2024-04-20T10:59:24.233 INFO:tasks.workunit.client.0.smithi027.stdout:1/937: mknod d3/d7/d28/db0/d11c/c151 0 2024-04-20T10:59:24.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/811: rename d9/de/f1a to d9/de/d1e/d4a/d6d/d79/f112 0 2024-04-20T10:59:24.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/812: fdatasync d9/de/d1e/d4a/d6d/dea/d59/ffc 0 2024-04-20T10:59:24.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/813: fsync d9/f15 0 2024-04-20T10:59:24.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/814: chown d9/de/d1e/d4a/d6d/db7/f102 11414 1 2024-04-20T10:59:24.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/815: truncate d9/de/d1e/d4a/d6d/dea/d59/da0/dbd/ff3 332098 0 2024-04-20T10:59:24.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/747: link d0/d14/d6b/da3/d18/f4b d0/d14/d6b/da3/d18/d10a/f110 0 2024-04-20T10:59:24.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/748: fdatasync d0/d14/d6b/da3/d1b/fa5 0 2024-04-20T10:59:24.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/749: fdatasync d0/d14/d6b/da3/d1b/d33/fd0 0 2024-04-20T10:59:24.238 INFO:tasks.workunit.client.0.smithi027.stdout:7/757: mknod d4/d5/d8/d21/d94/da7/dce/cf5 0 2024-04-20T10:59:24.238 INFO:tasks.workunit.client.0.smithi027.stdout:7/758: readlink d4/d1b/d4a/lf0 0 2024-04-20T10:59:24.238 INFO:tasks.workunit.client.0.smithi027.stdout:7/759: write d4/d5/d81/fe8 [287747,126134] 0 2024-04-20T10:59:24.238 INFO:tasks.workunit.client.0.smithi027.stdout:6/817: dwrite da/d21/dab/f4f [0,4194304] 0 2024-04-20T10:59:24.239 INFO:tasks.workunit.client.0.smithi027.stdout:4/750: symlink d0/d14/d6b/da3/d18/d10a/l111 0 2024-04-20T10:59:24.240 INFO:tasks.workunit.client.0.smithi027.stdout:1/938: read d3/d7/d28/d92/f109 [3816898,122177] 0 2024-04-20T10:59:24.240 INFO:tasks.workunit.client.0.smithi027.stdout:7/760: creat d4/d1b/ff6 x:0 0 0 2024-04-20T10:59:24.240 INFO:tasks.workunit.client.0.smithi027.stdout:7/761: truncate d4/d5/d8/d21/d94/da7/dce/fdf 603792 0 2024-04-20T10:59:24.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/881: dwrite d0/d11/d2c/d2e/d2b/d7b/d34/f3b [4194304,4194304] 0 2024-04-20T10:59:24.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/818: unlink da/d21/dab/c98 0 2024-04-20T10:59:24.243 INFO:tasks.workunit.client.0.smithi027.stdout:7/762: creat d4/d5/d8/d45/ff7 x:0 0 0 2024-04-20T10:59:24.243 INFO:tasks.workunit.client.0.smithi027.stdout:8/816: dread d9/de/d1e/d4a/f35 [0,4194304] 0 2024-04-20T10:59:24.244 INFO:tasks.workunit.client.0.smithi027.stdout:0/882: dread d0/f42 [0,4194304] 0 2024-04-20T10:59:24.246 INFO:tasks.workunit.client.0.smithi027.stdout:5/699: dwrite f17 [4194304,4194304] 0 2024-04-20T10:59:24.246 INFO:tasks.workunit.client.0.smithi027.stdout:5/700: dread - d19/d1b/d1c/dbd/dbe/fb6 zero size 2024-04-20T10:59:24.246 INFO:tasks.workunit.client.0.smithi027.stdout:5/701: fdatasync d19/d1b/d1c/d1e/d25/fc0 0 2024-04-20T10:59:24.246 INFO:tasks.workunit.client.0.smithi027.stdout:8/817: dread d9/de/d1e/d4a/d6d/dea/d59/da0/dbd/ff3 [0,4194304] 0 2024-04-20T10:59:24.254 INFO:tasks.workunit.client.0.smithi027.stdout:6/819: rename da/d21/dab/d19/d2e to da/dcc/d108 0 2024-04-20T10:59:24.254 INFO:tasks.workunit.client.0.smithi027.stdout:4/751: write d0/fd [1851724,48141] 0 2024-04-20T10:59:24.255 INFO:tasks.workunit.client.0.smithi027.stdout:7/763: mknod d4/d1b/d4a/d62/d87/cf8 0 2024-04-20T10:59:24.255 INFO:tasks.workunit.client.0.smithi027.stdout:0/883: creat d0/d59/dbf/f12f x:0 0 0 2024-04-20T10:59:24.256 INFO:tasks.workunit.client.0.smithi027.stdout:8/818: getdents d9/de/d20/d53 0 2024-04-20T10:59:24.257 INFO:tasks.workunit.client.0.smithi027.stdout:6/820: mkdir da/d21/d109 0 2024-04-20T10:59:24.258 INFO:tasks.workunit.client.0.smithi027.stdout:6/821: readlink da/d21/dab/lbd 0 2024-04-20T10:59:24.258 INFO:tasks.workunit.client.0.smithi027.stdout:4/752: dread d0/d14/d6b/da3/d18/d10a/f110 [0,4194304] 0 2024-04-20T10:59:24.258 INFO:tasks.workunit.client.0.smithi027.stdout:4/753: stat d0/d14/d6b/da3/d1b/d33/d5c/d6f/f74 0 2024-04-20T10:59:24.259 INFO:tasks.workunit.client.0.smithi027.stdout:0/884: creat d0/d11/d2c/d2e/d2b/d7b/d34/d5a/f130 x:0 0 0 2024-04-20T10:59:24.259 INFO:tasks.workunit.client.0.smithi027.stdout:8/819: creat d9/de/d1e/d4a/d6d/dea/f113 x:0 0 0 2024-04-20T10:59:24.261 INFO:tasks.workunit.client.0.smithi027.stdout:6/822: symlink da/dcc/d108/d6b/de0/l10a 0 2024-04-20T10:59:24.261 INFO:tasks.workunit.client.0.smithi027.stdout:6/823: write da/d21/f5b [3289397,113397] 0 2024-04-20T10:59:24.261 INFO:tasks.workunit.client.0.smithi027.stdout:6/824: readlink da/db8/lc5 0 2024-04-20T10:59:24.274 INFO:tasks.workunit.client.0.smithi027.stdout:0/885: dread d0/d11/d2c/da1/fe3 [0,4194304] 0 2024-04-20T10:59:24.274 INFO:tasks.workunit.client.0.smithi027.stdout:0/886: dread - d0/d11/d2c/d2e/d55/f128 zero size 2024-04-20T10:59:24.274 INFO:tasks.workunit.client.0.smithi027.stdout:1/939: dwrite d3/d7/d28/d92/fb5 [0,4194304] 0 2024-04-20T10:59:24.275 INFO:tasks.workunit.client.0.smithi027.stdout:0/887: dread d0/d11/d2c/d2e/f2a [0,4194304] 0 2024-04-20T10:59:24.277 INFO:tasks.workunit.client.0.smithi027.stdout:5/702: dwrite f17 [4194304,4194304] 0 2024-04-20T10:59:24.277 INFO:tasks.workunit.client.0.smithi027.stdout:7/764: dwrite d4/d1b/fb6 [0,4194304] 0 2024-04-20T10:59:24.278 INFO:tasks.workunit.client.0.smithi027.stdout:4/754: stat d0/d14/d6b/da3/d1b/d33/d5c/d6f/l8c 0 2024-04-20T10:59:24.278 INFO:tasks.workunit.client.0.smithi027.stdout:8/820: mknod d9/de/d18/dff/c114 0 2024-04-20T10:59:24.278 INFO:tasks.workunit.client.0.smithi027.stdout:3/749: sync 2024-04-20T10:59:24.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/666: sync 2024-04-20T10:59:24.279 INFO:tasks.workunit.client.0.smithi027.stdout:3/750: getdents d2/dd/d57/de7 0 2024-04-20T10:59:24.279 INFO:tasks.workunit.client.0.smithi027.stdout:2/716: sync 2024-04-20T10:59:24.279 INFO:tasks.workunit.client.0.smithi027.stdout:5/703: creat d19/d1b/d1c/d1e/d58/db4/fe4 x:0 0 0 2024-04-20T10:59:24.282 INFO:tasks.workunit.client.0.smithi027.stdout:0/888: dread d0/d11/d2c/d2e/d55/fac [0,4194304] 0 2024-04-20T10:59:24.283 INFO:tasks.workunit.client.0.smithi027.stdout:0/889: fsync d0/d11/d2c/d2e/d2b/d7b/d12/fad 0 2024-04-20T10:59:24.283 INFO:tasks.workunit.client.0.smithi027.stdout:0/890: chown d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f86 56 1 2024-04-20T10:59:24.283 INFO:tasks.workunit.client.0.smithi027.stdout:0/891: fsync d0/d11/d2c/d2e/d2b/d7b/d34/d5a/fcc 0 2024-04-20T10:59:24.283 INFO:tasks.workunit.client.0.smithi027.stdout:7/765: link d4/d5/d8/d3b/f3e d4/d1b/ff9 0 2024-04-20T10:59:24.283 INFO:tasks.workunit.client.0.smithi027.stdout:2/717: dread d5/d26/d78/f7b [0,4194304] 0 2024-04-20T10:59:24.284 INFO:tasks.workunit.client.0.smithi027.stdout:4/755: creat d0/d14/d6b/da3/d1a/f112 x:0 0 0 2024-04-20T10:59:24.285 INFO:tasks.workunit.client.0.smithi027.stdout:8/821: link d9/de/d1e/d4a/d6d/dea/d59/d74/de8/c10f d9/de/d1e/d4a/d6d/dea/d59/c115 0 2024-04-20T10:59:24.286 INFO:tasks.workunit.client.0.smithi027.stdout:5/704: mkdir d19/d1b/d1c/d1e/d25/de5 0 2024-04-20T10:59:24.287 INFO:tasks.workunit.client.0.smithi027.stdout:0/892: rename d0/d11/d2c/d2e/d2b/d3d/f4c to d0/d11/d4b/d5b/d6f/d82/dd3/f131 0 2024-04-20T10:59:24.287 INFO:tasks.workunit.client.0.smithi027.stdout:0/893: stat d0/d11/d2c/fda 0 2024-04-20T10:59:24.287 INFO:tasks.workunit.client.0.smithi027.stdout:0/894: dread - d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f109 zero size 2024-04-20T10:59:24.287 INFO:tasks.workunit.client.0.smithi027.stdout:7/766: truncate d4/f3d 14429 0 2024-04-20T10:59:24.289 INFO:tasks.workunit.client.0.smithi027.stdout:2/718: symlink d5/dc/d16/d32/dab/le7 0 2024-04-20T10:59:24.292 INFO:tasks.workunit.client.0.smithi027.stdout:4/756: unlink d0/d14/d6b/da3/d1b/d33/ld3 0 2024-04-20T10:59:24.296 INFO:tasks.workunit.client.0.smithi027.stdout:0/895: truncate d0/d11/d4b/fcb 260194 0 2024-04-20T10:59:24.296 INFO:tasks.workunit.client.0.smithi027.stdout:8/822: truncate d9/de/d1e/d4a/d6d/d79/fe4 2383387 0 2024-04-20T10:59:24.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/719: creat d5/dc/d16/d2a/d59/da4/de1/fe8 x:0 0 0 2024-04-20T10:59:24.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/720: fdatasync d5/dc/d16/d37/de2/fbf 0 2024-04-20T10:59:24.296 INFO:tasks.workunit.client.0.smithi027.stdout:8/823: write d9/de/d1e/d4a/d6d/dea/d59/da0/dbd/ff3 [395442,105901] 0 2024-04-20T10:59:24.299 INFO:tasks.workunit.client.0.smithi027.stdout:4/757: creat d0/d14/d6b/da3/d53/f113 x:0 0 0 2024-04-20T10:59:24.299 INFO:tasks.workunit.client.0.smithi027.stdout:0/896: unlink d0/d11/d5c/df0/lff 0 2024-04-20T10:59:24.302 INFO:tasks.workunit.client.0.smithi027.stdout:1/940: dwrite d3/d7/d28/db0/f138 [0,4194304] 0 2024-04-20T10:59:24.305 INFO:tasks.workunit.client.0.smithi027.stdout:1/941: rmdir d3/d7/d38/ddf 39 2024-04-20T10:59:24.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/758: dwrite d0/d14/d6b/da3/d1b/d33/d5c/d6f/dd1/f102 [0,4194304] 0 2024-04-20T10:59:24.323 INFO:tasks.workunit.client.0.smithi027.stdout:3/751: dwrite d2/dd/f3d [0,4194304] 0 2024-04-20T10:59:24.327 INFO:tasks.workunit.client.0.smithi027.stdout:5/705: dwrite d19/d1b/d1c/d1e/d3c/fcd [0,4194304] 0 2024-04-20T10:59:24.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/752: dread d2/d94/f95 [0,4194304] 0 2024-04-20T10:59:24.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/753: fsync d2/dd/d57/fd4 0 2024-04-20T10:59:24.329 INFO:tasks.workunit.client.0.smithi027.stdout:7/767: dwrite d4/fc [0,4194304] 0 2024-04-20T10:59:24.329 INFO:tasks.workunit.client.0.smithi027.stdout:9/667: dwrite d4/d74/d72/f98 [0,4194304] 0 2024-04-20T10:59:24.333 INFO:tasks.workunit.client.0.smithi027.stdout:5/706: rename d19/d1b/d1c/d1e/d22/db1/f50 to d19/d1b/d1c/dbd/dbe/fe6 0 2024-04-20T10:59:24.334 INFO:tasks.workunit.client.0.smithi027.stdout:4/759: dread d0/d14/d6b/da3/d53/fb4 [0,4194304] 0 2024-04-20T10:59:24.334 INFO:tasks.workunit.client.0.smithi027.stdout:7/768: symlink d4/d1b/d4a/lfa 0 2024-04-20T10:59:24.335 INFO:tasks.workunit.client.0.smithi027.stdout:5/707: truncate d19/d1b/d1c/d1e/f88 2964707 0 2024-04-20T10:59:24.336 INFO:tasks.workunit.client.0.smithi027.stdout:5/708: write d19/d1b/d1c/d1e/d3c/fd2 [535668,32630] 0 2024-04-20T10:59:24.336 INFO:tasks.workunit.client.0.smithi027.stdout:5/709: readlink d19/d1b/d1c/d1e/d25/l37 0 2024-04-20T10:59:24.336 INFO:tasks.workunit.client.0.smithi027.stdout:5/710: fsync d19/d1b/d1c/d1e/d83/dcc/fd8 0 2024-04-20T10:59:24.336 INFO:tasks.workunit.client.0.smithi027.stdout:5/711: write d19/d1b/d1c/dbd/dbe/fb6 [832329,32165] 0 2024-04-20T10:59:24.336 INFO:tasks.workunit.client.0.smithi027.stdout:5/712: write d19/d1b/d1c/d1e/d22/db1/dcf/fa2 [1036787,6273] 0 2024-04-20T10:59:24.336 INFO:tasks.workunit.client.0.smithi027.stdout:3/754: link d2/d18/d2c/d4c/f7e d2/dd/d57/d8d/feb 0 2024-04-20T10:59:24.338 INFO:tasks.workunit.client.0.smithi027.stdout:5/713: rename d19/d1b/d1c/d1e/d25/fa5 to d19/d1b/d1c/d1e/d25/d40/fe7 0 2024-04-20T10:59:24.338 INFO:tasks.workunit.client.0.smithi027.stdout:5/714: read d19/d1b/d1c/d1e/d22/db1/f55 [527836,104677] 0 2024-04-20T10:59:24.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/897: dwrite d0/d11/d4b/d5b/d6f/d82/dd1/deb/ff9 [0,4194304] 0 2024-04-20T10:59:24.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/898: readlink d0/d11/d2c/d2e/d55/lf6 0 2024-04-20T10:59:24.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/899: dread - d0/d11/d2c/d2e/d2b/d7b/dcf/f126 zero size 2024-04-20T10:59:24.338 INFO:tasks.workunit.client.0.smithi027.stdout:0/900: chown d0/d11/d2c/d2e/d2b/d7b/f26 0 1 2024-04-20T10:59:24.338 INFO:tasks.workunit.client.0.smithi027.stdout:3/755: rmdir d2/d18/d2c/d4c/d67/d6c/dd0 39 2024-04-20T10:59:24.340 INFO:tasks.workunit.client.0.smithi027.stdout:5/715: creat d19/d1b/d1c/d1e/d22/fe8 x:0 0 0 2024-04-20T10:59:24.342 INFO:tasks.workunit.client.0.smithi027.stdout:8/824: dwrite d9/de/da3/f7f [0,4194304] 0 2024-04-20T10:59:24.342 INFO:tasks.workunit.client.0.smithi027.stdout:8/825: write d9/de/d1e/d4a/d6d/db7/d5b/ff7 [1927033,66391] 0 2024-04-20T10:59:24.342 INFO:tasks.workunit.client.0.smithi027.stdout:1/942: dwrite d3/d7/f57 [0,4194304] 0 2024-04-20T10:59:24.342 INFO:tasks.workunit.client.0.smithi027.stdout:1/943: chown d3/cc2 8996 1 2024-04-20T10:59:24.342 INFO:tasks.workunit.client.0.smithi027.stdout:0/901: getdents d0/d11/d2c/d2e/d2b/d7b/d34/d5a 0 2024-04-20T10:59:24.343 INFO:tasks.workunit.client.0.smithi027.stdout:3/756: mknod d2/d18/d2c/d4c/d67/d6c/dd0/dd5/cec 0 2024-04-20T10:59:24.344 INFO:tasks.workunit.client.0.smithi027.stdout:5/716: getdents d19/d1b/d1c/d1e/d22/db1/dcf/d91 0 2024-04-20T10:59:24.346 INFO:tasks.workunit.client.0.smithi027.stdout:8/826: symlink d9/de/d20/dbf/l116 0 2024-04-20T10:59:24.349 INFO:tasks.workunit.client.0.smithi027.stdout:1/944: unlink d3/d7/d38/ddf/d73/le8 0 2024-04-20T10:59:24.349 INFO:tasks.workunit.client.0.smithi027.stdout:0/902: creat d0/d11/d2c/da1/f132 x:0 0 0 2024-04-20T10:59:24.349 INFO:tasks.workunit.client.0.smithi027.stdout:0/903: fdatasync d0/d11/d2c/d2e/d2b/d7b/d34/f3b 0 2024-04-20T10:59:24.349 INFO:tasks.workunit.client.0.smithi027.stdout:3/757: creat d2/d18/d2c/d4c/d67/d6c/dd0/dd5/fed x:0 0 0 2024-04-20T10:59:24.349 INFO:tasks.workunit.client.0.smithi027.stdout:3/758: stat d2/d18/d2c/lb2 0 2024-04-20T10:59:24.350 INFO:tasks.workunit.client.0.smithi027.stdout:1/945: rmdir d3/d7e/d123 39 2024-04-20T10:59:24.350 INFO:tasks.workunit.client.0.smithi027.stdout:1/946: write d3/d7/d38/ddf/fde [718064,7749] 0 2024-04-20T10:59:24.350 INFO:tasks.workunit.client.0.smithi027.stdout:3/759: creat d2/dd/d57/d8c/fee x:0 0 0 2024-04-20T10:59:24.351 INFO:tasks.workunit.client.0.smithi027.stdout:3/760: creat d2/dd/df/d23/fef x:0 0 0 2024-04-20T10:59:24.351 INFO:tasks.workunit.client.0.smithi027.stdout:3/761: stat d2/d7f 0 2024-04-20T10:59:24.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/762: mkdir d2/dd/df/d23/df0 0 2024-04-20T10:59:24.353 INFO:tasks.workunit.client.0.smithi027.stdout:9/668: dread d4/d1b/f22 [0,4194304] 0 2024-04-20T10:59:24.353 INFO:tasks.workunit.client.0.smithi027.stdout:3/763: mkdir d2/d18/d2c/df1 0 2024-04-20T10:59:24.354 INFO:tasks.workunit.client.0.smithi027.stdout:9/669: mknod d4/d1b/daf/cc6 0 2024-04-20T10:59:24.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/670: creat d4/d14/d43/fc7 x:0 0 0 2024-04-20T10:59:24.364 INFO:tasks.workunit.client.0.smithi027.stdout:8/827: write d9/de/d1e/d4a/d6d/d79/f112 [7684986,10584] 0 2024-04-20T10:59:24.364 INFO:tasks.workunit.client.0.smithi027.stdout:8/828: chown d9/de/d20/d95/lac 14770196 1 2024-04-20T10:59:24.365 INFO:tasks.workunit.client.0.smithi027.stdout:9/671: dread d4/d74/d72/f98 [0,4194304] 0 2024-04-20T10:59:24.368 INFO:tasks.workunit.client.0.smithi027.stdout:4/760: dwrite d0/d14/d6b/da3/d1b/d2f/ff6 [0,4194304] 0 2024-04-20T10:59:24.368 INFO:tasks.workunit.client.0.smithi027.stdout:0/904: dread d0/d59/d9e/f10b [0,4194304] 0 2024-04-20T10:59:24.368 INFO:tasks.workunit.client.0.smithi027.stdout:4/761: chown d0/d14/d6b/d92/dcd/cdd 916065 1 2024-04-20T10:59:24.368 INFO:tasks.workunit.client.0.smithi027.stdout:0/905: dread - d0/d59/dbf/f12f zero size 2024-04-20T10:59:24.369 INFO:tasks.workunit.client.0.smithi027.stdout:7/769: dwrite d4/d5/f40 [0,4194304] 0 2024-04-20T10:59:24.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/770: mkdir d4/da2/dd1/dfb 0 2024-04-20T10:59:24.374 INFO:tasks.workunit.client.0.smithi027.stdout:0/906: mkdir d0/d11/d5c/df0/d133 0 2024-04-20T10:59:24.374 INFO:tasks.workunit.client.0.smithi027.stdout:7/771: dread - d4/f98 zero size 2024-04-20T10:59:24.374 INFO:tasks.workunit.client.0.smithi027.stdout:7/772: readlink d4/d5/d8/d21/l2c 0 2024-04-20T10:59:24.374 INFO:tasks.workunit.client.0.smithi027.stdout:7/773: creat d4/d1b/d4a/d62/d87/ffc x:0 0 0 2024-04-20T10:59:24.376 INFO:tasks.workunit.client.0.smithi027.stdout:9/672: dread d4/d14/d43/f52 [0,4194304] 0 2024-04-20T10:59:24.377 INFO:tasks.workunit.client.0.smithi027.stdout:7/774: dread d4/d5/f33 [0,4194304] 0 2024-04-20T10:59:24.390 INFO:tasks.workunit.client.0.smithi027.stdout:1/947: dwrite d3/d7/d38/da4/dac/f11a [0,4194304] 0 2024-04-20T10:59:24.390 INFO:tasks.workunit.client.0.smithi027.stdout:1/948: stat d3/d7/d38/da4/dac/f11a 0 2024-04-20T10:59:24.401 INFO:tasks.workunit.client.0.smithi027.stdout:9/673: read d4/d14/d43/d91/f96 [1497739,126029] 0 2024-04-20T10:59:24.402 INFO:tasks.workunit.client.0.smithi027.stdout:5/717: dwrite d19/d1b/d1c/d1e/d58/d9d/fe0 [0,4194304] 0 2024-04-20T10:59:24.402 INFO:tasks.workunit.client.0.smithi027.stdout:8/829: dwrite d9/de/d1e/d4a/d6d/dea/d59/da0/dbd/ff3 [0,4194304] 0 2024-04-20T10:59:24.403 INFO:tasks.workunit.client.0.smithi027.stdout:9/674: creat d4/fc8 x:0 0 0 2024-04-20T10:59:24.404 INFO:tasks.workunit.client.0.smithi027.stdout:4/762: dwrite d0/d14/d6b/da3/d1b/f21 [0,4194304] 0 2024-04-20T10:59:24.404 INFO:tasks.workunit.client.0.smithi027.stdout:3/764: dwrite d2/dd/f1a [4194304,4194304] 0 2024-04-20T10:59:24.404 INFO:tasks.workunit.client.0.smithi027.stdout:0/907: dwrite d0/d11/d2c/d2e/d2b/d7b/d64/f7c [4194304,4194304] 0 2024-04-20T10:59:24.405 INFO:tasks.workunit.client.0.smithi027.stdout:0/908: write d0/d11/d2c/d2e/d2b/d7b/d12/f3e [2089461,57802] 0 2024-04-20T10:59:24.405 INFO:tasks.workunit.client.0.smithi027.stdout:5/718: unlink d19/d1b/d1c/d1e/d25/d40/c8c 0 2024-04-20T10:59:24.405 INFO:tasks.workunit.client.0.smithi027.stdout:5/719: chown d19/d1b/d1c/d1e/d22/d68/l80 56 1 2024-04-20T10:59:24.406 INFO:tasks.workunit.client.0.smithi027.stdout:7/775: dwrite d4/d5/d8/d46/d77/fcf [0,4194304] 0 2024-04-20T10:59:24.406 INFO:tasks.workunit.client.0.smithi027.stdout:7/776: fsync d4/d1b/d58/fe2 0 2024-04-20T10:59:24.409 INFO:tasks.workunit.client.0.smithi027.stdout:1/949: dwrite d3/d7/f8 [4194304,4194304] 0 2024-04-20T10:59:24.410 INFO:tasks.workunit.client.0.smithi027.stdout:8/830: mknod d9/de/d20/dfd/c117 0 2024-04-20T10:59:24.410 INFO:tasks.workunit.client.0.smithi027.stdout:8/831: dread - d9/de/d1e/d4a/d6d/dea/d62/fe5 zero size 2024-04-20T10:59:24.410 INFO:tasks.workunit.client.0.smithi027.stdout:8/832: write d9/de/f3a [2418769,62933] 0 2024-04-20T10:59:24.410 INFO:tasks.workunit.client.0.smithi027.stdout:9/675: creat d4/d1b/d1f/d5a/fc9 x:0 0 0 2024-04-20T10:59:24.412 INFO:tasks.workunit.client.0.smithi027.stdout:4/763: rename d0/d14/d6b/da3/d18/d10a/l84 to d0/d5e/d6c/l114 0 2024-04-20T10:59:24.423 INFO:tasks.workunit.client.0.smithi027.stdout:3/765: creat d2/dd/d22/ff2 x:0 0 0 2024-04-20T10:59:24.424 INFO:tasks.workunit.client.0.smithi027.stdout:5/720: stat d19/c23 0 2024-04-20T10:59:24.424 INFO:tasks.workunit.client.0.smithi027.stdout:5/721: write d19/d1b/d1c/f98 [4600546,11139] 0 2024-04-20T10:59:24.425 INFO:tasks.workunit.client.0.smithi027.stdout:1/950: creat d3/d7/d28/db0/d11c/f152 x:0 0 0 2024-04-20T10:59:24.425 INFO:tasks.workunit.client.0.smithi027.stdout:0/909: dwrite d0/d11/d4b/fcb [0,4194304] 0 2024-04-20T10:59:24.425 INFO:tasks.workunit.client.0.smithi027.stdout:4/764: dread d0/d14/d6b/da3/d18/db9/fd6 [0,4194304] 0 2024-04-20T10:59:24.425 INFO:tasks.workunit.client.0.smithi027.stdout:0/910: truncate d0/d11/d2c/fcd 1236231 0 2024-04-20T10:59:24.425 INFO:tasks.workunit.client.0.smithi027.stdout:0/911: fsync d0/d11/d5c/df0/f102 0 2024-04-20T10:59:24.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/833: mkdir d9/de/d1e/d4a/d6d/dea/d59/d74/de8/d118 0 2024-04-20T10:59:24.426 INFO:tasks.workunit.client.0.smithi027.stdout:4/765: dread d0/d5e/fba [0,4194304] 0 2024-04-20T10:59:24.427 INFO:tasks.workunit.client.0.smithi027.stdout:9/676: rename d4/d1b/d1f/l8d to d4/d1b/d1f/d45/d8a/db2/lca 0 2024-04-20T10:59:24.430 INFO:tasks.workunit.client.0.smithi027.stdout:3/766: truncate d2/d18/d2c/d4c/d67/fab 3464161 0 2024-04-20T10:59:24.431 INFO:tasks.workunit.client.0.smithi027.stdout:3/767: fdatasync d2/d18/d2c/f69 0 2024-04-20T10:59:24.431 INFO:tasks.workunit.client.0.smithi027.stdout:3/768: chown d2/dd/df/d23/d68/d88/f89 150 1 2024-04-20T10:59:24.431 INFO:tasks.workunit.client.0.smithi027.stdout:1/951: mknod d3/d7/d38/ddf/de9/c153 0 2024-04-20T10:59:24.433 INFO:tasks.workunit.client.0.smithi027.stdout:0/912: getdents d0/d11/d2c/d2e/d2b/d7b/d12 0 2024-04-20T10:59:24.434 INFO:tasks.workunit.client.0.smithi027.stdout:8/834: rmdir d9/de/d1e/d4a/d6d/d75 39 2024-04-20T10:59:24.434 INFO:tasks.workunit.client.0.smithi027.stdout:4/766: write d0/d14/d6b/da3/f2e [76958,56140] 0 2024-04-20T10:59:24.435 INFO:tasks.workunit.client.0.smithi027.stdout:9/677: rename d4/d74/d72/cc3 to d4/d14/da8/ccb 0 2024-04-20T10:59:24.437 INFO:tasks.workunit.client.0.smithi027.stdout:0/913: link d0/d11/d2c/d2e/d35/d8c/cc6 d0/d11/d5c/c134 0 2024-04-20T10:59:24.437 INFO:tasks.workunit.client.0.smithi027.stdout:0/914: chown d0/d11/d4b/d5b/d6f/d82/dd1/deb/f100 1 1 2024-04-20T10:59:24.440 INFO:tasks.workunit.client.0.smithi027.stdout:9/678: rmdir d4/d1b/d1f/d5a/db7/dc0 0 2024-04-20T10:59:24.440 INFO:tasks.workunit.client.0.smithi027.stdout:6/825: sync 2024-04-20T10:59:24.440 INFO:tasks.workunit.client.0.smithi027.stdout:2/721: sync 2024-04-20T10:59:24.440 INFO:tasks.workunit.client.0.smithi027.stdout:6/826: readlink da/l6c 0 2024-04-20T10:59:24.440 INFO:tasks.workunit.client.0.smithi027.stdout:6/827: dread - da/d21/dab/d1f/dbf/fc4 zero size 2024-04-20T10:59:24.441 INFO:tasks.workunit.client.0.smithi027.stdout:6/828: fsync da/d21/dab/d7e/d86/f104 0 2024-04-20T10:59:24.441 INFO:tasks.workunit.client.0.smithi027.stdout:9/679: read d4/d1b/d31/d23/f99 [2591801,110256] 0 2024-04-20T10:59:24.441 INFO:tasks.workunit.client.0.smithi027.stdout:9/680: chown d4/d1b/daf/cc6 2832164 1 2024-04-20T10:59:24.445 INFO:tasks.workunit.client.0.smithi027.stdout:9/681: rename d4/d14/d43/fa1 to d4/d14/d43/fcc 0 2024-04-20T10:59:24.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/722: dwrite d19/d1b/d1c/d1e/d25/fa0 [0,4194304] 0 2024-04-20T10:59:24.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/723: truncate f3 5364274 0 2024-04-20T10:59:24.445 INFO:tasks.workunit.client.0.smithi027.stdout:5/724: write d19/d1b/d1c/d1e/d22/db1/f6b [1865289,86491] 0 2024-04-20T10:59:24.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/682: rmdir d4/d1b/d1f/d57/db0/db3 0 2024-04-20T10:59:24.450 INFO:tasks.workunit.client.0.smithi027.stdout:5/725: dread f16 [0,4194304] 0 2024-04-20T10:59:24.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/683: rmdir d4/d14/d43/d91 39 2024-04-20T10:59:24.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/684: readlink d4/d1b/d1f/l65 0 2024-04-20T10:59:24.450 INFO:tasks.workunit.client.0.smithi027.stdout:3/769: fsync d2/dd/d57/d8c/fb8 0 2024-04-20T10:59:24.450 INFO:tasks.workunit.client.0.smithi027.stdout:3/770: write d2/d18/f76 [5142996,78037] 0 2024-04-20T10:59:24.451 INFO:tasks.workunit.client.0.smithi027.stdout:5/726: mkdir d19/d1b/d1c/d1e/d22/d68/d75/de9 0 2024-04-20T10:59:24.451 INFO:tasks.workunit.client.0.smithi027.stdout:8/835: dwrite d9/de/d1e/d4a/d6d/dea/d62/fe5 [0,4194304] 0 2024-04-20T10:59:24.452 INFO:tasks.workunit.client.0.smithi027.stdout:9/685: dread d4/d1b/d1f/d45/f83 [0,4194304] 0 2024-04-20T10:59:24.452 INFO:tasks.workunit.client.0.smithi027.stdout:9/686: fdatasync d4/fc8 0 2024-04-20T10:59:24.452 INFO:tasks.workunit.client.0.smithi027.stdout:3/771: creat d2/dd/d57/d8d/ff3 x:0 0 0 2024-04-20T10:59:24.452 INFO:tasks.workunit.client.0.smithi027.stdout:3/772: chown d2/dd/f19 29995 1 2024-04-20T10:59:24.455 INFO:tasks.workunit.client.0.smithi027.stdout:4/767: dwrite d0/d14/d6b/da3/d53/fb4 [0,4194304] 0 2024-04-20T10:59:24.461 INFO:tasks.workunit.client.0.smithi027.stdout:1/952: dwrite d3/d7/fc [4194304,4194304] 0 2024-04-20T10:59:24.461 INFO:tasks.workunit.client.0.smithi027.stdout:3/773: dread d2/dd/f1a [0,4194304] 0 2024-04-20T10:59:24.461 INFO:tasks.workunit.client.0.smithi027.stdout:3/774: chown d2/d18/c93 0 1 2024-04-20T10:59:24.462 INFO:tasks.workunit.client.0.smithi027.stdout:2/722: dwrite d5/dc/d16/d37/de2/dac/f50 [4194304,4194304] 0 2024-04-20T10:59:24.465 INFO:tasks.workunit.client.0.smithi027.stdout:4/768: dread d0/d14/d6b/da3/d1b/d2f/ff6 [0,4194304] 0 2024-04-20T10:59:24.470 INFO:tasks.workunit.client.0.smithi027.stdout:1/953: write d3/d7/d38/f56 [1613889,103122] 0 2024-04-20T10:59:24.470 INFO:tasks.workunit.client.0.smithi027.stdout:1/954: chown d3/d45/d5d/cc7 16362 1 2024-04-20T10:59:24.705 INFO:tasks.workunit.client.0.smithi027.stdout:5/727: truncate d19/d1b/d1c/d1e/d3c/fcd 1508196 0 2024-04-20T10:59:24.706 INFO:tasks.workunit.client.0.smithi027.stdout:8/836: mknod d9/de/d1e/c119 0 2024-04-20T10:59:24.706 INFO:tasks.workunit.client.0.smithi027.stdout:9/687: unlink d4/d1b/d1f/d45/f83 0 2024-04-20T10:59:24.706 INFO:tasks.workunit.client.0.smithi027.stdout:8/837: truncate d9/de/d1e/d4a/d6d/db7/d5b/fa8 484233 0 2024-04-20T10:59:24.706 INFO:tasks.workunit.client.0.smithi027.stdout:8/838: write d9/de/d1e/d4a/f8f [5115074,60317] 0 2024-04-20T10:59:24.706 INFO:tasks.workunit.client.0.smithi027.stdout:3/775: symlink d2/dd/d57/de7/lf4 0 2024-04-20T10:59:24.707 INFO:tasks.workunit.client.0.smithi027.stdout:2/723: rename d5/dc/d16/d2a/d59/fd6 to d5/d1e/d9d/fe9 0 2024-04-20T10:59:24.708 INFO:tasks.workunit.client.0.smithi027.stdout:4/769: creat d0/d14/d71/dfd/f115 x:0 0 0 2024-04-20T10:59:24.711 INFO:tasks.workunit.client.0.smithi027.stdout:1/955: creat d3/d7/d38/ddf/d73/f154 x:0 0 0 2024-04-20T10:59:24.711 INFO:tasks.workunit.client.0.smithi027.stdout:1/956: write d3/d20/d85/f86 [864231,65465] 0 2024-04-20T10:59:24.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/688: rmdir d4/d14 39 2024-04-20T10:59:24.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/689: stat d4/d14/f27 0 2024-04-20T10:59:24.714 INFO:tasks.workunit.client.0.smithi027.stdout:3/776: unlink d2/d18/fac 0 2024-04-20T10:59:24.715 INFO:tasks.workunit.client.0.smithi027.stdout:8/839: rename d9/de/d1e/d4a/d6d/d75/d7c/fbc to d9/de/d64/d6c/f11a 0 2024-04-20T10:59:24.717 INFO:tasks.workunit.client.0.smithi027.stdout:2/724: creat d5/d26/d47/fea x:0 0 0 2024-04-20T10:59:24.719 INFO:tasks.workunit.client.0.smithi027.stdout:4/770: mkdir d0/d14/d6b/da3/d53/d116 0 2024-04-20T10:59:24.720 INFO:tasks.workunit.client.0.smithi027.stdout:1/957: symlink d3/d45/d5d/d6c/l155 0 2024-04-20T10:59:24.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/958: fdatasync d3/d7/f34 0 2024-04-20T10:59:24.721 INFO:tasks.workunit.client.0.smithi027.stdout:5/728: dwrite d19/d1b/d1c/dbd/dbe/fb6 [0,4194304] 0 2024-04-20T10:59:24.722 INFO:tasks.workunit.client.0.smithi027.stdout:3/777: rename d2/dd/c1c to d2/dd/d57/d8c/cf5 0 2024-04-20T10:59:24.722 INFO:tasks.workunit.client.0.smithi027.stdout:3/778: write d2/dd/df/d23/d68/d88/f89 [60698,32337] 0 2024-04-20T10:59:24.723 INFO:tasks.workunit.client.0.smithi027.stdout:9/690: dread d4/d1b/d8f/f90 [0,4194304] 0 2024-04-20T10:59:24.723 INFO:tasks.workunit.client.0.smithi027.stdout:2/725: link d5/d26/d30/c31 d5/dc/d16/d37/de2/d8e/ceb 0 2024-04-20T10:59:24.725 INFO:tasks.workunit.client.0.smithi027.stdout:4/771: mknod d0/d5e/d6c/de8/c117 0 2024-04-20T10:59:24.725 INFO:tasks.workunit.client.0.smithi027.stdout:4/772: stat d0/d14/d6b/da3/d1a/f8a 0 2024-04-20T10:59:24.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/915: sync 2024-04-20T10:59:24.727 INFO:tasks.workunit.client.0.smithi027.stdout:6/829: sync 2024-04-20T10:59:24.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/916: fdatasync d0/d11/d2c/d2e/d55/f6b 0 2024-04-20T10:59:24.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/917: chown d0/d11/d4b/ld6 819630 1 2024-04-20T10:59:24.727 INFO:tasks.workunit.client.0.smithi027.stdout:7/777: sync 2024-04-20T10:59:24.727 INFO:tasks.workunit.client.0.smithi027.stdout:5/729: creat d19/d1b/d1c/d1e/d22/db1/dcf/fea x:0 0 0 2024-04-20T10:59:24.728 INFO:tasks.workunit.client.0.smithi027.stdout:7/778: write d4/d1b/d4a/d62/fa5 [206510,51252] 0 2024-04-20T10:59:24.728 INFO:tasks.workunit.client.0.smithi027.stdout:3/779: creat d2/dd/d57/d8d/dc6/ff6 x:0 0 0 2024-04-20T10:59:24.730 INFO:tasks.workunit.client.0.smithi027.stdout:9/691: rename d4/d14/l16 to d4/d1b/d1f/d45/lcd 0 2024-04-20T10:59:24.730 INFO:tasks.workunit.client.0.smithi027.stdout:9/692: chown d4/d1b/d31/d51/f94 1901576 1 2024-04-20T10:59:24.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/726: symlink d5/dc/lec 0 2024-04-20T10:59:24.731 INFO:tasks.workunit.client.0.smithi027.stdout:0/918: creat d0/d11/d2c/d2e/d55/f135 x:0 0 0 2024-04-20T10:59:24.733 INFO:tasks.workunit.client.0.smithi027.stdout:9/693: dread d4/d1b/d1f/d45/d8a/f6e [0,4194304] 0 2024-04-20T10:59:24.733 INFO:tasks.workunit.client.0.smithi027.stdout:9/694: dread - d4/f9f zero size 2024-04-20T10:59:24.733 INFO:tasks.workunit.client.0.smithi027.stdout:9/695: chown d4/d14/l7e 15 1 2024-04-20T10:59:24.734 INFO:tasks.workunit.client.0.smithi027.stdout:9/696: fdatasync d4/d1b/d31/f75 0 2024-04-20T10:59:24.736 INFO:tasks.workunit.client.0.smithi027.stdout:5/730: mkdir d19/d1b/d1c/d1e/d25/d40/deb 0 2024-04-20T10:59:24.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/779: unlink d4/d1b/d58/f75 0 2024-04-20T10:59:24.736 INFO:tasks.workunit.client.0.smithi027.stdout:4/773: rename d0/d14/d71/dfd/f115 to d0/d14/d6b/da3/d53/d6a/da1/f118 0 2024-04-20T10:59:24.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/919: mknod d0/d11/d2c/d2e/d2b/d7b/c136 0 2024-04-20T10:59:24.736 INFO:tasks.workunit.client.0.smithi027.stdout:5/731: chown d19/l97 971 1 2024-04-20T10:59:24.736 INFO:tasks.workunit.client.0.smithi027.stdout:9/697: mknod d4/d74/cce 0 2024-04-20T10:59:24.737 INFO:tasks.workunit.client.0.smithi027.stdout:9/698: getdents d4/d74/dae 0 2024-04-20T10:59:24.738 INFO:tasks.workunit.client.0.smithi027.stdout:7/780: unlink d4/f3d 0 2024-04-20T10:59:24.740 INFO:tasks.workunit.client.0.smithi027.stdout:6/830: rename da/dd/f59 to da/d21/d28/d2d/d53/dae/ddf/f10b 0 2024-04-20T10:59:24.740 INFO:tasks.workunit.client.0.smithi027.stdout:0/920: getdents d0/d11/d4b/d5b/d6f 0 2024-04-20T10:59:24.742 INFO:tasks.workunit.client.0.smithi027.stdout:1/959: dwrite d3/d7/d38/ddf/dcc/fd0 [4194304,4194304] 0 2024-04-20T10:59:24.742 INFO:tasks.workunit.client.0.smithi027.stdout:1/960: chown d3/d41/dc6/fd3 422 1 2024-04-20T10:59:24.742 INFO:tasks.workunit.client.0.smithi027.stdout:1/961: fsync d3/d20/f46 0 2024-04-20T10:59:24.742 INFO:tasks.workunit.client.0.smithi027.stdout:5/732: mkdir d19/d1b/d1c/d1e/d22/dec 0 2024-04-20T10:59:24.743 INFO:tasks.workunit.client.0.smithi027.stdout:8/840: dwrite d9/de/d1e/d4a/d6d/db7/d97/ff2 [0,4194304] 0 2024-04-20T10:59:24.743 INFO:tasks.workunit.client.0.smithi027.stdout:3/780: dread d2/f9 [0,4194304] 0 2024-04-20T10:59:24.748 INFO:tasks.workunit.client.0.smithi027.stdout:2/727: dwrite d5/d44/f90 [0,4194304] 0 2024-04-20T10:59:24.768 INFO:tasks.workunit.client.0.smithi027.stdout:2/728: dwrite d5/dc/d16/d37/f3f [0,4194304] 0 2024-04-20T10:59:24.783 INFO:tasks.workunit.client.0.smithi027.stdout:2/729: dwrite d5/d26/d47/fea [0,4194304] 0 2024-04-20T10:59:24.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:24.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:24.999+0000 7fc660814640 1 Processor -- start 2024-04-20T10:59:24.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:24.999+0000 7fc660814640 1 -- start start 2024-04-20T10:59:25.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:24.999+0000 7fc660814640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c0b54f0 0x7fc65c0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:24.999+0000 7fc660814640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc65c0d0c90 0x7fc65c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:24.999+0000 7fc660814640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc65c0b5e70 con 0x7fc65c155950 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:24.999+0000 7fc660814640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc65c15b9c0 con 0x7fc65c0d0c90 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:24.999+0000 7fc660814640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc65c15bca0 con 0x7fc65c0b54f0 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:24.999+0000 7fc65ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc65c0d0c90 0x7fc65c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.000+0000 7fc65ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc65c0d0c90 0x7fc65c0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39444/0 (socket says 172.21.15.27:39444) 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.000+0000 7fc65ad9d640 1 -- 172.21.15.27:0/4198266042 learned_addr learned my addr 172.21.15.27:0/4198266042 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.000+0000 7fc659d9b640 1 --2- 172.21.15.27:0/4198266042 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c0b54f0 0x7fc65c0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.000+0000 7fc65ad9d640 1 -- 172.21.15.27:0/4198266042 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c0b54f0 msgr2=0x7fc65c0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.000+0000 7fc65ad9d640 1 --2- 172.21.15.27:0/4198266042 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c0b54f0 0x7fc65c0b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.000+0000 7fc65ad9d640 1 -- 172.21.15.27:0/4198266042 >> v1:172.21.15.137:6789/0 conn(0x7fc65c155950 legacy=0x7fc65c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.000+0000 7fc65ad9d640 1 -- 172.21.15.27:0/4198266042 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c15bf80 con 0x7fc65c0d0c90 2024-04-20T10:59:25.002 INFO:tasks.workunit.client.0.smithi027.stdout:7/781: unlink d4/d5/d8/c25 0 2024-04-20T10:59:25.002 INFO:tasks.workunit.client.0.smithi027.stdout:4/774: rename d0/d14/d6b/da3/d18/f38 to d0/d14/d6b/da3/d18/dbf/f119 0 2024-04-20T10:59:25.002 INFO:tasks.workunit.client.0.smithi027.stdout:6/831: creat da/d21/dab/d7e/d86/f10c x:0 0 0 2024-04-20T10:59:25.002 INFO:tasks.workunit.client.0.smithi027.stdout:0/921: truncate d0/d11/d4b/fcb 1350847 0 2024-04-20T10:59:25.003 INFO:tasks.workunit.client.0.smithi027.stdout:9/699: unlink d4/d1b/d1f/l26 0 2024-04-20T10:59:25.003 INFO:tasks.workunit.client.0.smithi027.stdout:4/775: write d0/d14/d6b/da3/d1a/f8a [538945,130651] 0 2024-04-20T10:59:25.003 INFO:tasks.workunit.client.0.smithi027.stdout:5/733: unlink d19/d1b/d1c/d1e/d83/dcc/fd8 0 2024-04-20T10:59:25.003 INFO:tasks.workunit.client.0.smithi027.stdout:3/781: link d2/fcb d2/dd/d57/d8c/ff7 0 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.001+0000 7fc65ad9d640 1 --2- 172.21.15.27:0/4198266042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0d0c90 0x7fc65c0d1100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc654009e50 tx=0x7fc65402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14af979f5558c97a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.001+0000 7fc65959a640 1 -- 172.21.15.27:0/4198266042 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65402cb90 con 0x7fc65c0d0c90 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.001+0000 7fc65959a640 1 -- 172.21.15.27:0/4198266042 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc65402cd30 con 0x7fc65c0d0c90 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.001+0000 7fc65959a640 1 -- 172.21.15.27:0/4198266042 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc654034610 con 0x7fc65c0d0c90 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.002+0000 7fc660814640 1 -- 172.21.15.27:0/4198266042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0d0c90 msgr2=0x7fc65c0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.002+0000 7fc660814640 1 --2- 172.21.15.27:0/4198266042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0d0c90 0x7fc65c0d1100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc654009e50 tx=0x7fc65402bff0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.002+0000 7fc660814640 1 -- 172.21.15.27:0/4198266042 shutdown_connections 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.002+0000 7fc660814640 1 --2- 172.21.15.27:0/4198266042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0d0c90 0x7fc65c0d1100 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.002+0000 7fc660814640 1 --2- 172.21.15.27:0/4198266042 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c0b54f0 0x7fc65c0b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.002+0000 7fc660814640 1 -- 172.21.15.27:0/4198266042 >> 172.21.15.27:0/4198266042 conn(0x7fc65c0676c0 msgr2=0x7fc65c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:25.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.002+0000 7fc660814640 1 -- 172.21.15.27:0/4198266042 shutdown_connections 2024-04-20T10:59:25.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.002+0000 7fc660814640 1 -- 172.21.15.27:0/4198266042 wait complete. 2024-04-20T10:59:25.005 INFO:tasks.workunit.client.0.smithi027.stdout:2/730: mknod d5/d44/d7c/dca/ced 0 2024-04-20T10:59:25.005 INFO:tasks.workunit.client.0.smithi027.stdout:7/782: rmdir d4/d1b 39 2024-04-20T10:59:25.005 INFO:tasks.workunit.client.0.smithi027.stdout:1/962: rename d3/d20/d85 to d3/d20/d107/d132/d156 0 2024-04-20T10:59:25.005 INFO:tasks.workunit.client.0.smithi027.stdout:1/963: read - d3/d7/d28/d35/f11e zero size 2024-04-20T10:59:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.004+0000 7fc660814640 1 Processor -- start 2024-04-20T10:59:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.004+0000 7fc660814640 1 -- start start 2024-04-20T10:59:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc660814640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0b54f0 0x7fc65c153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc660814640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c0d0c90 0x7fc65c14c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc660814640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c155950 0x7fc65c14cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc660814640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c0d1640 con 0x7fc65c0b54f0 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc660814640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c15c540 con 0x7fc65c0d0c90 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc660814640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc65c15fb90 con 0x7fc65c155950 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc65a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0b54f0 0x7fc65c153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc65a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0b54f0 0x7fc65c153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39446/0 (socket says 172.21.15.27:39446) 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc65a59c640 1 -- 172.21.15.27:0/651665962 learned_addr learned my addr 172.21.15.27:0/651665962 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc659d9b640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c0d0c90 0x7fc65c14c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.005+0000 7fc65ad9d640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c155950 0x7fc65c14cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc65a59c640 1 -- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c155950 msgr2=0x7fc65c14cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc65a59c640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c155950 0x7fc65c14cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc65a59c640 1 -- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c0d0c90 msgr2=0x7fc65c14c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc65a59c640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c0d0c90 0x7fc65c14c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc65a59c640 1 -- 172.21.15.27:0/651665962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c159e20 con 0x7fc65c0b54f0 2024-04-20T10:59:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc65ad9d640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c155950 0x7fc65c14cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:59:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc65a59c640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0b54f0 0x7fc65c153d60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00c400 tx=0x7fc64c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc64affd640 1 -- 172.21.15.27:0/651665962 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0052d0 con 0x7fc65c0b54f0 2024-04-20T10:59:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc64affd640 1 -- 172.21.15.27:0/651665962 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64c005490 con 0x7fc65c0b54f0 2024-04-20T10:59:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c1f15e0 con 0x7fc65c0b54f0 2024-04-20T10:59:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c1f1ba0 con 0x7fc65c0b54f0 2024-04-20T10:59:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.006+0000 7fc64affd640 1 -- 172.21.15.27:0/651665962 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c007260 con 0x7fc65c0b54f0 2024-04-20T10:59:25.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.007+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc640005290 con 0x7fc65c0b54f0 2024-04-20T10:59:25.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.008+0000 7fc64affd640 1 -- 172.21.15.27:0/651665962 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc64c007400 con 0x7fc65c0b54f0 2024-04-20T10:59:25.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.008+0000 7fc64affd640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc63c06c8f0 0x7fc63c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:25.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.009+0000 7fc64affd640 1 -- 172.21.15.27:0/651665962 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7fc64c08b1e0 con 0x7fc65c0b54f0 2024-04-20T10:59:25.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.009+0000 7fc659d9b640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc63c06c8f0 0x7fc63c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:25.011 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.011+0000 7fc659d9b640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc63c06c8f0 0x7fc63c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc65c1545a0 tx=0x7fc650009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:25.011 INFO:tasks.workunit.client.0.smithi027.stdout:3/782: dread d2/d18/d2c/f54 [0,4194304] 0 2024-04-20T10:59:25.013 INFO:tasks.workunit.client.0.smithi027.stdout:6/832: mkdir da/dcc/d108/d6b/d10d 0 2024-04-20T10:59:25.013 INFO:tasks.workunit.client.0.smithi027.stdout:4/776: mkdir d0/d48/d68/d11a 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:2/731: symlink d5/dc/d16/d32/dab/dba/lee 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:8/841: rename d9/de/d1e/d4a/d6d/dea/d59/da0 to d9/de/d1e/d11b 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:9/700: getdents d4/d74 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:1/964: creat d3/d41/dc6/f157 x:0 0 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:8/842: write d9/de/da3/d3c/fd7 [234795,63120] 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:1/965: fdatasync d3/d7/d38/da4/dac/ff9 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:7/783: mknod d4/cfd 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:3/783: write d2/dd/d57/d8d/fba [901043,52507] 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:3/784: readlink d2/d18/lb4 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:3/785: truncate d2/d18/d2c/d4c/f6e 954526 0 2024-04-20T10:59:25.014 INFO:tasks.workunit.client.0.smithi027.stdout:3/786: readlink d2/dd/d57/l92 0 2024-04-20T10:59:25.015 INFO:tasks.workunit.client.0.smithi027.stdout:4/777: rename d0/d14/d6b/da3/d53/d6a/l8f to d0/d14/d6b/da3/d18/l11b 0 2024-04-20T10:59:25.015 INFO:tasks.workunit.client.0.smithi027.stdout:4/778: chown d0/d14/d6b/da3/d53/d6a/cdf 26234813 1 2024-04-20T10:59:25.015 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.014+0000 7fc64affd640 1 -- 172.21.15.27:0/651665962 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc64c058e10 con 0x7fc65c0b54f0 2024-04-20T10:59:25.016 INFO:tasks.workunit.client.0.smithi027.stdout:2/732: link d5/dc/f1c d5/dc/d16/db1/fef 0 2024-04-20T10:59:25.016 INFO:tasks.workunit.client.0.smithi027.stdout:9/701: rmdir d4/d1b/d1f/d5a 39 2024-04-20T10:59:25.018 INFO:tasks.workunit.client.0.smithi027.stdout:1/966: rmdir d3/d20/dfb 39 2024-04-20T10:59:25.018 INFO:tasks.workunit.client.0.smithi027.stdout:1/967: getdents d3/d7/d38/db8/d137 0 2024-04-20T10:59:25.018 INFO:tasks.workunit.client.0.smithi027.stdout:1/968: fdatasync d3/d20/ffc 0 2024-04-20T10:59:25.018 INFO:tasks.workunit.client.0.smithi027.stdout:7/784: mkdir d4/d4d/dfe 0 2024-04-20T10:59:25.018 INFO:tasks.workunit.client.0.smithi027.stdout:7/785: write d4/d5/fd3 [480697,77914] 0 2024-04-20T10:59:25.019 INFO:tasks.workunit.client.0.smithi027.stdout:8/843: write d9/ff [343319,24578] 0 2024-04-20T10:59:25.019 INFO:tasks.workunit.client.0.smithi027.stdout:3/787: symlink d2/dd/d57/d8d/dc6/de1/lf8 0 2024-04-20T10:59:25.023 INFO:tasks.workunit.client.0.smithi027.stdout:9/702: link d4/d1b/d8f/f90 d4/d1b/d1f/d5a/db7/fcf 0 2024-04-20T10:59:25.023 INFO:tasks.workunit.client.0.smithi027.stdout:4/779: creat d0/d48/f11c x:0 0 0 2024-04-20T10:59:25.023 INFO:tasks.workunit.client.0.smithi027.stdout:1/969: getdents d3/d7/d38/da4/dbc 0 2024-04-20T10:59:25.025 INFO:tasks.workunit.client.0.smithi027.stdout:7/786: creat d4/d5/d8/d21/d94/fff x:0 0 0 2024-04-20T10:59:25.026 INFO:tasks.workunit.client.0.smithi027.stdout:5/734: dread d19/d1b/d1c/d1e/f57 [4194304,4194304] 0 2024-04-20T10:59:25.026 INFO:tasks.workunit.client.0.smithi027.stdout:8/844: mkdir d9/d11c 0 2024-04-20T10:59:25.027 INFO:tasks.workunit.client.0.smithi027.stdout:3/788: rename d2/dd/df/d23/d68/fbb to d2/dd/df/d23/df0/ff9 0 2024-04-20T10:59:25.027 INFO:tasks.workunit.client.0.smithi027.stdout:3/789: readlink d2/d18/d2c/d4c/l4d 0 2024-04-20T10:59:25.028 INFO:tasks.workunit.client.0.smithi027.stdout:4/780: mknod d0/d14/d6b/c11d 0 2024-04-20T10:59:25.028 INFO:tasks.workunit.client.0.smithi027.stdout:1/970: creat d3/d7/d38/da4/dac/de3/f158 x:0 0 0 2024-04-20T10:59:25.030 INFO:tasks.workunit.client.0.smithi027.stdout:0/922: dwrite d0/d11/d4b/d5b/fbd [0,4194304] 0 2024-04-20T10:59:25.032 INFO:tasks.workunit.client.0.smithi027.stdout:7/787: rename d4/d5b/ca4 to d4/d5/d8/d21/d94/c100 0 2024-04-20T10:59:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/790: creat d2/dd/d57/ffa x:0 0 0 2024-04-20T10:59:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/791: fdatasync d2/d80/f5a 0 2024-04-20T10:59:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/792: truncate d2/dd/df/f85 974492 0 2024-04-20T10:59:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:3/793: dread - d2/fd9 zero size 2024-04-20T10:59:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:4/781: dread d0/d14/d6b/da3/d53/d6a/faa [0,4194304] 0 2024-04-20T10:59:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:4/782: dread - d0/d14/d6b/da3/fe0 zero size 2024-04-20T10:59:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:4/783: truncate d0/d14/d6b/da3/d53/d6a/ff3 254229 0 2024-04-20T10:59:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:5/735: read d19/d1b/d1c/d1e/d25/f34 [11011096,85944] 0 2024-04-20T10:59:25.034 INFO:tasks.workunit.client.0.smithi027.stdout:5/736: stat d19/d1b/d1c/d1e/d22/c95 0 2024-04-20T10:59:25.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/737: write f16 [1263169,111252] 0 2024-04-20T10:59:25.035 INFO:tasks.workunit.client.0.smithi027.stdout:1/971: creat d3/d7/d38/da4/dac/d13a/f159 x:0 0 0 2024-04-20T10:59:25.035 INFO:tasks.workunit.client.0.smithi027.stdout:1/972: truncate d3/d7/d38/d62/f122 816787 0 2024-04-20T10:59:25.039 INFO:tasks.workunit.client.0.smithi027.stdout:0/923: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/f137 x:0 0 0 2024-04-20T10:59:25.040 INFO:tasks.workunit.client.0.smithi027.stdout:0/924: fdatasync d0/d11/d2c/d2e/d35/d8c/f12e 0 2024-04-20T10:59:25.040 INFO:tasks.workunit.client.0.smithi027.stdout:0/925: fsync d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/f104 0 2024-04-20T10:59:25.040 INFO:tasks.workunit.client.0.smithi027.stdout:7/788: symlink d4/d5b/l101 0 2024-04-20T10:59:25.040 INFO:tasks.workunit.client.0.smithi027.stdout:7/789: dread - d4/d5/d6f/fed zero size 2024-04-20T10:59:25.040 INFO:tasks.workunit.client.0.smithi027.stdout:7/790: readlink d4/d5/d8/l36 0 2024-04-20T10:59:25.041 INFO:tasks.workunit.client.0.smithi027.stdout:7/791: truncate d4/d5/d8/d21/d94/fbd 212898 0 2024-04-20T10:59:25.041 INFO:tasks.workunit.client.0.smithi027.stdout:7/792: write d4/d5/d8/d21/d94/fbd [747245,129005] 0 2024-04-20T10:59:25.041 INFO:tasks.workunit.client.0.smithi027.stdout:4/784: unlink d0/ce 0 2024-04-20T10:59:25.041 INFO:tasks.workunit.client.0.smithi027.stdout:5/738: mkdir d19/d1b/d1c/d1e/d58/db4/ded 0 2024-04-20T10:59:25.042 INFO:tasks.workunit.client.0.smithi027.stdout:0/926: creat d0/d59/d9e/f138 x:0 0 0 2024-04-20T10:59:25.043 INFO:tasks.workunit.client.0.smithi027.stdout:3/794: dread d2/d18/d2c/d4c/d67/fab [0,4194304] 0 2024-04-20T10:59:25.043 INFO:tasks.workunit.client.0.smithi027.stdout:4/785: getdents d0/d14/d71 0 2024-04-20T10:59:25.043 INFO:tasks.workunit.client.0.smithi027.stdout:5/739: truncate d19/d1b/d1c/dbd/dbe/fe6 3166662 0 2024-04-20T10:59:25.043 INFO:tasks.workunit.client.0.smithi027.stdout:0/927: mkdir d0/d11/d5c/df0/d133/d139 0 2024-04-20T10:59:25.046 INFO:tasks.workunit.client.0.smithi027.stdout:6/833: dwrite da/d21/fd9 [0,4194304] 0 2024-04-20T10:59:25.046 INFO:tasks.workunit.client.0.smithi027.stdout:0/928: mkdir d0/d11/d2c/d2e/d2b/d7b/d64/d13a 0 2024-04-20T10:59:25.046 INFO:tasks.workunit.client.0.smithi027.stdout:3/795: read d2/dd/d22/fbf [5253,38787] 0 2024-04-20T10:59:25.047 INFO:tasks.workunit.client.0.smithi027.stdout:5/740: rename d19/c4c to d19/d1b/d1c/d1e/d25/d40/d82/cee 0 2024-04-20T10:59:25.051 INFO:tasks.workunit.client.0.smithi027.stdout:7/793: dread d4/da2/fb5 [0,4194304] 0 2024-04-20T10:59:25.052 INFO:tasks.workunit.client.0.smithi027.stdout:6/834: symlink da/d21/dab/d1f/dbf/l10e 0 2024-04-20T10:59:25.053 INFO:tasks.workunit.client.0.smithi027.stdout:2/733: dwrite d5/d26/fb3 [0,4194304] 0 2024-04-20T10:59:25.053 INFO:tasks.workunit.client.0.smithi027.stdout:2/734: chown d5/dbc/fd2 2 1 2024-04-20T10:59:25.053 INFO:tasks.workunit.client.0.smithi027.stdout:0/929: symlink d0/d11/d2c/d2e/d2b/d7b/d12/d25/l13b 0 2024-04-20T10:59:25.053 INFO:tasks.workunit.client.0.smithi027.stdout:0/930: chown d0/d11/d2c/d2e/d2b/d7b/d12/l83 2017456 1 2024-04-20T10:59:25.054 INFO:tasks.workunit.client.0.smithi027.stdout:3/796: mkdir d2/dd/df/d33/dfb 0 2024-04-20T10:59:25.054 INFO:tasks.workunit.client.0.smithi027.stdout:3/797: stat d2/dd/d57/d8d/dc6/de1/fe8 0 2024-04-20T10:59:25.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/835: dread da/dd/d79/fb7 [0,4194304] 0 2024-04-20T10:59:25.055 INFO:tasks.workunit.client.0.smithi027.stdout:1/973: dwrite d3/d45/f116 [0,4194304] 0 2024-04-20T10:59:25.056 INFO:tasks.workunit.client.0.smithi027.stdout:5/741: mknod d19/d1b/d1c/d1e/d22/d68/d75/cef 0 2024-04-20T10:59:25.060 INFO:tasks.workunit.client.0.smithi027.stdout:2/735: symlink d5/dc/lf0 0 2024-04-20T10:59:25.062 INFO:tasks.workunit.client.0.smithi027.stdout:0/931: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/f13c x:0 0 0 2024-04-20T10:59:25.063 INFO:tasks.workunit.client.0.smithi027.stdout:0/932: write d0/d11/d2c/d2e/d2b/d7b/d12/d1a/fa5 [576686,71083] 0 2024-04-20T10:59:25.065 INFO:tasks.workunit.client.0.smithi027.stdout:6/836: link da/dcc/d108/d6b/ca0 da/d21/dab/d7e/d86/dcd/de1/c10f 0 2024-04-20T10:59:25.066 INFO:tasks.workunit.client.0.smithi027.stdout:1/974: getdents d3/d7/d38/da4/dac 0 2024-04-20T10:59:25.068 INFO:tasks.workunit.client.0.smithi027.stdout:5/742: creat d19/d1b/d1c/d1e/d22/d68/d75/de9/ff0 x:0 0 0 2024-04-20T10:59:25.069 INFO:tasks.workunit.client.0.smithi027.stdout:0/933: creat d0/d11/d5c/df0/d133/d139/f13d x:0 0 0 2024-04-20T10:59:25.070 INFO:tasks.workunit.client.0.smithi027.stdout:3/798: link d2/dd/f3d d2/dd/d57/d8d/d9d/ffc 0 2024-04-20T10:59:25.072 INFO:tasks.workunit.client.0.smithi027.stdout:6/837: creat da/f110 x:0 0 0 2024-04-20T10:59:25.074 INFO:tasks.workunit.client.0.smithi027.stdout:5/743: dread fc [0,4194304] 0 2024-04-20T10:59:25.074 INFO:tasks.workunit.client.0.smithi027.stdout:5/744: chown d19/d1b/d1c/d1e/d58/d9d/cc2 1901633076 1 2024-04-20T10:59:25.074 INFO:tasks.workunit.client.0.smithi027.stdout:5/745: read - d19/fc9 zero size 2024-04-20T10:59:25.074 INFO:tasks.workunit.client.0.smithi027.stdout:3/799: dread d2/dd/d22/fa2 [0,4194304] 0 2024-04-20T10:59:25.075 INFO:tasks.workunit.client.0.smithi027.stdout:7/794: dwrite d4/d5b/f67 [0,4194304] 0 2024-04-20T10:59:25.075 INFO:tasks.workunit.client.0.smithi027.stdout:7/795: fdatasync d4/d5/f33 0 2024-04-20T10:59:25.076 INFO:tasks.workunit.client.0.smithi027.stdout:1/975: creat d3/d7/d38/db8/d137/f15a x:0 0 0 2024-04-20T10:59:25.076 INFO:tasks.workunit.client.0.smithi027.stdout:1/976: dread - d3/d20/f8e zero size 2024-04-20T10:59:25.076 INFO:tasks.workunit.client.0.smithi027.stdout:1/977: readlink d3/d7/d38/l10b 0 2024-04-20T10:59:25.076 INFO:tasks.workunit.client.0.smithi027.stdout:1/978: truncate d3/d7e/d123/f13b 192345 0 2024-04-20T10:59:25.076 INFO:tasks.workunit.client.0.smithi027.stdout:1/979: chown d3/d7/c2a 3300081 1 2024-04-20T10:59:25.076 INFO:tasks.workunit.client.0.smithi027.stdout:0/934: dread d0/d11/f21 [0,4194304] 0 2024-04-20T10:59:25.077 INFO:tasks.workunit.client.0.smithi027.stdout:1/980: write d3/d20/faf [395092,20530] 0 2024-04-20T10:59:25.077 INFO:tasks.workunit.client.0.smithi027.stdout:5/746: rename d19/d1b/d1c/d1e/d22/db1/f9f to d19/d1b/d1c/ff1 0 2024-04-20T10:59:25.077 INFO:tasks.workunit.client.0.smithi027.stdout:7/796: rmdir d4/d1b 39 2024-04-20T10:59:25.078 INFO:tasks.workunit.client.0.smithi027.stdout:0/935: mknod d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/c13e 0 2024-04-20T10:59:25.078 INFO:tasks.workunit.client.0.smithi027.stdout:0/936: dread - d0/d11/d2c/d2e/d2b/d7b/d12/f111 zero size 2024-04-20T10:59:25.079 INFO:tasks.workunit.client.0.smithi027.stdout:0/937: stat d0/d11/d4b/d5b/d6f/d82/dd3/f131 0 2024-04-20T10:59:25.079 INFO:tasks.workunit.client.0.smithi027.stdout:2/736: dwrite d5/dc/f41 [0,4194304] 0 2024-04-20T10:59:25.079 INFO:tasks.workunit.client.0.smithi027.stdout:1/981: rmdir d3/d45/d5d 39 2024-04-20T10:59:25.079 INFO:tasks.workunit.client.0.smithi027.stdout:1/982: fsync d3/d7/d38/d62/f81 0 2024-04-20T10:59:25.080 INFO:tasks.workunit.client.0.smithi027.stdout:3/800: rename d2/d18/lb4 to d2/dd/d22/de2/lfd 0 2024-04-20T10:59:25.081 INFO:tasks.workunit.client.0.smithi027.stdout:5/747: rmdir d19/d1b/d1c/d1e/d22 39 2024-04-20T10:59:25.084 INFO:tasks.workunit.client.0.smithi027.stdout:0/938: getdents d0/d11/d2c/d2e/d2b/d7b/d34/d5a/d71 0 2024-04-20T10:59:25.085 INFO:tasks.workunit.client.0.smithi027.stdout:1/983: symlink d3/d7/d38/da4/dbc/l15b 0 2024-04-20T10:59:25.087 INFO:tasks.workunit.client.0.smithi027.stdout:1/984: dread d3/d7/d28/f8b [0,4194304] 0 2024-04-20T10:59:25.088 INFO:tasks.workunit.client.0.smithi027.stdout:7/797: rename d4/d5/d8/d46/d77/deb to d4/d5/d6f/d102 0 2024-04-20T10:59:25.088 INFO:tasks.workunit.client.0.smithi027.stdout:7/798: write d4/d5/d6f/fed [148385,89319] 0 2024-04-20T10:59:25.088 INFO:tasks.workunit.client.0.smithi027.stdout:7/799: chown d4/d5/d6f/fca 3 1 2024-04-20T10:59:25.089 INFO:tasks.workunit.client.0.smithi027.stdout:0/939: creat d0/d11/d2c/d2e/d35/d8c/f13f x:0 0 0 2024-04-20T10:59:25.090 INFO:tasks.workunit.client.0.smithi027.stdout:2/737: dread d5/dc/f1c [0,4194304] 0 2024-04-20T10:59:25.090 INFO:tasks.workunit.client.0.smithi027.stdout:2/738: truncate d5/d26/d30/f79 981079 0 2024-04-20T10:59:25.090 INFO:tasks.workunit.client.0.smithi027.stdout:2/739: dread - d5/dc/d16/db1/fe5 zero size 2024-04-20T10:59:25.091 INFO:tasks.workunit.client.0.smithi027.stdout:1/985: creat d3/d7/d38/ddf/dcc/f15c x:0 0 0 2024-04-20T10:59:25.091 INFO:tasks.workunit.client.0.smithi027.stdout:7/800: dread d4/d5/d8/d45/f71 [0,4194304] 0 2024-04-20T10:59:25.092 INFO:tasks.workunit.client.0.smithi027.stdout:0/940: rename d0/d11/d2c/d2e/d2b/d7b/d34/l76 to d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/l140 0 2024-04-20T10:59:25.092 INFO:tasks.workunit.client.0.smithi027.stdout:0/941: stat d0/d11/d2c/d2e/d2b/d7b/f27 0 2024-04-20T10:59:25.093 INFO:tasks.workunit.client.0.smithi027.stdout:0/942: fsync d0/d59/d9e/fb9 0 2024-04-20T10:59:25.093 INFO:tasks.workunit.client.0.smithi027.stdout:0/943: truncate d0/d11/d2c/d2e/d2b/d7b/d12/dd7/fdf 852990 0 2024-04-20T10:59:25.093 INFO:tasks.workunit.client.0.smithi027.stdout:3/801: dread d2/d18/d2c/f78 [0,4194304] 0 2024-04-20T10:59:25.093 INFO:tasks.workunit.client.0.smithi027.stdout:0/944: truncate d0/d11/d2c/d2e/d2b/d7b/d64/f7c 6553385 0 2024-04-20T10:59:25.094 INFO:tasks.workunit.client.0.smithi027.stdout:2/740: symlink d5/lf1 0 2024-04-20T10:59:25.097 INFO:tasks.workunit.client.0.smithi027.stdout:3/802: symlink d2/d18/d2c/d4c/lfe 0 2024-04-20T10:59:25.097 INFO:tasks.workunit.client.0.smithi027.stdout:3/803: chown d2/d80/c55 2142 1 2024-04-20T10:59:25.097 INFO:tasks.workunit.client.0.smithi027.stdout:3/804: stat d2/d18/d2c 0 2024-04-20T10:59:25.097 INFO:tasks.workunit.client.0.smithi027.stdout:3/805: write d2/d18/d2c/d4c/d67/d6c/f87 [518584,50430] 0 2024-04-20T10:59:25.097 INFO:tasks.workunit.client.0.smithi027.stdout:1/986: read d3/d7/d28/f59 [418878,32310] 0 2024-04-20T10:59:25.098 INFO:tasks.workunit.client.0.smithi027.stdout:3/806: rmdir d2/d94/dc4 39 2024-04-20T10:59:25.100 INFO:tasks.workunit.client.0.smithi027.stdout:1/987: creat d3/d7/f15d x:0 0 0 2024-04-20T10:59:25.100 INFO:tasks.workunit.client.0.smithi027.stdout:0/945: dread d0/d11/d2c/d2e/d2b/d3d/f84 [4194304,4194304] 0 2024-04-20T10:59:25.101 INFO:tasks.workunit.client.0.smithi027.stdout:0/946: fsync d0/d11/d2c/d2e/d55/f6b 0 2024-04-20T10:59:25.101 INFO:tasks.workunit.client.0.smithi027.stdout:3/807: getdents d2/d18/d2c/d4c/d67 0 2024-04-20T10:59:25.102 INFO:tasks.workunit.client.0.smithi027.stdout:1/988: rmdir d3/d7/d38/db8 39 2024-04-20T10:59:25.102 INFO:tasks.workunit.client.0.smithi027.stdout:3/808: dread d2/d18/f3f [0,4194304] 0 2024-04-20T10:59:25.102 INFO:tasks.workunit.client.0.smithi027.stdout:1/989: write d3/d7/d38/da4/dbc/f145 [108435,4938] 0 2024-04-20T10:59:25.103 INFO:tasks.workunit.client.0.smithi027.stdout:1/990: dread d3/d41/f8c [0,4194304] 0 2024-04-20T10:59:25.103 INFO:tasks.workunit.client.0.smithi027.stdout:1/991: write d3/d7/d38/da4/dac/fc0 [516850,103582] 0 2024-04-20T10:59:25.103 INFO:tasks.workunit.client.0.smithi027.stdout:1/992: readlink d3/d45/dba/l14b 0 2024-04-20T10:59:25.104 INFO:tasks.workunit.client.0.smithi027.stdout:0/947: creat d0/d11/d2c/d2e/d2b/d7b/d64/d13a/f141 x:0 0 0 2024-04-20T10:59:25.106 INFO:tasks.workunit.client.0.smithi027.stdout:3/809: mkdir d2/dd/d57/de7/dff 0 2024-04-20T10:59:25.106 INFO:tasks.workunit.client.0.smithi027.stdout:3/810: read - d2/dd/d57/ffa zero size 2024-04-20T10:59:25.106 INFO:tasks.workunit.client.0.smithi027.stdout:3/811: stat d2/dd/d57/d8d/dc6/de1/lf8 0 2024-04-20T10:59:25.108 INFO:tasks.workunit.client.0.smithi027.stdout:0/948: unlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f2d 0 2024-04-20T10:59:25.108 INFO:tasks.workunit.client.0.smithi027.stdout:0/949: write d0/d11/d2c/d2e/d2b/d7b/d12/d1a/f50 [555840,66148] 0 2024-04-20T10:59:25.111 INFO:tasks.workunit.client.0.smithi027.stdout:3/812: unlink d2/dd/d57/cbe 0 2024-04-20T10:59:25.122 INFO:tasks.workunit.client.0.smithi027.stdout:0/950: creat d0/d11/d2c/d2e/d2b/d7b/d34/d5a/f142 x:0 0 0 2024-04-20T10:59:25.126 INFO:tasks.workunit.client.0.smithi027.stdout:0/951: link d0/d11/d2c/d2e/d2b/c66 d0/d11/d4b/d5b/d6f/d82/dd1/deb/c143 0 2024-04-20T10:59:25.127 INFO:tasks.workunit.client.0.smithi027.stdout:3/813: dwrite d2/d18/d2c/d4c/d67/f6d [0,4194304] 0 2024-04-20T10:59:25.127 INFO:tasks.workunit.client.0.smithi027.stdout:3/814: stat d2/d80/c49 0 2024-04-20T10:59:25.132 INFO:tasks.workunit.client.0.smithi027.stdout:0/952: unlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/f69 0 2024-04-20T10:59:25.132 INFO:tasks.workunit.client.0.smithi027.stdout:0/953: write d0/d11/d2c/d2e/d2b/d7b/d12/d25/f31 [1126760,16841] 0 2024-04-20T10:59:25.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/815: getdents d2/dd/d57/d8d/d9d 0 2024-04-20T10:59:25.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/816: readlink d2/d7f/l8a 0 2024-04-20T10:59:25.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/817: truncate d2/dd/d57/d8d/fc2 754188 0 2024-04-20T10:59:25.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/818: dread - d2/dd/f39 zero size 2024-04-20T10:59:25.147 INFO:tasks.workunit.client.0.smithi027.stdout:3/819: readlink d2/d18/d2c/d4c/d67/lb6 0 2024-04-20T10:59:25.148 INFO:tasks.workunit.client.0.smithi027.stdout:0/954: rename d0/d11/d2c/d2e/d2b/d7b/d34/lf3 to d0/d11/d2c/d2e/d2b/d7b/d34/d5a/l144 0 2024-04-20T10:59:25.148 INFO:tasks.workunit.client.0.smithi027.stdout:0/955: readlink d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/lfe 0 2024-04-20T10:59:25.153 INFO:tasks.workunit.client.0.smithi027.stdout:3/820: dread d2/d18/d2e/fa7 [0,4194304] 0 2024-04-20T10:59:25.160 INFO:tasks.workunit.client.0.smithi027.stdout:0/956: dread d0/d11/d2c/d2e/d2b/d7b/fce [0,4194304] 0 2024-04-20T10:59:25.173 INFO:tasks.workunit.client.0.smithi027.stdout:9/703: sync 2024-04-20T10:59:25.174 INFO:tasks.workunit.client.0.smithi027.stdout:8/845: sync 2024-04-20T10:59:25.174 INFO:tasks.workunit.client.0.smithi027.stdout:9/704: readlink d4/d74/d72/l92 0 2024-04-20T10:59:25.174 INFO:tasks.workunit.client.0.smithi027.stdout:3/821: mkdir d2/dd/d57/de7/dff/d100 0 2024-04-20T10:59:25.177 INFO:tasks.workunit.client.0.smithi027.stdout:8/846: rename d9/de/d1e/d4a/d6d/d79/f10c to d9/de/d1e/d107/f11d 0 2024-04-20T10:59:25.178 INFO:tasks.workunit.client.0.smithi027.stdout:3/822: rmdir d2/d18/d2c/d4c/d67/d6c/dd0/dd5 39 2024-04-20T10:59:25.178 INFO:tasks.workunit.client.0.smithi027.stdout:3/823: write d2/fd9 [139859,84418] 0 2024-04-20T10:59:25.195 INFO:tasks.workunit.client.0.smithi027.stdout:9/705: dwrite d4/d1b/f22 [0,4194304] 0 2024-04-20T10:59:25.195 INFO:tasks.workunit.client.0.smithi027.stdout:9/706: fsync d4/d14/d43/fc7 0 2024-04-20T10:59:25.195 INFO:tasks.workunit.client.0.smithi027.stdout:9/707: stat d4/d1b/d1f/l65 0 2024-04-20T10:59:25.195 INFO:tasks.workunit.client.0.smithi027.stdout:3/824: dwrite d2/dd/d57/d8d/fc2 [0,4194304] 0 2024-04-20T10:59:25.195 INFO:tasks.workunit.client.0.smithi027.stdout:3/825: chown d2/dd/df/d23/c5d 27074957 1 2024-04-20T10:59:25.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/847: rmdir d9/de/d20/dbf 39 2024-04-20T10:59:25.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/848: dread - d9/de/d1e/d11b/fef zero size 2024-04-20T10:59:25.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/849: chown d9/de/d64/le9 15545732 1 2024-04-20T10:59:25.197 INFO:tasks.workunit.client.0.smithi027.stdout:9/708: mkdir d4/d1b/d31/dd0 0 2024-04-20T10:59:25.198 INFO:tasks.workunit.client.0.smithi027.stdout:3/826: mkdir d2/d18/d2c/d4c/d101 0 2024-04-20T10:59:25.199 INFO:tasks.workunit.client.0.smithi027.stdout:8/850: rename d9/de/f2b to d9/de/d1e/d4a/d6d/d75/f11e 0 2024-04-20T10:59:25.199 INFO:tasks.workunit.client.0.smithi027.stdout:8/851: chown d9/de/d1e/d11b/dbd/fc7 215691429 1 2024-04-20T10:59:25.201 INFO:tasks.workunit.client.0.smithi027.stdout:9/709: unlink d4/d14/f49 0 2024-04-20T10:59:25.231 INFO:tasks.workunit.client.0.smithi027.stdout:9/710: dwrite d4/f1c [8388608,4194304] 0 2024-04-20T10:59:25.231 INFO:tasks.workunit.client.0.smithi027.stdout:0/957: dwrite d0/d11/d2c/d2e/d55/fca [0,4194304] 0 2024-04-20T10:59:25.231 INFO:tasks.workunit.client.0.smithi027.stdout:3/827: symlink d2/d18/l102 0 2024-04-20T10:59:25.232 INFO:tasks.workunit.client.0.smithi027.stdout:3/828: chown d2/dd/df/c4e 654260 1 2024-04-20T10:59:25.232 INFO:tasks.workunit.client.0.smithi027.stdout:8/852: mknod d9/de/d1e/d4a/d6d/d7d/c11f 0 2024-04-20T10:59:25.236 INFO:tasks.workunit.client.0.smithi027.stdout:0/958: unlink d0/d11/f65 0 2024-04-20T10:59:25.236 INFO:tasks.workunit.client.0.smithi027.stdout:0/959: readlink d0/d11/d4b/d5b/d6f/d82/lbb 0 2024-04-20T10:59:25.236 INFO:tasks.workunit.client.0.smithi027.stdout:3/829: creat d2/dd/d57/d8d/dc6/de1/f103 x:0 0 0 2024-04-20T10:59:25.236 INFO:tasks.workunit.client.0.smithi027.stdout:3/830: truncate d2/dd/df/d33/fca 233313 0 2024-04-20T10:59:25.237 INFO:tasks.workunit.client.0.smithi027.stdout:8/853: mkdir d9/de/d1e/d49/d120 0 2024-04-20T10:59:25.237 INFO:tasks.workunit.client.0.smithi027.stdout:8/854: stat d9/de/d1e/d11b/fe0 0 2024-04-20T10:59:25.238 INFO:tasks.workunit.client.0.smithi027.stdout:9/711: rename d4/d1b/d1f/f9b to d4/d14/d43/fd1 0 2024-04-20T10:59:25.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/855: dread d9/de/d1e/d11b/fe0 [0,4194304] 0 2024-04-20T10:59:25.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/856: fsync d9/de/d20/d53/f77 0 2024-04-20T10:59:25.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/960: creat d0/d59/dbf/f145 x:0 0 0 2024-04-20T10:59:25.243 INFO:tasks.workunit.client.0.smithi027.stdout:4/786: sync 2024-04-20T10:59:25.260 INFO:tasks.workunit.client.0.smithi027.stdout:9/712: dwrite d4/d1b/d1f/d5a/db7/fcf [0,4194304] 0 2024-04-20T10:59:25.261 INFO:tasks.workunit.client.0.smithi027.stdout:3/831: mkdir d2/d94/dc4/d104 0 2024-04-20T10:59:25.262 INFO:tasks.workunit.client.0.smithi027.stdout:8/857: unlink d9/de/d1e/d4a/d6d/d7d/c105 0 2024-04-20T10:59:25.266 INFO:tasks.workunit.client.0.smithi027.stdout:0/961: dwrite d0/d11/d2c/d2e/d2b/d3d/fb3 [0,4194304] 0 2024-04-20T10:59:25.267 INFO:tasks.workunit.client.0.smithi027.stdout:0/962: stat d0/d59/d9e/f10b 0 2024-04-20T10:59:25.268 INFO:tasks.workunit.client.0.smithi027.stdout:8/858: unlink d9/de/d1e/d4a/d6d/laa 0 2024-04-20T10:59:25.268 INFO:tasks.workunit.client.0.smithi027.stdout:3/832: write d2/dd/f10 [4485838,115831] 0 2024-04-20T10:59:25.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/713: link d4/d1b/d8f/f90 d4/d2a/fd2 0 2024-04-20T10:59:25.270 INFO:tasks.workunit.client.0.smithi027.stdout:8/859: getdents d9/de/d20/dbf 0 2024-04-20T10:59:25.271 INFO:tasks.workunit.client.0.smithi027.stdout:4/787: dwrite d0/d14/d6b/da3/d1b/d33/f9a [0,4194304] 0 2024-04-20T10:59:25.271 INFO:tasks.workunit.client.0.smithi027.stdout:3/833: mkdir d2/dd/df/d33/d105 0 2024-04-20T10:59:25.272 INFO:tasks.workunit.client.0.smithi027.stdout:9/714: truncate d4/d1b/d31/d23/d29/f35 512395 0 2024-04-20T10:59:25.272 INFO:tasks.workunit.client.0.smithi027.stdout:9/715: write d4/d1b/d1f/d5a/fbc [392691,47134] 0 2024-04-20T10:59:25.272 INFO:tasks.workunit.client.0.smithi027.stdout:3/834: write d2/d18/f7b [1212672,86090] 0 2024-04-20T10:59:25.273 INFO:tasks.workunit.client.0.smithi027.stdout:0/963: rename d0/d11/d2c/d2e/d2b/d7b/f1c to d0/d11/d2c/d2e/d35/f146 0 2024-04-20T10:59:25.275 INFO:tasks.workunit.client.0.smithi027.stdout:4/788: getdents d0/d14/d6b/da3/d18/d10a 0 2024-04-20T10:59:25.276 INFO:tasks.workunit.client.0.smithi027.stdout:9/716: getdents d4/d1b/d1f/d5a/db7 0 2024-04-20T10:59:25.278 INFO:tasks.workunit.client.0.smithi027.stdout:3/835: mkdir d2/dd/d106 0 2024-04-20T10:59:25.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/860: rename d9/de/d1e/d4a/d6d/db7/d97/fa4 to d9/de/d1e/d49/d120/f121 0 2024-04-20T10:59:25.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/861: stat d9/de/d1e/f68 0 2024-04-20T10:59:25.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/862: write d9/de/d20/f9d [377104,27888] 0 2024-04-20T10:59:25.282 INFO:tasks.workunit.client.0.smithi027.stdout:4/789: mkdir d0/d14/d6b/d92/d11e 0 2024-04-20T10:59:25.283 INFO:tasks.workunit.client.0.smithi027.stdout:4/790: read d0/d14/d6b/da3/d1b/d33/d5c/fd2 [74634,124218] 0 2024-04-20T10:59:25.283 INFO:tasks.workunit.client.0.smithi027.stdout:4/791: truncate d0/d14/d6b/da3/d18/f54 4212874 0 2024-04-20T10:59:25.283 INFO:tasks.workunit.client.0.smithi027.stdout:9/717: creat d4/d1b/d8f/fd3 x:0 0 0 2024-04-20T10:59:25.284 INFO:tasks.workunit.client.0.smithi027.stdout:3/836: rmdir d2/d94 39 2024-04-20T10:59:25.288 INFO:tasks.workunit.client.0.smithi027.stdout:6/838: sync 2024-04-20T10:59:25.288 INFO:tasks.workunit.client.0.smithi027.stdout:2/741: sync 2024-04-20T10:59:25.288 INFO:tasks.workunit.client.0.smithi027.stdout:7/801: sync 2024-04-20T10:59:25.288 INFO:tasks.workunit.client.0.smithi027.stdout:5/748: sync 2024-04-20T10:59:25.288 INFO:tasks.workunit.client.0.smithi027.stdout:7/802: chown d4/l50 2 1 2024-04-20T10:59:25.288 INFO:tasks.workunit.client.0.smithi027.stdout:1/993: sync 2024-04-20T10:59:25.290 INFO:tasks.workunit.client.0.smithi027.stdout:0/964: dwrite d0/d59/dbf/f12f [0,4194304] 0 2024-04-20T10:59:25.290 INFO:tasks.workunit.client.0.smithi027.stdout:8/863: mkdir d9/de/d20/dfd/d122 0 2024-04-20T10:59:25.290 INFO:tasks.workunit.client.0.smithi027.stdout:8/864: chown d9/de/d1e/d4a/d6d/d75/fee 3379 1 2024-04-20T10:59:25.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.290+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc640005850 con 0x7fc65c0b54f0 2024-04-20T10:59:25.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.291+0000 7fc64affd640 1 -- 172.21.15.27:0/651665962 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6933 (secure 0 0 0) 0x7fc64c05d110 con 0x7fc65c0b54f0 2024-04-20T10:59:25.292 INFO:tasks.workunit.client.0.smithi027.stdout:9/718: creat d4/d1b/daf/fd4 x:0 0 0 2024-04-20T10:59:25.292 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:25.292 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5029,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:59:25.292 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-04-20T10:59:25.293 INFO:tasks.workunit.client.0.smithi027.stdout:6/839: getdents da/d21 0 2024-04-20T10:59:25.293 INFO:tasks.workunit.client.0.smithi027.stdout:2/742: creat d5/dc/d16/dd0/dd5/ff2 x:0 0 0 2024-04-20T10:59:25.294 INFO:tasks.workunit.client.0.smithi027.stdout:5/749: symlink d19/d1b/d1c/d1e/d83/lf2 0 2024-04-20T10:59:25.294 INFO:tasks.workunit.client.0.smithi027.stdout:7/803: getdents d4/d5/d8/d21 0 2024-04-20T10:59:25.295 INFO:tasks.workunit.client.0.smithi027.stdout:1/994: mkdir d3/d7/d28/d92/d15e 0 2024-04-20T10:59:25.295 INFO:tasks.workunit.client.0.smithi027.stdout:1/995: write d3/d41/f61 [472462,8022] 0 2024-04-20T10:59:25.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc63c06c8f0 msgr2=0x7fc63c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:25.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc63c06c8f0 0x7fc63c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc65c1545a0 tx=0x7fc650009190 comp rx=0 tx=0).stop 2024-04-20T10:59:25.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0b54f0 msgr2=0x7fc65c153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:25.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0b54f0 0x7fc65c153d60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00c400 tx=0x7fc64c00c8c0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 shutdown_connections 2024-04-20T10:59:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc63c06c8f0 0x7fc63c06edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc65c155950 0x7fc65c14cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc65c0d0c90 0x7fc65c14c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 --2- 172.21.15.27:0/651665962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65c0b54f0 0x7fc65c153d60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 >> 172.21.15.27:0/651665962 conn(0x7fc65c0676c0 msgr2=0x7fc65c067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 shutdown_connections 2024-04-20T10:59:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:25.295+0000 7fc660814640 1 -- 172.21.15.27:0/651665962 wait complete. 2024-04-20T10:59:25.297 INFO:tasks.workunit.client.0.smithi027.stdout:0/965: truncate d0/d11/d2c/fcd 358797 0 2024-04-20T10:59:25.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/719: rename d4/d1b/d31/d23/f99 to d4/d1b/d1f/d5a/db7/fd5 0 2024-04-20T10:59:25.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/720: dread - d4/d1b/daf/fd4 zero size 2024-04-20T10:59:25.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/721: dread - d4/d1b/d1f/d5a/fc9 zero size 2024-04-20T10:59:25.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/722: readlink d4/d1b/d1f/d45/lcd 0 2024-04-20T10:59:25.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/723: getdents d4/d1b/d31/dd0 0 2024-04-20T10:59:25.298 INFO:tasks.workunit.client.0.smithi027.stdout:6/840: getdents da/d21/dab/d1f/dbf 0 2024-04-20T10:59:25.299 INFO:tasks.workunit.client.0.smithi027.stdout:2/743: rmdir d5/d44/d7c 39 2024-04-20T10:59:25.299 INFO:tasks.workunit.client.0.smithi027.stdout:2/744: chown d5/dc/d16/d2a/c83 0 1 2024-04-20T10:59:25.299 INFO:tasks.workunit.client.0.smithi027.stdout:7/804: dread d4/d5/d8/f38 [0,4194304] 0 2024-04-20T10:59:25.304 INFO:tasks.workunit.client.0.smithi027.stdout:9/724: creat d4/d1b/d1f/d57/db0/fd6 x:0 0 0 2024-04-20T10:59:25.304 INFO:tasks.workunit.client.0.smithi027.stdout:4/792: dwrite d0/f13 [4194304,4194304] 0 2024-04-20T10:59:25.305 INFO:tasks.workunit.client.0.smithi027.stdout:6/841: mkdir da/dcc/d108/df1/d111 0 2024-04-20T10:59:25.307 INFO:tasks.workunit.client.0.smithi027.stdout:7/805: getdents d4/d1b/d4a 0 2024-04-20T10:59:25.309 INFO:tasks.workunit.client.0.smithi027.stdout:9/725: creat d4/d1b/d50/d5d/fd7 x:0 0 0 2024-04-20T10:59:25.309 INFO:tasks.workunit.client.0.smithi027.stdout:3/837: dwrite d2/dd/df/d23/fe6 [0,4194304] 0 2024-04-20T10:59:25.309 INFO:tasks.workunit.client.0.smithi027.stdout:3/838: fdatasync d2/dd/f58 0 2024-04-20T10:59:25.309 INFO:tasks.workunit.client.0.smithi027.stdout:3/839: readlink d2/dd/d57/l92 0 2024-04-20T10:59:25.309 INFO:tasks.workunit.client.0.smithi027.stdout:3/840: stat d2/d18/f29 0 2024-04-20T10:59:25.309 INFO:tasks.workunit.client.0.smithi027.stdout:3/841: fdatasync d2/d18/d2c/d4c/d67/fe0 0 2024-04-20T10:59:25.309 INFO:tasks.workunit.client.0.smithi027.stdout:3/842: write d2/dd/d57/ffa [911849,7244] 0 2024-04-20T10:59:25.311 INFO:tasks.workunit.client.0.smithi027.stdout:7/806: creat d4/da2/dd1/f103 x:0 0 0 2024-04-20T10:59:25.311 INFO:tasks.workunit.client.0.smithi027.stdout:7/807: write d4/d1b/f63 [4737780,120618] 0 2024-04-20T10:59:25.312 INFO:tasks.workunit.client.0.smithi027.stdout:6/842: mknod da/dcc/d108/d6b/c112 0 2024-04-20T10:59:25.313 INFO:tasks.workunit.client.0.smithi027.stdout:3/843: mkdir d2/d18/d2c/d4c/d107 0 2024-04-20T10:59:25.316 INFO:tasks.workunit.client.0.smithi027.stdout:7/808: mkdir d4/da2/dd1/dfb/d104 0 2024-04-20T10:59:25.316 INFO:tasks.workunit.client.0.smithi027.stdout:7/809: fdatasync d4/d5/d6f/fe4 0 2024-04-20T10:59:25.316 INFO:tasks.workunit.client.0.smithi027.stdout:4/793: dread d0/d14/d6b/da3/d53/d6a/da1/db0/fbb [4194304,4194304] 0 2024-04-20T10:59:25.316 INFO:tasks.workunit.client.0.smithi027.stdout:4/794: fdatasync d0/d5e/d6c/fb2 0 2024-04-20T10:59:25.316 INFO:tasks.workunit.client.0.smithi027.stdout:6/843: rmdir da/d21 39 2024-04-20T10:59:25.319 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T10:59:25.319 INFO:tasks.workunit.client.0.smithi027.stdout:3/844: link d2/d18/d2c/f78 d2/d80/f108 0 2024-04-20T10:59:25.320 INFO:tasks.workunit.client.0.smithi027.stdout:3/845: dread d2/dd/d22/fbf [0,4194304] 0 2024-04-20T10:59:25.320 INFO:tasks.workunit.client.0.smithi027.stdout:5/750: dwrite d19/d1b/d1c/d1e/d22/db1/f55 [0,4194304] 0 2024-04-20T10:59:25.321 INFO:tasks.workunit.client.0.smithi027.stdout:8/865: getdents d9/de/d1e/d4a/d6d/db7/d97 0 2024-04-20T10:59:25.322 INFO:tasks.workunit.client.0.smithi027.stdout:1/996: write d3/d7/d38/ddf/d73/fc4 [1780354,6577] 0 2024-04-20T10:59:25.322 INFO:tasks.workunit.client.0.smithi027.stdout:1/997: fsync d3/d7/d38/ddf/dcc/f121 0 2024-04-20T10:59:25.323 INFO:tasks.workunit.client.0.smithi027.stdout:2/745: dwrite d5/dc/d16/d37/fc8 [0,4194304] 0 2024-04-20T10:59:25.324 INFO:tasks.workunit.client.0.smithi027.stdout:7/810: rename d4/d5/d8/d46/f89 to d4/d5/d8/d21/f105 0 2024-04-20T10:59:25.325 INFO:tasks.workunit.client.0.smithi027.stdout:3/846: creat d2/d18/d2c/f109 x:0 0 0 2024-04-20T10:59:25.325 INFO:tasks.workunit.client.0.smithi027.stdout:3/847: write d2/dd/d57/d8d/fe9 [73119,65387] 0 2024-04-20T10:59:25.326 INFO:tasks.workunit.client.0.smithi027.stdout:5/751: mkdir d19/d1b/d1c/d1e/da3/df3 0 2024-04-20T10:59:25.328 INFO:tasks.workunit.client.0.smithi027.stdout:8/866: creat d9/de/d1e/d4a/d6d/dea/db3/f123 x:0 0 0 2024-04-20T10:59:25.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/746: dread d5/dc/fd7 [0,4194304] 0 2024-04-20T10:59:25.334 INFO:tasks.workunit.client.0.smithi027.stdout:1/998: rename d3/d20/c5b to d3/d45/c15f 0 2024-04-20T10:59:25.335 INFO:tasks.workunit.client.0.smithi027.stdout:7/811: mkdir d4/d5/d8/d46/d106 0 2024-04-20T10:59:25.335 INFO:tasks.workunit.client.0.smithi027.stdout:7/812: readlink d4/d1b/d4a/d62/l7b 0 2024-04-20T10:59:25.335 INFO:tasks.workunit.client.0.smithi027.stdout:1/999: read - d3/d7/d38/ddf/dcc/f15c zero size 2024-04-20T10:59:25.335 INFO:tasks.workunit.client.0.smithi027.stdout:0/966: dwrite d0/d11/d2c/d2e/d2b/d7b/d34/d5a/fcc [0,4194304] 0 2024-04-20T10:59:25.335 INFO:tasks.workunit.client.0.smithi027.stdout:9/726: dwrite d4/d74/d72/f73 [0,4194304] 0 2024-04-20T10:59:25.335 INFO:tasks.workunit.client.0.smithi027.stdout:9/727: dread - d4/d14/fa3 zero size 2024-04-20T10:59:25.335 INFO:tasks.workunit.client.0.smithi027.stdout:9/728: stat d4/d14/d43 0 2024-04-20T10:59:25.336 INFO:tasks.workunit.client.0.smithi027.stdout:8/867: creat d9/de/d20/dfd/f124 x:0 0 0 2024-04-20T10:59:25.336 INFO:tasks.workunit.client.0.smithi027.stdout:8/868: write d9/de/d1e/d4a/fde [974512,128330] 0 2024-04-20T10:59:25.338 INFO:tasks.workunit.client.0.smithi027.stdout:2/747: symlink d5/dc/d16/d37/lf3 0 2024-04-20T10:59:25.338 INFO:tasks.workunit.client.0.smithi027.stdout:2/748: stat d5/dc/d16/d37/de2/d8e/fc6 0 2024-04-20T10:59:25.339 INFO:tasks.workunit.client.0.smithi027.stdout:7/813: mknod d4/da2/c107 0 2024-04-20T10:59:25.340 INFO:tasks.workunit.client.0.smithi027.stdout:5/752: dread d19/d1b/d1c/d1e/d3c/f31 [4194304,4194304] 0 2024-04-20T10:59:25.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/848: dwrite d2/d94/f95 [0,4194304] 0 2024-04-20T10:59:25.342 INFO:tasks.workunit.client.0.smithi027.stdout:0/967: creat d0/d11/d4b/d5b/d6f/d82/f147 x:0 0 0 2024-04-20T10:59:25.344 INFO:tasks.workunit.client.0.smithi027.stdout:6/844: dread da/d21/dab/d19/f6a [0,4194304] 0 2024-04-20T10:59:25.344 INFO:tasks.workunit.client.0.smithi027.stdout:6/845: chown da/d21/d28/d2d/d53/d9a/lcb 6876 1 2024-04-20T10:59:25.344 INFO:tasks.workunit.client.0.smithi027.stdout:6/846: fdatasync da/d21/d96/fe3 0 2024-04-20T10:59:25.347 INFO:tasks.workunit.client.0.smithi027.stdout:9/729: symlink d4/d1b/d1f/ld8 0 2024-04-20T10:59:25.347 INFO:tasks.workunit.client.0.smithi027.stdout:9/730: readlink d4/d14/l1d 0 2024-04-20T10:59:25.347 INFO:tasks.workunit.client.0.smithi027.stdout:8/869: mknod d9/de/d1e/dc1/c125 0 2024-04-20T10:59:25.348 INFO:tasks.workunit.client.0.smithi027.stdout:8/870: write d9/de/d1e/d4a/d6d/db7/d5b/ff7 [98049,111859] 0 2024-04-20T10:59:25.348 INFO:tasks.workunit.client.0.smithi027.stdout:8/871: write d9/de/d1e/d4a/d6d/dea/d59/d74/fb9 [1286063,55583] 0 2024-04-20T10:59:25.348 INFO:tasks.workunit.client.0.smithi027.stdout:9/731: dread d4/d1b/d31/f3d [0,4194304] 0 2024-04-20T10:59:25.349 INFO:tasks.workunit.client.0.smithi027.stdout:2/749: mknod d5/d26/d47/cf4 0 2024-04-20T10:59:25.350 INFO:tasks.workunit.client.0.smithi027.stdout:3/849: dread d2/f66 [0,4194304] 0 2024-04-20T10:59:25.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/753: unlink d19/d1b/d1c/d1e/d22/c95 0 2024-04-20T10:59:25.355 INFO:tasks.workunit.client.0.smithi027.stdout:6/847: creat da/dcc/d108/d6b/de0/f113 x:0 0 0 2024-04-20T10:59:25.358 INFO:tasks.workunit.client.0.smithi027.stdout:8/872: rename d9/de/da3/dab to d9/de/d20/dfd/d122/d126 0 2024-04-20T10:59:25.358 INFO:tasks.workunit.client.0.smithi027.stdout:8/873: truncate d9/de/d20/dfd/d122/d126/fdb 185790 0 2024-04-20T10:59:25.358 INFO:tasks.workunit.client.0.smithi027.stdout:8/874: write d9/de/d1e/d11b/dbd/fc7 [4850465,111896] 0 2024-04-20T10:59:25.359 INFO:tasks.workunit.client.0.smithi027.stdout:5/754: symlink d19/d1b/d1c/d1e/da3/df3/lf4 0 2024-04-20T10:59:25.361 INFO:tasks.workunit.client.0.smithi027.stdout:2/750: rename d5/dc/lc1 to d5/d26/d47/lf5 0 2024-04-20T10:59:25.362 INFO:tasks.workunit.client.0.smithi027.stdout:0/968: dwrite d0/d59/f73 [0,4194304] 0 2024-04-20T10:59:25.369 INFO:tasks.workunit.client.0.smithi027.stdout:5/755: write d19/d1b/d1c/f45 [2186742,67201] 0 2024-04-20T10:59:25.369 INFO:tasks.workunit.client.0.smithi027.stdout:5/756: write d19/d1b/d1c/d1e/d58/db4/fe3 [338754,113111] 0 2024-04-20T10:59:25.369 INFO:tasks.workunit.client.0.smithi027.stdout:5/757: truncate d19/d1b/d1c/d1e/d58/db4/fe3 577846 0 2024-04-20T10:59:25.369 INFO:tasks.workunit.client.0.smithi027.stdout:5/758: write d19/d1b/d1c/d1e/d25/d40/d82/fbc [1446494,14008] 0 2024-04-20T10:59:25.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/875: creat d9/de/da3/f127 x:0 0 0 2024-04-20T10:59:25.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/876: write d9/de/d1e/d4a/d6d/dea/f113 [680819,5415] 0 2024-04-20T10:59:25.370 INFO:tasks.workunit.client.0.smithi027.stdout:2/751: link d5/dc/l1d d5/dc/d16/d32/dab/db9/lf6 0 2024-04-20T10:59:25.372 INFO:tasks.workunit.client.0.smithi027.stdout:5/759: rename d19/f9b to d19/d1b/d1c/d1e/d22/ff5 0 2024-04-20T10:59:25.372 INFO:tasks.workunit.client.0.smithi027.stdout:8/877: mknod d9/de/d64/d6c/c128 0 2024-04-20T10:59:25.372 INFO:tasks.workunit.client.0.smithi027.stdout:8/878: readlink d9/de/d1e/lb2 0 2024-04-20T10:59:25.373 INFO:tasks.workunit.client.0.smithi027.stdout:2/752: mknod d5/d44/d7c/dca/cf7 0 2024-04-20T10:59:25.374 INFO:tasks.workunit.client.0.smithi027.stdout:5/760: rmdir d19/d1b/d1c/d1e/da3/db9 39 2024-04-20T10:59:25.374 INFO:tasks.workunit.client.0.smithi027.stdout:5/761: write d19/d1b/d1c/dbd/dbe/f9e [1939877,88486] 0 2024-04-20T10:59:25.374 INFO:tasks.workunit.client.0.smithi027.stdout:6/848: dwrite da/d21/dab/d1f/dbf/fd7 [0,4194304] 0 2024-04-20T10:59:25.374 INFO:tasks.workunit.client.0.smithi027.stdout:6/849: fsync da/d21/dab/d19/f93 0 2024-04-20T10:59:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:8/879: truncate d9/f34 1355999 0 2024-04-20T10:59:25.375 INFO:tasks.workunit.client.0.smithi027.stdout:8/880: readlink d9/de/da3/l58 0 2024-04-20T10:59:25.376 INFO:tasks.workunit.client.0.smithi027.stdout:2/753: rename d5/dc/d16/d37/de2/l87 to d5/dc/d16/d37/de2/dac/lf8 0 2024-04-20T10:59:25.376 INFO:tasks.workunit.client.0.smithi027.stdout:5/762: chown d19/d1b/d1c/d1e/d3c/f3d 15 1 2024-04-20T10:59:25.376 INFO:tasks.workunit.client.0.smithi027.stdout:5/763: dread - d19/fc9 zero size 2024-04-20T10:59:25.376 INFO:tasks.workunit.client.0.smithi027.stdout:5/764: fdatasync fc 0 2024-04-20T10:59:25.377 INFO:tasks.workunit.client.0.smithi027.stdout:6/850: mknod da/d21/d28/d2d/d53/dae/ddf/c114 0 2024-04-20T10:59:25.378 INFO:tasks.workunit.client.0.smithi027.stdout:8/881: creat d9/de/d1e/d11b/dbd/f129 x:0 0 0 2024-04-20T10:59:25.378 INFO:tasks.workunit.client.0.smithi027.stdout:8/882: write d9/de/d1e/d4a/d6d/d75/d7c/f109 [441120,74883] 0 2024-04-20T10:59:25.380 INFO:tasks.workunit.client.0.smithi027.stdout:2/754: creat d5/dc/d16/d32/dab/dba/ff9 x:0 0 0 2024-04-20T10:59:25.380 INFO:tasks.workunit.client.0.smithi027.stdout:2/755: truncate d5/dc/d16/dd0/dd5/ff2 581227 0 2024-04-20T10:59:25.380 INFO:tasks.workunit.client.0.smithi027.stdout:5/765: mknod d19/d1b/d1c/d1e/d22/db1/cf6 0 2024-04-20T10:59:25.380 INFO:tasks.workunit.client.0.smithi027.stdout:5/766: dread - d19/d1b/d1c/d1e/d58/db4/fe4 zero size 2024-04-20T10:59:25.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/851: symlink da/dcc/d108/d6b/d65/l115 0 2024-04-20T10:59:25.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/852: dread - da/dcc/d108/d6b/de0/f113 zero size 2024-04-20T10:59:25.382 INFO:tasks.workunit.client.0.smithi027.stdout:2/756: rmdir d5/dc/d16/d32/dab 39 2024-04-20T10:59:25.384 INFO:tasks.workunit.client.0.smithi027.stdout:5/767: rename d19/d1b/d1c/d1e/d22/d68/d75/cef to d19/d1b/d1c/d1e/cf7 0 2024-04-20T10:59:25.385 INFO:tasks.workunit.client.0.smithi027.stdout:0/969: dwrite d0/d11/d2c/da1/fe3 [4194304,4194304] 0 2024-04-20T10:59:25.388 INFO:tasks.workunit.client.0.smithi027.stdout:2/757: rename d5/dc/d16/db1/fd4 to d5/d26/d47/ffa 0 2024-04-20T10:59:25.388 INFO:tasks.workunit.client.0.smithi027.stdout:6/853: mkdir da/dcc/d108/d6b/d10d/d116 0 2024-04-20T10:59:25.388 INFO:tasks.workunit.client.0.smithi027.stdout:6/854: chown da/dcc/d108/d32/l7a 13110036 1 2024-04-20T10:59:25.388 INFO:tasks.workunit.client.0.smithi027.stdout:6/855: dread - da/d21/d28/d54/fdb zero size 2024-04-20T10:59:25.388 INFO:tasks.workunit.client.0.smithi027.stdout:5/768: rmdir d19 39 2024-04-20T10:59:25.388 INFO:tasks.workunit.client.0.smithi027.stdout:5/769: stat d19/d1b/d1c/d1e/d22/db1/dcf/c7c 0 2024-04-20T10:59:25.389 INFO:tasks.workunit.client.0.smithi027.stdout:0/970: rename d0/d11/d2c/d2e/d2b/d7b/dcf/f112 to d0/d11/f148 0 2024-04-20T10:59:25.390 INFO:tasks.workunit.client.0.smithi027.stdout:0/971: dread - d0/d11/d2c/d2e/d2b/d7b/d64/d13a/f141 zero size 2024-04-20T10:59:25.390 INFO:tasks.workunit.client.0.smithi027.stdout:2/758: creat d5/d26/ffb x:0 0 0 2024-04-20T10:59:25.390 INFO:tasks.workunit.client.0.smithi027.stdout:6/856: symlink da/dcc/d108/d6b/d65/l117 0 2024-04-20T10:59:25.391 INFO:tasks.workunit.client.0.smithi027.stdout:5/770: unlink d19/d1b/d1c/d1e/d3c/f39 0 2024-04-20T10:59:25.392 INFO:tasks.workunit.client.0.smithi027.stdout:0/972: mknod d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/c149 0 2024-04-20T10:59:25.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/759: symlink d5/dc/d16/d37/de2/d8e/lfc 0 2024-04-20T10:59:25.394 INFO:tasks.workunit.client.0.smithi027.stdout:8/883: dwrite d9/de/d1e/d4a/d6d/d75/f11e [0,4194304] 0 2024-04-20T10:59:25.394 INFO:tasks.workunit.client.0.smithi027.stdout:6/857: getdents da/d21/dab/d7e/daf 0 2024-04-20T10:59:25.398 INFO:tasks.workunit.client.0.smithi027.stdout:8/884: mknod d9/de/d1e/d4a/d6d/db7/d97/c12a 0 2024-04-20T10:59:25.399 INFO:tasks.workunit.client.0.smithi027.stdout:6/858: truncate da/dcc/d108/fb4 1994729 0 2024-04-20T10:59:25.400 INFO:tasks.workunit.client.0.smithi027.stdout:5/771: dread d19/d1b/d1c/d1e/d58/f94 [0,4194304] 0 2024-04-20T10:59:25.401 INFO:tasks.workunit.client.0.smithi027.stdout:8/885: creat d9/de/d1e/d4a/d6d/d75/f12b x:0 0 0 2024-04-20T10:59:25.406 INFO:tasks.workunit.client.0.smithi027.stdout:6/859: link da/d21/dab/d7e/lc8 da/dcc/d108/d6b/d10d/l118 0 2024-04-20T10:59:25.407 INFO:tasks.workunit.client.0.smithi027.stdout:2/760: dread d5/dc/d16/d37/f46 [0,4194304] 0 2024-04-20T10:59:25.407 INFO:tasks.workunit.client.0.smithi027.stdout:2/761: chown d5/dc/f1c 103099296 1 2024-04-20T10:59:25.408 INFO:tasks.workunit.client.0.smithi027.stdout:8/886: creat d9/de/d20/f12c x:0 0 0 2024-04-20T10:59:25.418 INFO:tasks.workunit.client.0.smithi027.stdout:0/973: write d0/d11/d2c/d2e/d55/f56 [2518642,86678] 0 2024-04-20T10:59:25.418 INFO:tasks.workunit.client.0.smithi027.stdout:0/974: chown d0/d11/d2c/d2e/d2b/d7b/d12/d1a/l10c 2 1 2024-04-20T10:59:25.419 INFO:tasks.workunit.client.0.smithi027.stdout:8/887: mknod d9/de/da3/d3c/c12d 0 2024-04-20T10:59:25.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/888: read d9/f54 [662275,118622] 0 2024-04-20T10:59:25.423 INFO:tasks.workunit.client.0.smithi027.stdout:0/975: unlink d0/d11/d2c/d2e/d2b/d7b/d12/f111 0 2024-04-20T10:59:25.424 INFO:tasks.workunit.client.0.smithi027.stdout:8/889: mkdir d9/de/d1e/d4a/d6d/dea/d62/d12e 0 2024-04-20T10:59:25.429 INFO:tasks.workunit.client.0.smithi027.stdout:2/762: dwrite d5/d1e/d21/f3e [0,4194304] 0 2024-04-20T10:59:25.429 INFO:tasks.workunit.client.0.smithi027.stdout:3/850: dwrite d2/dd/d22/fae [4194304,4194304] 0 2024-04-20T10:59:25.430 INFO:tasks.workunit.client.0.smithi027.stdout:0/976: symlink d0/d11/d2c/d2e/d2b/d7b/d12/d25/d10a/l14a 0 2024-04-20T10:59:25.432 INFO:tasks.workunit.client.0.smithi027.stdout:2/763: truncate d5/dc/d16/d37/de2/fbf 1130964 0 2024-04-20T10:59:25.433 INFO:tasks.workunit.client.0.smithi027.stdout:3/851: fsync d2/dd/d57/d8d/fd2 0 2024-04-20T10:59:25.433 INFO:tasks.workunit.client.0.smithi027.stdout:3/852: dread - d2/d18/d2c/d4c/d67/d6c/fdc zero size 2024-04-20T10:59:25.434 INFO:tasks.workunit.client.0.smithi027.stdout:0/977: creat d0/d11/d2c/d2e/d2b/d7b/d34/d5a/f14b x:0 0 0 2024-04-20T10:59:25.440 INFO:tasks.workunit.client.0.smithi027.stdout:2/764: rename d5/d44/d7c/dca to d5/dc/d16/d37/de2/dac/d8f/dfd 0 2024-04-20T10:59:25.441 INFO:tasks.workunit.client.0.smithi027.stdout:3/853: symlink d2/d94/l10a 0 2024-04-20T10:59:25.442 INFO:tasks.workunit.client.0.smithi027.stdout:2/765: creat d5/d44/ffe x:0 0 0 2024-04-20T10:59:25.442 INFO:tasks.workunit.client.0.smithi027.stdout:6/860: dwrite da/dd/f69 [0,4194304] 0 2024-04-20T10:59:25.444 INFO:tasks.workunit.client.0.smithi027.stdout:3/854: getdents d2/dd 0 2024-04-20T10:59:25.444 INFO:tasks.workunit.client.0.smithi027.stdout:3/855: chown d2/d18/d2c/d4c/d67/d6c/dd0/dd5/cec 1 1 2024-04-20T10:59:25.444 INFO:tasks.workunit.client.0.smithi027.stdout:3/856: fdatasync d2/dd/d57/f98 0 2024-04-20T10:59:25.444 INFO:tasks.workunit.client.0.smithi027.stdout:3/857: truncate d2/dd/d57/d8d/ff3 377892 0 2024-04-20T10:59:25.445 INFO:tasks.workunit.client.0.smithi027.stdout:6/861: unlink da/dcc/d108/d32/f44 0 2024-04-20T10:59:25.446 INFO:tasks.workunit.client.0.smithi027.stdout:6/862: chown da/d21/d28/d2d/d53/d9a/lcb 1 1 2024-04-20T10:59:25.446 INFO:tasks.workunit.client.0.smithi027.stdout:6/863: fdatasync da/d21/d96/f106 0 2024-04-20T10:59:25.446 INFO:tasks.workunit.client.0.smithi027.stdout:6/864: fdatasync da/dcc/d108/f41 0 2024-04-20T10:59:25.446 INFO:tasks.workunit.client.0.smithi027.stdout:6/865: truncate da/d21/dab/fb5 496137 0 2024-04-20T10:59:25.446 INFO:tasks.workunit.client.0.smithi027.stdout:3/858: getdents d2/d7f 0 2024-04-20T10:59:25.448 INFO:tasks.workunit.client.0.smithi027.stdout:3/859: dread d2/dd/df/f3a [0,4194304] 0 2024-04-20T10:59:25.450 INFO:tasks.workunit.client.0.smithi027.stdout:6/866: rename da/d21/d28/d2d/fd4 to da/dd/d3d/db9/f119 0 2024-04-20T10:59:25.451 INFO:tasks.workunit.client.0.smithi027.stdout:3/860: mkdir d2/d18/d2c/d4c/d10b 0 2024-04-20T10:59:25.451 INFO:tasks.workunit.client.0.smithi027.stdout:3/861: readlink d2/d18/d2c/lb2 0 2024-04-20T10:59:25.451 INFO:tasks.workunit.client.0.smithi027.stdout:3/862: write d2/fd9 [684746,93752] 0 2024-04-20T10:59:25.452 INFO:tasks.workunit.client.0.smithi027.stdout:6/867: creat da/d21/f11a x:0 0 0 2024-04-20T10:59:25.452 INFO:tasks.workunit.client.0.smithi027.stdout:8/890: dwrite d9/de/d1e/d4a/d6d/dea/d62/ff9 [0,4194304] 0 2024-04-20T10:59:25.453 INFO:tasks.workunit.client.0.smithi027.stdout:3/863: creat d2/dd/d57/de7/dff/f10c x:0 0 0 2024-04-20T10:59:25.461 INFO:tasks.workunit.client.0.smithi027.stdout:3/864: write d2/d18/d2c/f54 [2803670,78363] 0 2024-04-20T10:59:25.463 INFO:tasks.workunit.client.0.smithi027.stdout:3/865: dread d2/dd/df/f3a [0,4194304] 0 2024-04-20T10:59:25.463 INFO:tasks.workunit.client.0.smithi027.stdout:3/866: fdatasync d2/d18/d2e/fb3 0 2024-04-20T10:59:25.463 INFO:tasks.workunit.client.0.smithi027.stdout:3/867: dread - d2/d18/d2e/fb3 zero size 2024-04-20T10:59:25.463 INFO:tasks.workunit.client.0.smithi027.stdout:3/868: fsync d2/d80/f108 0 2024-04-20T10:59:25.463 INFO:tasks.workunit.client.0.smithi027.stdout:6/868: creat da/dcc/d108/d6b/d65/de8/f11b x:0 0 0 2024-04-20T10:59:25.464 INFO:tasks.workunit.client.0.smithi027.stdout:3/869: creat d2/d18/d2c/d4c/d67/d6c/dd0/f10d x:0 0 0 2024-04-20T10:59:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:0/978: dwrite d0/d11/d2c/fcd [0,4194304] 0 2024-04-20T10:59:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:0/979: fsync d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/f104 0 2024-04-20T10:59:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:0/980: chown d0/d11/d2c/d2e/d61 138 1 2024-04-20T10:59:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:0/981: chown d0/d11/d4b/d5b/d6f/d82/c116 1306 1 2024-04-20T10:59:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:0/982: write d0/d11/d5c/f74 [1856378,44351] 0 2024-04-20T10:59:25.469 INFO:tasks.workunit.client.0.smithi027.stdout:6/869: getdents da/dcc/d108 0 2024-04-20T10:59:25.471 INFO:tasks.workunit.client.0.smithi027.stdout:0/983: rename d0/d11/d4b/d5b/d6f/d82/f147 to d0/f14c 0 2024-04-20T10:59:25.473 INFO:tasks.workunit.client.0.smithi027.stdout:6/870: symlink da/d21/dab/d7e/l11c 0 2024-04-20T10:59:25.473 INFO:tasks.workunit.client.0.smithi027.stdout:6/871: chown da/d21/dab/f4d 16246059 1 2024-04-20T10:59:25.473 INFO:tasks.workunit.client.0.smithi027.stdout:0/984: getdents d0/d11/d2c/d2e/d55 0 2024-04-20T10:59:25.475 INFO:tasks.workunit.client.0.smithi027.stdout:0/985: dread d0/d59/d9e/fb7 [0,4194304] 0 2024-04-20T10:59:25.476 INFO:tasks.workunit.client.0.smithi027.stdout:6/872: link da/dcc/d108/d6b/c8e da/d21/d96/c11d 0 2024-04-20T10:59:25.477 INFO:tasks.workunit.client.0.smithi027.stdout:0/986: creat d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d5d/f14d x:0 0 0 2024-04-20T10:59:25.477 INFO:tasks.workunit.client.0.smithi027.stdout:0/987: write d0/d11/d2c/d2e/d2b/d7b/d34/d5a/f14b [962470,85199] 0 2024-04-20T10:59:25.478 INFO:tasks.workunit.client.0.smithi027.stdout:6/873: mkdir da/dcc/d108/d6b/d10d/d11e 0 2024-04-20T10:59:25.494 INFO:tasks.workunit.client.0.smithi027.stdout:5/772: dwrite d19/d1b/d1c/d1e/d3c/f46 [12582912,4194304] 0 2024-04-20T10:59:25.494 INFO:tasks.workunit.client.0.smithi027.stdout:5/773: truncate d19/d1b/d1c/dbd/dbe/fbf 4460639 0 2024-04-20T10:59:25.494 INFO:tasks.workunit.client.0.smithi027.stdout:0/988: unlink d0/d11/d4b/d5b/d6f/d82/dd3/c121 0 2024-04-20T10:59:25.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/774: mknod d19/d1b/d1c/dbd/dbe/cf8 0 2024-04-20T10:59:25.498 INFO:tasks.workunit.client.0.smithi027.stdout:0/989: mkdir d0/d11/d4b/d5b/d14e 0 2024-04-20T10:59:25.502 INFO:tasks.workunit.client.0.smithi027.stdout:4/795: sync 2024-04-20T10:59:25.502 INFO:tasks.workunit.client.0.smithi027.stdout:9/732: sync 2024-04-20T10:59:25.502 INFO:tasks.workunit.client.0.smithi027.stdout:7/814: sync 2024-04-20T10:59:25.502 INFO:tasks.workunit.client.0.smithi027.stdout:7/815: stat d4/d1b/ff6 0 2024-04-20T10:59:25.503 INFO:tasks.workunit.client.0.smithi027.stdout:0/990: creat d0/d11/d2c/d2e/d61/f14f x:0 0 0 2024-04-20T10:59:25.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/733: creat d4/d1b/d1f/d45/d8a/fd9 x:0 0 0 2024-04-20T10:59:25.510 INFO:tasks.workunit.client.0.smithi027.stdout:7/816: symlink d4/d1b/l108 0 2024-04-20T10:59:25.511 INFO:tasks.workunit.client.0.smithi027.stdout:4/796: mkdir d0/d14/d6b/da3/d1b/dc3/d11f 0 2024-04-20T10:59:25.512 INFO:tasks.workunit.client.0.smithi027.stdout:7/817: creat d4/d5/d8/d46/d106/f109 x:0 0 0 2024-04-20T10:59:25.513 INFO:tasks.workunit.client.0.smithi027.stdout:4/797: creat d0/d14/d6b/da3/d53/f120 x:0 0 0 2024-04-20T10:59:25.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/818: creat d4/d1b/d4a/d7a/f10a x:0 0 0 2024-04-20T10:59:25.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/819: dread - d4/d5/d8/d21/f6e zero size 2024-04-20T10:59:25.515 INFO:tasks.workunit.client.0.smithi027.stdout:4/798: creat d0/d5e/d6c/dd8/f121 x:0 0 0 2024-04-20T10:59:25.515 INFO:tasks.workunit.client.0.smithi027.stdout:7/820: creat d4/d1b/d4a/d62/f10b x:0 0 0 2024-04-20T10:59:25.518 INFO:tasks.workunit.client.0.smithi027.stdout:3/870: dwrite d2/d18/d2c/d4c/d67/d6c/dd0/f10d [0,4194304] 0 2024-04-20T10:59:25.518 INFO:tasks.workunit.client.0.smithi027.stdout:3/871: fdatasync d2/dd/d22/fa2 0 2024-04-20T10:59:25.520 INFO:tasks.workunit.client.0.smithi027.stdout:9/734: dwrite d4/f1e [0,4194304] 0 2024-04-20T10:59:25.521 INFO:tasks.workunit.client.0.smithi027.stdout:3/872: unlink d2/d18/d2c/d4c/c51 0 2024-04-20T10:59:25.521 INFO:tasks.workunit.client.0.smithi027.stdout:3/873: chown d2/d94/f95 414720496 1 2024-04-20T10:59:25.522 INFO:tasks.workunit.client.0.smithi027.stdout:9/735: mkdir d4/d1b/d31/dd0/dda 0 2024-04-20T10:59:25.525 INFO:tasks.workunit.client.0.smithi027.stdout:7/821: dread d4/d5/d81/f8e [0,4194304] 0 2024-04-20T10:59:25.526 INFO:tasks.workunit.client.0.smithi027.stdout:3/874: mknod d2/d18/d2c/df1/c10e 0 2024-04-20T10:59:25.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/736: write d4/d14/d43/fcc [943625,26420] 0 2024-04-20T10:59:25.527 INFO:tasks.workunit.client.0.smithi027.stdout:7/822: symlink d4/d5/l10c 0 2024-04-20T10:59:25.527 INFO:tasks.workunit.client.0.smithi027.stdout:7/823: write d4/d1b/f53 [5225511,80405] 0 2024-04-20T10:59:25.527 INFO:tasks.workunit.client.0.smithi027.stdout:3/875: creat d2/dd/d106/f10f x:0 0 0 2024-04-20T10:59:25.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/737: unlink d4/d1b/d31/fab 0 2024-04-20T10:59:25.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/738: fsync d4/d1b/d50/d5d/fb1 0 2024-04-20T10:59:25.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/739: chown d4/d14/fa3 24408 1 2024-04-20T10:59:25.538 INFO:tasks.workunit.client.0.smithi027.stdout:0/991: dwrite d0/d11/d2c/d2e/d55/f128 [0,4194304] 0 2024-04-20T10:59:25.540 INFO:tasks.workunit.client.0.smithi027.stdout:4/799: dwrite d0/d14/d6b/da3/d1a/f112 [0,4194304] 0 2024-04-20T10:59:25.573 INFO:tasks.workunit.client.0.smithi027.stdout:6/874: dwrite da/d21/dab/d7e/d86/fb0 [4194304,4194304] 0 2024-04-20T10:59:25.573 INFO:tasks.workunit.client.0.smithi027.stdout:5/775: dwrite d19/d1b/d1c/d1e/d3c/f74 [0,4194304] 0 2024-04-20T10:59:25.573 INFO:tasks.workunit.client.0.smithi027.stdout:4/800: dwrite d0/d14/d6b/da3/d1b/d33/d5c/d6f/dd1/f102 [4194304,4194304] 0 2024-04-20T10:59:25.574 INFO:tasks.workunit.client.0.smithi027.stdout:3/876: dwrite d2/dd/f1a [0,4194304] 0 2024-04-20T10:59:25.574 INFO:tasks.workunit.client.0.smithi027.stdout:3/877: write d2/dd/d57/f83 [3338201,32203] 0 2024-04-20T10:59:25.574 INFO:tasks.workunit.client.0.smithi027.stdout:3/878: truncate d2/dd/d22/ff2 873462 0 2024-04-20T10:59:25.788 INFO:tasks.workunit.client.0.smithi027.stdout:7/824: dread d4/d5/d8/d21/fbb [0,4194304] 0 2024-04-20T10:59:25.788 INFO:tasks.workunit.client.0.smithi027.stdout:7/825: write d4/d1b/d4a/fb3 [2547449,100876] 0 2024-04-20T10:59:25.788 INFO:tasks.workunit.client.0.smithi027.stdout:7/826: write d4/d1b/d4a/d62/d87/f8a [645622,123904] 0 2024-04-20T10:59:25.793 INFO:tasks.workunit.client.0.smithi027.stdout:9/740: mkdir d4/d1b/d1f/d45/ddb 0 2024-04-20T10:59:25.793 INFO:tasks.workunit.client.0.smithi027.stdout:0/992: mknod d0/d11/d2c/da1/c150 0 2024-04-20T10:59:25.793 INFO:tasks.workunit.client.0.smithi027.stdout:0/993: stat d0/d11/d2c/d2e/d2b/d7b/d64/d13a/f141 0 2024-04-20T10:59:25.794 INFO:tasks.workunit.client.0.smithi027.stdout:0/994: readlink d0/d11/d2c/d2e/d55/lef 0 2024-04-20T10:59:25.794 INFO:tasks.workunit.client.0.smithi027.stdout:7/827: write d4/d4d/fc5 [3044853,120776] 0 2024-04-20T10:59:25.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/875: symlink da/dcc/d108/d6b/de0/l11f 0 2024-04-20T10:59:25.798 INFO:tasks.workunit.client.0.smithi027.stdout:5/776: symlink d19/d1b/d1c/d1e/d58/lf9 0 2024-04-20T10:59:25.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/801: write d0/d14/d6b/da3/d1a/f112 [4646441,116152] 0 2024-04-20T10:59:25.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/802: write d0/d14/d6b/da3/d18/d10a/f89 [577341,24710] 0 2024-04-20T10:59:25.801 INFO:tasks.workunit.client.0.smithi027.stdout:3/879: mknod d2/dd/df/d33/dfb/c110 0 2024-04-20T10:59:25.801 INFO:tasks.workunit.client.0.smithi027.stdout:3/880: write d2/dd/d57/d8d/feb [1193070,52478] 0 2024-04-20T10:59:25.802 INFO:tasks.workunit.client.0.smithi027.stdout:9/741: creat d4/d1b/d31/fdc x:0 0 0 2024-04-20T10:59:25.803 INFO:tasks.workunit.client.0.smithi027.stdout:9/742: read - d4/d1b/d31/fdc zero size 2024-04-20T10:59:25.803 INFO:tasks.workunit.client.0.smithi027.stdout:6/876: creat da/dcc/d108/df1/f120 x:0 0 0 2024-04-20T10:59:25.805 INFO:tasks.workunit.client.0.smithi027.stdout:5/777: creat d19/d1b/d1c/d1e/d58/db4/ded/ffa x:0 0 0 2024-04-20T10:59:25.805 INFO:tasks.workunit.client.0.smithi027.stdout:5/778: fsync d19/d1b/d1c/dbd/dbe/fd9 0 2024-04-20T10:59:25.807 INFO:tasks.workunit.client.0.smithi027.stdout:9/743: mknod d4/d14/d43/d91/cdd 0 2024-04-20T10:59:25.807 INFO:tasks.workunit.client.0.smithi027.stdout:2/766: sync 2024-04-20T10:59:25.808 INFO:tasks.workunit.client.0.smithi027.stdout:5/779: mknod d19/d1b/d1c/d1e/d22/dca/cfb 0 2024-04-20T10:59:25.809 INFO:tasks.workunit.client.0.smithi027.stdout:9/744: symlink d4/d74/dae/lde 0 2024-04-20T10:59:25.809 INFO:tasks.workunit.client.0.smithi027.stdout:2/767: truncate d5/d26/d47/d8a/f9a 1497727 0 2024-04-20T10:59:25.811 INFO:tasks.workunit.client.0.smithi027.stdout:6/877: mknod da/c121 0 2024-04-20T10:59:25.811 INFO:tasks.workunit.client.0.smithi027.stdout:6/878: chown da/dcc/d108/d6b/d10d/l118 1351558 1 2024-04-20T10:59:25.813 INFO:tasks.workunit.client.0.smithi027.stdout:9/745: creat d4/d1b/d31/dd0/dda/fdf x:0 0 0 2024-04-20T10:59:25.814 INFO:tasks.workunit.client.0.smithi027.stdout:0/995: dwrite d0/d11/d2c/d2e/d2b/d7b/d12/d1a/d47/d7f/f8a [0,4194304] 0 2024-04-20T10:59:25.815 INFO:tasks.workunit.client.0.smithi027.stdout:6/879: rename da/dcc/d108/f41 to da/dd/d3c/f122 0 2024-04-20T10:59:25.815 INFO:tasks.workunit.client.0.smithi027.stdout:6/880: fdatasync da/dd/d3d/f7f 0 2024-04-20T10:59:25.815 INFO:tasks.workunit.client.0.smithi027.stdout:6/881: fdatasync da/d21/d28/d54/f9f 0 2024-04-20T10:59:25.815 INFO:tasks.workunit.client.0.smithi027.stdout:6/882: write da/d21/d96/fa4 [1213823,69876] 0 2024-04-20T10:59:25.816 INFO:tasks.workunit.client.0.smithi027.stdout:8/891: sync 2024-04-20T10:59:25.819 INFO:tasks.workunit.client.0.smithi027.stdout:7/828: dwrite d4/d4d/fc5 [4194304,4194304] 0 2024-04-20T10:59:25.819 INFO:tasks.workunit.client.0.smithi027.stdout:7/829: write d4/d5/f33 [994433,51478] 0 2024-04-20T10:59:25.830 INFO:tasks.workunit.client.0.smithi027.stdout:2/768: dwrite d5/dc/d16/d2a/f93 [0,4194304] 0 2024-04-20T10:59:25.830 INFO:tasks.workunit.client.0.smithi027.stdout:2/769: fsync d5/d26/d30/f79 0 2024-04-20T10:59:25.833 INFO:tasks.workunit.client.0.smithi027.stdout:5/780: dwrite d19/d1b/d1c/d1e/f56 [0,4194304] 0 2024-04-20T10:59:25.839 INFO:tasks.workunit.client.0.smithi027.stdout:7/830: dwrite d4/d1b/f53 [0,4194304] 0 2024-04-20T10:59:26.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/996: rmdir d0/d11/d2c/d2e/d2b/d7b/d64 39 2024-04-20T10:59:26.074 INFO:tasks.workunit.client.0.smithi027.stdout:6/883: symlink da/dcc/l123 0 2024-04-20T10:59:26.074 INFO:tasks.workunit.client.0.smithi027.stdout:8/892: symlink d9/de/d1e/d49/d120/l12f 0 2024-04-20T10:59:26.079 INFO:tasks.workunit.client.0.smithi027.stdout:5/781: creat d19/d1b/d1c/d1e/d22/dca/ffc x:0 0 0 2024-04-20T10:59:26.081 INFO:tasks.workunit.client.0.smithi027.stdout:7/831: rename d4/d5/d81/c8d to d4/d5/d6f/c10d 0 2024-04-20T10:59:26.085 INFO:tasks.workunit.client.0.smithi027.stdout:5/782: dread d19/d1b/d1c/dbd/dbe/fe6 [0,4194304] 0 2024-04-20T10:59:26.086 INFO:tasks.workunit.client.0.smithi027.stdout:0/997: link d0/d11/d2c/d2e/d61/f14f d0/d59/dbf/f151 0 2024-04-20T10:59:26.087 INFO:tasks.workunit.client.0.smithi027.stdout:0/998: chown d0/d11/f148 12055381 1 2024-04-20T10:59:26.088 INFO:tasks.workunit.client.0.smithi027.stdout:6/884: creat da/d21/dab/d19/f124 x:0 0 0 2024-04-20T10:59:26.088 INFO:tasks.workunit.client.0.smithi027.stdout:6/885: write da/d21/dab/f11 [353134,93644] 0 2024-04-20T10:59:26.088 INFO:tasks.workunit.client.0.smithi027.stdout:6/886: fdatasync da/d21/dab/d19/f74 0 2024-04-20T10:59:26.089 INFO:tasks.workunit.client.0.smithi027.stdout:8/893: mknod d9/de/d1e/d4a/d6d/dea/d59/d74/de8/c130 0 2024-04-20T10:59:26.090 INFO:tasks.workunit.client.0.smithi027.stdout:7/832: getdents d4/d5/d8/d46 0 2024-04-20T10:59:26.092 INFO:tasks.workunit.client.0.smithi027.stdout:5/783: symlink d19/d1b/d1c/d1e/d25/de5/lfd 0 2024-04-20T10:59:26.093 INFO:tasks.workunit.client.0.smithi027.stdout:0/999: getdents d0/d11/d5c/df0/d133/d139 0 2024-04-20T10:59:26.094 INFO:tasks.workunit.client.0.smithi027.stdout:6/887: getdents da/dd/d3c 0 2024-04-20T10:59:26.097 INFO:tasks.workunit.client.0.smithi027.stdout:8/894: write d9/de/d64/d6a/f67 [1009104,108155] 0 2024-04-20T10:59:26.097 INFO:tasks.workunit.client.0.smithi027.stdout:6/888: symlink da/d21/dab/d1f/dbf/l125 0 2024-04-20T10:59:26.099 INFO:tasks.workunit.client.0.smithi027.stdout:8/895: mkdir d9/de/d1e/d4a/d6d/dea/d62/d12e/d131 0 2024-04-20T10:59:26.101 INFO:tasks.workunit.client.0.smithi027.stdout:8/896: link d9/de/d1e/d4a/d6d/d7d/c11f d9/de/d20/d53/c132 0 2024-04-20T10:59:26.102 INFO:tasks.workunit.client.0.smithi027.stdout:8/897: symlink d9/d11c/l133 0 2024-04-20T10:59:26.102 INFO:tasks.workunit.client.0.smithi027.stdout:8/898: read - d9/de/d1e/d11b/fef zero size 2024-04-20T10:59:26.102 INFO:tasks.workunit.client.0.smithi027.stdout:8/899: creat d9/de/d1e/d4a/d6d/db7/d97/f134 x:0 0 0 2024-04-20T10:59:26.103 INFO:tasks.workunit.client.0.smithi027.stdout:8/900: getdents d9/de/d20/dfd/d122/d126 0 2024-04-20T10:59:26.104 INFO:tasks.workunit.client.0.smithi027.stdout:8/901: rmdir d9/de/d1e/d4a/d6d/dea/d59/d74/de8 39 2024-04-20T10:59:26.105 INFO:tasks.workunit.client.0.smithi027.stdout:8/902: rename d9/de/d1e/d4a/ff4 to d9/de/d18/dff/f135 0 2024-04-20T10:59:26.106 INFO:tasks.workunit.client.0.smithi027.stdout:8/903: mkdir d9/de/d1e/d11b/dbd/d136 0 2024-04-20T10:59:26.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/833: dwrite d4/d5/d8/d21/d94/fbd [0,4194304] 0 2024-04-20T10:59:26.109 INFO:tasks.workunit.client.0.smithi027.stdout:7/834: symlink d4/d5/d8/d21/d94/da7/l10e 0 2024-04-20T10:59:26.110 INFO:tasks.workunit.client.0.smithi027.stdout:7/835: link d4/d5/f6c d4/d5/d8/d46/d106/f10f 0 2024-04-20T10:59:26.117 INFO:tasks.workunit.client.0.smithi027.stdout:5/784: dwrite d19/d1b/d1c/d1e/d58/f7b [0,4194304] 0 2024-04-20T10:59:26.117 INFO:tasks.workunit.client.0.smithi027.stdout:6/889: dwrite da/d21/f66 [0,4194304] 0 2024-04-20T10:59:26.117 INFO:tasks.workunit.client.0.smithi027.stdout:6/890: chown da/dcc/d108/d6b/fe4 115197 1 2024-04-20T10:59:26.117 INFO:tasks.workunit.client.0.smithi027.stdout:8/904: dread d9/de/d1e/f65 [0,4194304] 0 2024-04-20T10:59:26.117 INFO:tasks.workunit.client.0.smithi027.stdout:8/905: dread - d9/de/d18/dff/f135 zero size 2024-04-20T10:59:26.120 INFO:tasks.workunit.client.0.smithi027.stdout:9/746: sync 2024-04-20T10:59:26.120 INFO:tasks.workunit.client.0.smithi027.stdout:4/803: sync 2024-04-20T10:59:26.120 INFO:tasks.workunit.client.0.smithi027.stdout:3/881: sync 2024-04-20T10:59:26.120 INFO:tasks.workunit.client.0.smithi027.stdout:2/770: sync 2024-04-20T10:59:26.120 INFO:tasks.workunit.client.0.smithi027.stdout:9/747: write d4/d2a/f69 [1809199,38840] 0 2024-04-20T10:59:26.121 INFO:tasks.workunit.client.0.smithi027.stdout:9/748: fsync d4/d74/d72/f73 0 2024-04-20T10:59:26.121 INFO:tasks.workunit.client.0.smithi027.stdout:5/785: rmdir d19/d1b/d1c/dbd/dbe 39 2024-04-20T10:59:26.122 INFO:tasks.workunit.client.0.smithi027.stdout:8/906: dread d9/de/da3/d3c/fd3 [0,4194304] 0 2024-04-20T10:59:26.122 INFO:tasks.workunit.client.0.smithi027.stdout:8/907: write d9/de/d1e/d4a/d6d/db7/d5b/ff7 [1311622,20678] 0 2024-04-20T10:59:26.122 INFO:tasks.workunit.client.0.smithi027.stdout:6/891: creat da/dd/d3c/f126 x:0 0 0 2024-04-20T10:59:26.125 INFO:tasks.workunit.client.0.smithi027.stdout:3/882: mknod d2/dd/d57/d8c/c111 0 2024-04-20T10:59:26.125 INFO:tasks.workunit.client.0.smithi027.stdout:3/883: chown d2/dd/df/d23/fe6 59048 1 2024-04-20T10:59:26.125 INFO:tasks.workunit.client.0.smithi027.stdout:3/884: write d2/f1f [707807,47406] 0 2024-04-20T10:59:26.125 INFO:tasks.workunit.client.0.smithi027.stdout:6/892: dread da/d21/dab/f11 [0,4194304] 0 2024-04-20T10:59:26.125 INFO:tasks.workunit.client.0.smithi027.stdout:6/893: write da/d21/dab/d7e/f85 [1191336,42577] 0 2024-04-20T10:59:26.125 INFO:tasks.workunit.client.0.smithi027.stdout:2/771: creat d5/dc/fff x:0 0 0 2024-04-20T10:59:26.126 INFO:tasks.workunit.client.0.smithi027.stdout:4/804: symlink d0/d14/d6b/d92/d11e/l122 0 2024-04-20T10:59:26.126 INFO:tasks.workunit.client.0.smithi027.stdout:4/805: write d0/f108 [449284,34540] 0 2024-04-20T10:59:26.127 INFO:tasks.workunit.client.0.smithi027.stdout:5/786: creat d19/d1b/d1c/d1e/d58/db4/ffe x:0 0 0 2024-04-20T10:59:26.129 INFO:tasks.workunit.client.0.smithi027.stdout:8/908: symlink d9/de/d20/dfd/d122/l137 0 2024-04-20T10:59:26.131 INFO:tasks.workunit.client.0.smithi027.stdout:8/909: read d9/de/d64/d6c/f93 [480152,1679] 0 2024-04-20T10:59:26.131 INFO:tasks.workunit.client.0.smithi027.stdout:8/910: write d9/de/d20/f12c [585453,31310] 0 2024-04-20T10:59:26.136 INFO:tasks.workunit.client.0.smithi027.stdout:2/772: read d5/d26/d47/fdc [4518919,130783] 0 2024-04-20T10:59:26.136 INFO:tasks.workunit.client.0.smithi027.stdout:3/885: rmdir d2/d18/d2c/d4c/d67/d6c/dd0 39 2024-04-20T10:59:26.136 INFO:tasks.workunit.client.0.smithi027.stdout:6/894: getdents da/d21/dab/d7e/d86/dcd 0 2024-04-20T10:59:26.137 INFO:tasks.workunit.client.0.smithi027.stdout:4/806: rename d0/d14/d6b/da3/d1b to d0/d48/d123 0 2024-04-20T10:59:26.137 INFO:tasks.workunit.client.0.smithi027.stdout:4/807: write d0/fd [3702009,106261] 0 2024-04-20T10:59:26.138 INFO:tasks.workunit.client.0.smithi027.stdout:2/773: creat d5/d26/d30/f100 x:0 0 0 2024-04-20T10:59:26.140 INFO:tasks.workunit.client.0.smithi027.stdout:3/886: creat d2/d18/d2c/d4c/f112 x:0 0 0 2024-04-20T10:59:26.140 INFO:tasks.workunit.client.0.smithi027.stdout:3/887: chown d2/l9f 399394091 1 2024-04-20T10:59:26.140 INFO:tasks.workunit.client.0.smithi027.stdout:3/888: dread - d2/d18/d2c/d4c/f112 zero size 2024-04-20T10:59:26.140 INFO:tasks.workunit.client.0.smithi027.stdout:3/889: stat d2/d18/l102 0 2024-04-20T10:59:26.141 INFO:tasks.workunit.client.0.smithi027.stdout:6/895: rename da/dcc/d108/d6b/d65 to da/dd/d3d/db9/d127 0 2024-04-20T10:59:26.142 INFO:tasks.workunit.client.0.smithi027.stdout:4/808: symlink d0/d48/d68/d76/l124 0 2024-04-20T10:59:26.144 INFO:tasks.workunit.client.0.smithi027.stdout:3/890: creat d2/dd/d57/de7/f113 x:0 0 0 2024-04-20T10:59:26.145 INFO:tasks.workunit.client.0.smithi027.stdout:3/891: write d2/dd/d22/fa2 [2850749,89347] 0 2024-04-20T10:59:26.149 INFO:tasks.workunit.client.0.smithi027.stdout:4/809: rename d0/d14/d6b/da3/d1a/f112 to d0/d14/d6b/f125 0 2024-04-20T10:59:26.153 INFO:tasks.workunit.client.0.smithi027.stdout:5/787: dwrite d19/d1b/d1c/dbd/dbe/fbf [0,4194304] 0 2024-04-20T10:59:26.153 INFO:tasks.workunit.client.0.smithi027.stdout:8/911: dwrite d9/de/d1e/d4a/d6d/db7/d97/fdf [0,4194304] 0 2024-04-20T10:59:26.154 INFO:tasks.workunit.client.0.smithi027.stdout:5/788: mkdir d19/d1b/d1c/d1e/d58/db4/ded/dff 0 2024-04-20T10:59:26.154 INFO:tasks.workunit.client.0.smithi027.stdout:5/789: chown d19/d1b/d1c/d1e/d22/d68/l80 43381355 1 2024-04-20T10:59:26.157 INFO:tasks.workunit.client.0.smithi027.stdout:8/912: unlink d9/de/fd5 0 2024-04-20T10:59:26.157 INFO:tasks.workunit.client.0.smithi027.stdout:8/913: write d9/f15 [939953,93383] 0 2024-04-20T10:59:26.161 INFO:tasks.workunit.client.0.smithi027.stdout:8/914: creat d9/de/d1e/d4a/d6d/db7/f138 x:0 0 0 2024-04-20T10:59:26.164 INFO:tasks.workunit.client.0.smithi027.stdout:5/790: rename d19/c1a to d19/d1b/d1c/d1e/d58/db4/ded/dff/c100 0 2024-04-20T10:59:26.166 INFO:tasks.workunit.client.0.smithi027.stdout:8/915: rename d9/de/d1e/c38 to d9/de/d64/dfe/c139 0 2024-04-20T10:59:26.166 INFO:tasks.workunit.client.0.smithi027.stdout:2/774: dwrite d5/dc/d16/d2a/d59/f8d [0,4194304] 0 2024-04-20T10:59:26.166 INFO:tasks.workunit.client.0.smithi027.stdout:2/775: fdatasync d5/dc/d16/dd0/dd5/ff2 0 2024-04-20T10:59:26.167 INFO:tasks.workunit.client.0.smithi027.stdout:5/791: mknod d19/d1b/d1c/d1e/d58/c101 0 2024-04-20T10:59:26.168 INFO:tasks.workunit.client.0.smithi027.stdout:2/776: creat d5/dc/d16/d37/de2/f101 x:0 0 0 2024-04-20T10:59:26.172 INFO:tasks.workunit.client.0.smithi027.stdout:3/892: dwrite d2/dd/fc0 [0,4194304] 0 2024-04-20T10:59:26.172 INFO:tasks.workunit.client.0.smithi027.stdout:3/893: stat d2/dd/df/d23/d68 0 2024-04-20T10:59:26.173 INFO:tasks.workunit.client.0.smithi027.stdout:5/792: creat d19/d1b/d1c/d1e/d22/d68/d75/db7/f102 x:0 0 0 2024-04-20T10:59:26.173 INFO:tasks.workunit.client.0.smithi027.stdout:5/793: stat d19/d1b/d1c/dbd/cd7 0 2024-04-20T10:59:26.174 INFO:tasks.workunit.client.0.smithi027.stdout:4/810: dwrite d0/d14/d6b/da3/d18/ffb [0,4194304] 0 2024-04-20T10:59:26.174 INFO:tasks.workunit.client.0.smithi027.stdout:4/811: chown d0/f3a 227946 1 2024-04-20T10:59:26.175 INFO:tasks.workunit.client.0.smithi027.stdout:4/812: write d0/d5e/d6c/fb2 [331423,8120] 0 2024-04-20T10:59:26.175 INFO:tasks.workunit.client.0.smithi027.stdout:4/813: truncate d0/d14/d6b/da3/d1a/f8a 1052351 0 2024-04-20T10:59:26.175 INFO:tasks.workunit.client.0.smithi027.stdout:4/814: truncate d0/d14/d6b/da3/d18/f20 2214274 0 2024-04-20T10:59:26.175 INFO:tasks.workunit.client.0.smithi027.stdout:3/894: mknod d2/dd/df/d23/d68/c114 0 2024-04-20T10:59:26.176 INFO:tasks.workunit.client.0.smithi027.stdout:4/815: rmdir d0/d14/d6b/d92/d11e 39 2024-04-20T10:59:26.176 INFO:tasks.workunit.client.0.smithi027.stdout:4/816: readlink d0/d48/d123/d33/laf 0 2024-04-20T10:59:26.176 INFO:tasks.workunit.client.0.smithi027.stdout:2/777: read d5/f4b [1475545,83097] 0 2024-04-20T10:59:26.183 INFO:tasks.workunit.client.0.smithi027.stdout:8/916: dwrite d9/de/d64/d6c/f11a [4194304,4194304] 0 2024-04-20T10:59:26.183 INFO:tasks.workunit.client.0.smithi027.stdout:8/917: truncate d9/de/d64/d6a/f67 4588801 0 2024-04-20T10:59:26.183 INFO:tasks.workunit.client.0.smithi027.stdout:8/918: write d9/de/d1e/d4a/d6d/d75/f83 [5421494,8896] 0 2024-04-20T10:59:26.183 INFO:tasks.workunit.client.0.smithi027.stdout:8/919: stat d9/de/d1e/d4a/d6d/d79/f92 0 2024-04-20T10:59:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:3/895: symlink d2/dd/df/d33/l115 0 2024-04-20T10:59:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:3/896: getdents d2/dd/d57/d8d/d9d 0 2024-04-20T10:59:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:3/897: readlink d2/dd/d57/d8d/dc6/de1/lf8 0 2024-04-20T10:59:26.185 INFO:tasks.workunit.client.0.smithi027.stdout:8/920: creat d9/de/d1e/d4a/d6d/dea/d59/d74/de8/d118/f13a x:0 0 0 2024-04-20T10:59:26.186 INFO:tasks.workunit.client.0.smithi027.stdout:3/898: mknod d2/d18/d2c/d4c/d67/d6c/dd0/c116 0 2024-04-20T10:59:26.197 INFO:tasks.workunit.client.0.smithi027.stdout:3/899: read d2/dd/d22/f2f [7566044,38342] 0 2024-04-20T10:59:26.197 INFO:tasks.workunit.client.0.smithi027.stdout:2/778: dwrite d5/d1e/d21/f3e [0,4194304] 0 2024-04-20T10:59:26.198 INFO:tasks.workunit.client.0.smithi027.stdout:8/921: creat d9/de/d1e/d4a/d6d/d78/f13b x:0 0 0 2024-04-20T10:59:26.199 INFO:tasks.workunit.client.0.smithi027.stdout:3/900: symlink d2/dd/d57/d8d/dc6/l117 0 2024-04-20T10:59:26.199 INFO:tasks.workunit.client.0.smithi027.stdout:4/817: dread d0/d48/d123/f9c [0,4194304] 0 2024-04-20T10:59:26.199 INFO:tasks.workunit.client.0.smithi027.stdout:4/818: truncate d0/d48/d123/d33/d5c/d6f/f106 563855 0 2024-04-20T10:59:26.200 INFO:tasks.workunit.client.0.smithi027.stdout:4/819: chown d0/d14/d6b/da3/d18/d10a/l99 213 1 2024-04-20T10:59:26.200 INFO:tasks.workunit.client.0.smithi027.stdout:4/820: fdatasync d0/d14/d6b/da3/d18/f72 0 2024-04-20T10:59:26.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/922: getdents d9/de/d20/dfd/d122 0 2024-04-20T10:59:26.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/923: dread - d9/de/d20/dfd/d122/d126/fc9 zero size 2024-04-20T10:59:26.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/924: write d9/de/d18/dff/f135 [663326,122159] 0 2024-04-20T10:59:26.205 INFO:tasks.workunit.client.0.smithi027.stdout:3/901: getdents d2/d18/d2c/d4c/d67/d6c 0 2024-04-20T10:59:26.208 INFO:tasks.workunit.client.0.smithi027.stdout:4/821: dread d0/f3a [0,4194304] 0 2024-04-20T10:59:26.209 INFO:tasks.workunit.client.0.smithi027.stdout:3/902: symlink d2/dd/df/d33/d105/l118 0 2024-04-20T10:59:26.210 INFO:tasks.workunit.client.0.smithi027.stdout:4/822: creat d0/d48/d123/dc3/dc5/f126 x:0 0 0 2024-04-20T10:59:26.210 INFO:tasks.workunit.client.0.smithi027.stdout:4/823: readlink d0/d48/d123/d33/d5c/d6f/d7b/l8d 0 2024-04-20T10:59:26.211 INFO:tasks.workunit.client.0.smithi027.stdout:3/903: symlink d2/dd/d22/de2/l119 0 2024-04-20T10:59:26.211 INFO:tasks.workunit.client.0.smithi027.stdout:4/824: getdents d0/d14/d6b/da3/d53/d6a 0 2024-04-20T10:59:26.212 INFO:tasks.workunit.client.0.smithi027.stdout:3/904: symlink d2/d18/d2c/d4c/l11a 0 2024-04-20T10:59:26.212 INFO:tasks.workunit.client.0.smithi027.stdout:3/905: chown d2/dd/df/l2b 5719 1 2024-04-20T10:59:26.213 INFO:tasks.workunit.client.0.smithi027.stdout:4/825: unlink d0/d5e/d6c/dd8/f121 0 2024-04-20T10:59:26.214 INFO:tasks.workunit.client.0.smithi027.stdout:4/826: stat d0/d14/d6b/d92/d11e/l122 0 2024-04-20T10:59:26.216 INFO:tasks.workunit.client.0.smithi027.stdout:2/779: dread d5/d26/d47/d8a/f9a [0,4194304] 0 2024-04-20T10:59:26.216 INFO:tasks.workunit.client.0.smithi027.stdout:2/780: chown d5 1303814 1 2024-04-20T10:59:26.217 INFO:tasks.workunit.client.0.smithi027.stdout:3/906: mkdir d2/dd/d22/de2/d11b 0 2024-04-20T10:59:26.218 INFO:tasks.workunit.client.0.smithi027.stdout:2/781: creat d5/dbc/f102 x:0 0 0 2024-04-20T10:59:26.220 INFO:tasks.workunit.client.0.smithi027.stdout:2/782: write d5/dc/d16/db1/fef [846374,49474] 0 2024-04-20T10:59:26.221 INFO:tasks.workunit.client.0.smithi027.stdout:3/907: rename d2/dd/df/d33/fca to d2/dd/d22/de2/d11b/f11c 0 2024-04-20T10:59:26.222 INFO:tasks.workunit.client.0.smithi027.stdout:2/783: link d5/dc/lec d5/dc/d16/d37/de2/dac/l103 0 2024-04-20T10:59:26.240 INFO:tasks.workunit.client.0.smithi027.stdout:2/784: dwrite d5/dc/d16/d37/de2/dac/f50 [0,4194304] 0 2024-04-20T10:59:26.241 INFO:tasks.workunit.client.0.smithi027.stdout:3/908: creat d2/dd/d57/d8d/d9d/f11d x:0 0 0 2024-04-20T10:59:26.248 INFO:tasks.workunit.client.0.smithi027.stdout:4/827: read d0/d14/d6b/da3/d53/d6a/da1/db0/fbb [1913165,65116] 0 2024-04-20T10:59:26.248 INFO:tasks.workunit.client.0.smithi027.stdout:4/828: dread - d0/d14/d71/fe4 zero size 2024-04-20T10:59:26.258 INFO:tasks.workunit.client.0.smithi027.stdout:2/785: dwrite d5/f17 [0,4194304] 0 2024-04-20T10:59:26.258 INFO:tasks.workunit.client.0.smithi027.stdout:2/786: truncate d5/d26/fc0 522077 0 2024-04-20T10:59:26.258 INFO:tasks.workunit.client.0.smithi027.stdout:4/829: rmdir d0/d48/d68 39 2024-04-20T10:59:26.259 INFO:tasks.workunit.client.0.smithi027.stdout:2/787: dread d5/d26/d47/ffa [0,4194304] 0 2024-04-20T10:59:26.259 INFO:tasks.workunit.client.0.smithi027.stdout:2/788: fdatasync d5/dc/d16/d37/f3f 0 2024-04-20T10:59:26.264 INFO:tasks.workunit.client.0.smithi027.stdout:7/836: sync 2024-04-20T10:59:26.275 INFO:tasks.workunit.client.0.smithi027.stdout:4/830: dwrite d0/d48/d123/f7f [0,4194304] 0 2024-04-20T10:59:26.277 INFO:tasks.workunit.client.0.smithi027.stdout:3/909: read d2/d18/d2c/d4c/d67/d6c/dd0/f10d [1481342,29059] 0 2024-04-20T10:59:26.277 INFO:tasks.workunit.client.0.smithi027.stdout:3/910: dread - d2/dd/df/d23/fdb zero size 2024-04-20T10:59:26.277 INFO:tasks.workunit.client.0.smithi027.stdout:3/911: fsync d2/f16 0 2024-04-20T10:59:26.286 INFO:tasks.workunit.client.0.smithi027.stdout:7/837: dwrite d4/d5b/f67 [4194304,4194304] 0 2024-04-20T10:59:26.305 INFO:tasks.workunit.client.0.smithi027.stdout:2/789: dwrite d5/f58 [4194304,4194304] 0 2024-04-20T10:59:26.305 INFO:tasks.workunit.client.0.smithi027.stdout:2/790: chown d5/dc/d16/d37/lf3 7793 1 2024-04-20T10:59:26.305 INFO:tasks.workunit.client.0.smithi027.stdout:2/791: chown d5/dc/f1c 771001 1 2024-04-20T10:59:26.306 INFO:tasks.workunit.client.0.smithi027.stdout:7/838: mkdir d4/d5/d6f/d110 0 2024-04-20T10:59:26.321 INFO:tasks.workunit.client.0.smithi027.stdout:2/792: creat d5/dc/d16/d32/dab/dba/f104 x:0 0 0 2024-04-20T10:59:26.323 INFO:tasks.workunit.client.0.smithi027.stdout:7/839: dwrite d4/d5/d8/d21/d94/da7/fc0 [0,4194304] 0 2024-04-20T10:59:26.324 INFO:tasks.workunit.client.0.smithi027.stdout:7/840: chown d4/d5/d8/d45 0 1 2024-04-20T10:59:26.324 INFO:tasks.workunit.client.0.smithi027.stdout:2/793: symlink d5/dc/d16/d37/de2/dac/l105 0 2024-04-20T10:59:26.325 INFO:tasks.workunit.client.0.smithi027.stdout:2/794: read - d5/d44/ffe zero size 2024-04-20T10:59:26.325 INFO:tasks.workunit.client.0.smithi027.stdout:9/749: sync 2024-04-20T10:59:26.325 INFO:tasks.workunit.client.0.smithi027.stdout:9/750: read - d4/d1b/d1f/d5a/db7/fc5 zero size 2024-04-20T10:59:26.325 INFO:tasks.workunit.client.0.smithi027.stdout:9/751: readlink d4/d1b/d8f/la6 0 2024-04-20T10:59:26.325 INFO:tasks.workunit.client.0.smithi027.stdout:7/841: mkdir d4/d1b/d4a/d111 0 2024-04-20T10:59:26.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/795: rename d5/d44/d7c/fc5 to d5/d44/f106 0 2024-04-20T10:59:26.327 INFO:tasks.workunit.client.0.smithi027.stdout:7/842: dread d4/d4d/f60 [0,4194304] 0 2024-04-20T10:59:26.328 INFO:tasks.workunit.client.0.smithi027.stdout:9/752: creat d4/d1b/d50/fe0 x:0 0 0 2024-04-20T10:59:26.328 INFO:tasks.workunit.client.0.smithi027.stdout:2/796: creat d5/dc/d16/d37/de2/dac/d8f/dfd/f107 x:0 0 0 2024-04-20T10:59:26.329 INFO:tasks.workunit.client.0.smithi027.stdout:7/843: rmdir d4/d1b/d4a/d62 39 2024-04-20T10:59:26.330 INFO:tasks.workunit.client.0.smithi027.stdout:2/797: dread d5/dc/d16/d2a/d59/da4/de1/dcc/fcd [0,4194304] 0 2024-04-20T10:59:26.330 INFO:tasks.workunit.client.0.smithi027.stdout:2/798: stat d5/dc/d16/d37/de2/dac/d8f/dfd/cf7 0 2024-04-20T10:59:26.330 INFO:tasks.workunit.client.0.smithi027.stdout:7/844: write d4/d5/d8/d46/d77/d7c/f8c [121711,68161] 0 2024-04-20T10:59:26.330 INFO:tasks.workunit.client.0.smithi027.stdout:7/845: chown d4/d5/d8/d21/d94/da7/dce/cf5 10623 1 2024-04-20T10:59:26.330 INFO:tasks.workunit.client.0.smithi027.stdout:9/753: mkdir d4/d1b/d8f/de1 0 2024-04-20T10:59:26.331 INFO:tasks.workunit.client.0.smithi027.stdout:2/799: symlink d5/d44/d7c/l108 0 2024-04-20T10:59:26.332 INFO:tasks.workunit.client.0.smithi027.stdout:7/846: creat d4/d5/d8/d46/d106/f112 x:0 0 0 2024-04-20T10:59:26.333 INFO:tasks.workunit.client.0.smithi027.stdout:2/800: getdents d5 0 2024-04-20T10:59:26.349 INFO:tasks.workunit.client.0.smithi027.stdout:2/801: dwrite d5/d1e/f9c [4194304,4194304] 0 2024-04-20T10:59:26.353 INFO:tasks.workunit.client.0.smithi027.stdout:5/794: sync 2024-04-20T10:59:26.353 INFO:tasks.workunit.client.0.smithi027.stdout:8/925: sync 2024-04-20T10:59:26.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/896: sync 2024-04-20T10:59:26.353 INFO:tasks.workunit.client.0.smithi027.stdout:5/795: chown d19/d1b/d1c/d1e/d22/db1/dcf/fea 0 1 2024-04-20T10:59:26.353 INFO:tasks.workunit.client.0.smithi027.stdout:8/926: write d9/de/d1e/d4a/d6d/dea/f113 [1022164,40244] 0 2024-04-20T10:59:26.357 INFO:tasks.workunit.client.0.smithi027.stdout:5/796: dread d19/d1b/d1c/d1e/d3c/fcd [0,4194304] 0 2024-04-20T10:59:26.366 INFO:tasks.workunit.client.0.smithi027.stdout:5/797: read d19/d1b/d1c/f7e [2426692,93302] 0 2024-04-20T10:59:26.366 INFO:tasks.workunit.client.0.smithi027.stdout:5/798: chown f18 422585 1 2024-04-20T10:59:26.367 INFO:tasks.workunit.client.0.smithi027.stdout:8/927: symlink d9/de/d1e/l13c 0 2024-04-20T10:59:26.368 INFO:tasks.workunit.client.0.smithi027.stdout:5/799: creat d19/d1b/d1c/d1e/d58/db4/f103 x:0 0 0 2024-04-20T10:59:26.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/928: creat d9/de/d1e/d11b/dbd/f13d x:0 0 0 2024-04-20T10:59:26.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/929: chown d9/de/d1e/d4a/c36 3742677 1 2024-04-20T10:59:26.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/930: write d9/de/d1e/d4a/d6d/db7/f138 [534698,38310] 0 2024-04-20T10:59:26.370 INFO:tasks.workunit.client.0.smithi027.stdout:2/802: dwrite d5/dc/d16/d2a/d59/f95 [0,4194304] 0 2024-04-20T10:59:26.374 INFO:tasks.workunit.client.0.smithi027.stdout:6/897: dwrite da/dd/d3d/db9/f119 [0,4194304] 0 2024-04-20T10:59:26.377 INFO:tasks.workunit.client.0.smithi027.stdout:8/931: mknod d9/de/d1e/dc1/dd1/c13e 0 2024-04-20T10:59:26.377 INFO:tasks.workunit.client.0.smithi027.stdout:8/932: stat d9/de/d1e/d4a/d6d/dea/d59/d74 0 2024-04-20T10:59:26.378 INFO:tasks.workunit.client.0.smithi027.stdout:8/933: fsync d9/de/d1e/d4a/d6d/d75/f12b 0 2024-04-20T10:59:26.378 INFO:tasks.workunit.client.0.smithi027.stdout:8/934: write d9/de/d18/f22 [19075,130802] 0 2024-04-20T10:59:26.378 INFO:tasks.workunit.client.0.smithi027.stdout:8/935: write d9/de/da3/f127 [890933,26969] 0 2024-04-20T10:59:26.380 INFO:tasks.workunit.client.0.smithi027.stdout:6/898: unlink da/dd/l67 0 2024-04-20T10:59:26.382 INFO:tasks.workunit.client.0.smithi027.stdout:8/936: creat d9/de/d1e/d4a/d6d/d103/f13f x:0 0 0 2024-04-20T10:59:26.387 INFO:tasks.workunit.client.0.smithi027.stdout:5/800: dwrite d19/d1b/d1c/d1e/d22/fd6 [0,4194304] 0 2024-04-20T10:59:26.387 INFO:tasks.workunit.client.0.smithi027.stdout:3/912: sync 2024-04-20T10:59:26.387 INFO:tasks.workunit.client.0.smithi027.stdout:3/913: write d2/dd/f58 [308080,36820] 0 2024-04-20T10:59:26.389 INFO:tasks.workunit.client.0.smithi027.stdout:6/899: creat da/d21/f128 x:0 0 0 2024-04-20T10:59:26.391 INFO:tasks.workunit.client.0.smithi027.stdout:3/914: mknod d2/dd/d22/de2/c11e 0 2024-04-20T10:59:26.392 INFO:tasks.workunit.client.0.smithi027.stdout:2/803: dwrite d5/dc/d16/d32/dab/dba/f104 [0,4194304] 0 2024-04-20T10:59:26.392 INFO:tasks.workunit.client.0.smithi027.stdout:6/900: symlink da/dcc/d108/d6b/l129 0 2024-04-20T10:59:26.393 INFO:tasks.workunit.client.0.smithi027.stdout:3/915: truncate d2/d18/d2c/f69 3968064 0 2024-04-20T10:59:26.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/804: rename d5/dc/d16/d32/dab/dba to d5/dc/d16/d32/d109 0 2024-04-20T10:59:26.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/805: chown d5/d1e/d21/f33 113 1 2024-04-20T10:59:26.399 INFO:tasks.workunit.client.0.smithi027.stdout:6/901: truncate da/dcc/d108/d6b/f62 2754113 0 2024-04-20T10:59:26.399 INFO:tasks.workunit.client.0.smithi027.stdout:3/916: mkdir d2/dd/d106/d11f 0 2024-04-20T10:59:26.400 INFO:tasks.workunit.client.0.smithi027.stdout:6/902: truncate da/dcc/d108/d6b/fe4 1348748 0 2024-04-20T10:59:26.403 INFO:tasks.workunit.client.0.smithi027.stdout:5/801: dwrite d19/d1b/d1c/d1e/d22/fe8 [0,4194304] 0 2024-04-20T10:59:26.403 INFO:tasks.workunit.client.0.smithi027.stdout:5/802: write d19/d1b/d1c/f7e [2335612,7617] 0 2024-04-20T10:59:26.403 INFO:tasks.workunit.client.0.smithi027.stdout:5/803: fsync d19/d1b/d1c/d1e/d58/d9d/fe0 0 2024-04-20T10:59:26.404 INFO:tasks.workunit.client.0.smithi027.stdout:3/917: unlink d2/dd/d22/de2/l119 0 2024-04-20T10:59:26.404 INFO:tasks.workunit.client.0.smithi027.stdout:3/918: chown d2/dd/d106/d11f 63446 1 2024-04-20T10:59:26.407 INFO:tasks.workunit.client.0.smithi027.stdout:3/919: dread d2/f1f [0,4194304] 0 2024-04-20T10:59:26.407 INFO:tasks.workunit.client.0.smithi027.stdout:5/804: rmdir d19/d1b/d1c/d1e/da3/df3 39 2024-04-20T10:59:26.408 INFO:tasks.workunit.client.0.smithi027.stdout:4/831: write d0/d14/d6b/da3/f31 [2581888,92549] 0 2024-04-20T10:59:26.410 INFO:tasks.workunit.client.0.smithi027.stdout:3/920: creat d2/dd/df/d23/df0/f120 x:0 0 0 2024-04-20T10:59:26.411 INFO:tasks.workunit.client.0.smithi027.stdout:4/832: dread d0/d14/d6b/da3/d1a/f8a [0,4194304] 0 2024-04-20T10:59:26.415 INFO:tasks.workunit.client.0.smithi027.stdout:5/805: dread d19/d1b/d1c/d1e/d22/db1/f55 [0,4194304] 0 2024-04-20T10:59:26.416 INFO:tasks.workunit.client.0.smithi027.stdout:3/921: creat d2/d18/d2c/d4c/d67/d6c/dd0/dd5/f121 x:0 0 0 2024-04-20T10:59:26.418 INFO:tasks.workunit.client.0.smithi027.stdout:2/806: dwrite d5/f17 [0,4194304] 0 2024-04-20T10:59:26.418 INFO:tasks.workunit.client.0.smithi027.stdout:2/807: readlink d5/d26/d78/l62 0 2024-04-20T10:59:26.418 INFO:tasks.workunit.client.0.smithi027.stdout:2/808: write d5/dc/fff [297430,7377] 0 2024-04-20T10:59:26.419 INFO:tasks.workunit.client.0.smithi027.stdout:5/806: symlink d19/d1b/d1c/d1e/d22/d68/d75/db7/l104 0 2024-04-20T10:59:26.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/809: unlink d5/f4b 0 2024-04-20T10:59:26.423 INFO:tasks.workunit.client.0.smithi027.stdout:5/807: getdents d19/d1b/d1c/d1e/d58 0 2024-04-20T10:59:26.424 INFO:tasks.workunit.client.0.smithi027.stdout:2/810: mknod d5/dc/d16/d32/c10a 0 2024-04-20T10:59:26.443 INFO:tasks.workunit.client.0.smithi027.stdout:2/811: dwrite d5/dc/f41 [0,4194304] 0 2024-04-20T10:59:26.443 INFO:tasks.workunit.client.0.smithi027.stdout:8/937: dwrite d9/de/d1e/d4a/d6d/dea/d59/d74/de8/d118/f13a [0,4194304] 0 2024-04-20T10:59:26.443 INFO:tasks.workunit.client.0.smithi027.stdout:4/833: dwrite d0/d14/d6b/da3/d18/f4b [0,4194304] 0 2024-04-20T10:59:26.444 INFO:tasks.workunit.client.0.smithi027.stdout:2/812: truncate d5/d26/d30/f3c 1454106 0 2024-04-20T10:59:26.445 INFO:tasks.workunit.client.0.smithi027.stdout:8/938: symlink d9/de/d20/l140 0 2024-04-20T10:59:26.446 INFO:tasks.workunit.client.0.smithi027.stdout:5/808: symlink d19/d1b/d1c/d1e/d3c/l105 0 2024-04-20T10:59:26.447 INFO:tasks.workunit.client.0.smithi027.stdout:2/813: mknod d5/dc/d16/d32/dab/c10b 0 2024-04-20T10:59:26.448 INFO:tasks.workunit.client.0.smithi027.stdout:8/939: rename d9/de/d1e/d4a/d6d/db7/d5b/f5d to d9/f141 0 2024-04-20T10:59:26.449 INFO:tasks.workunit.client.0.smithi027.stdout:5/809: mknod d19/d1b/d1c/d1e/d22/db1/dcf/c106 0 2024-04-20T10:59:26.449 INFO:tasks.workunit.client.0.smithi027.stdout:4/834: dread d0/d14/d6b/da3/d18/db9/fd6 [0,4194304] 0 2024-04-20T10:59:26.450 INFO:tasks.workunit.client.0.smithi027.stdout:2/814: mkdir d5/dc/d16/d32/dab/db9/d10c 0 2024-04-20T10:59:26.450 INFO:tasks.workunit.client.0.smithi027.stdout:2/815: fsync d5/d44/f90 0 2024-04-20T10:59:26.451 INFO:tasks.workunit.client.0.smithi027.stdout:5/810: creat d19/d1b/d1c/d1e/d22/dca/f107 x:0 0 0 2024-04-20T10:59:26.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/835: rename d0/d14/d6b/d92/l100 to d0/d48/d123/d33/d97/l127 0 2024-04-20T10:59:26.457 INFO:tasks.workunit.client.0.smithi027.stdout:5/811: creat d19/d1b/d1c/d1e/d25/d40/d82/f108 x:0 0 0 2024-04-20T10:59:26.457 INFO:tasks.workunit.client.0.smithi027.stdout:5/812: readlink d19/d1b/d1c/d1e/d22/d68/d75/lab 0 2024-04-20T10:59:26.457 INFO:tasks.workunit.client.0.smithi027.stdout:5/813: fsync d19/d1b/d1c/d1e/d3c/fba 0 2024-04-20T10:59:26.459 INFO:tasks.workunit.client.0.smithi027.stdout:5/814: dread d19/d1b/d1c/d1e/d22/db1/fb3 [0,4194304] 0 2024-04-20T10:59:26.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/815: creat d19/d1b/d1c/d1e/d22/d68/f109 x:0 0 0 2024-04-20T10:59:26.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/816: write d19/d1b/d1c/d1e/d58/f60 [1159033,115441] 0 2024-04-20T10:59:26.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/817: chown d19/d1b/d1c/d1e/d22/d68/f90 4 1 2024-04-20T10:59:26.467 INFO:tasks.workunit.client.0.smithi027.stdout:5/818: dread f0 [0,4194304] 0 2024-04-20T10:59:26.476 INFO:tasks.workunit.client.0.smithi027.stdout:8/940: dwrite d9/de/d20/f9d [0,4194304] 0 2024-04-20T10:59:26.477 INFO:tasks.workunit.client.0.smithi027.stdout:4/836: dwrite d0/d48/d123/f30 [0,4194304] 0 2024-04-20T10:59:26.478 INFO:tasks.workunit.client.0.smithi027.stdout:5/819: rename d19/d1b/d1c/d1e/d22/d68/lac to d19/d1b/d1c/d1e/d25/d40/d82/l10a 0 2024-04-20T10:59:26.480 INFO:tasks.workunit.client.0.smithi027.stdout:4/837: creat d0/d48/d68/f128 x:0 0 0 2024-04-20T10:59:26.480 INFO:tasks.workunit.client.0.smithi027.stdout:5/820: dread d19/d1b/d1c/d1e/d22/f5a [0,4194304] 0 2024-04-20T10:59:26.480 INFO:tasks.workunit.client.0.smithi027.stdout:5/821: write d19/d1b/d1c/d1e/d3c/f3d [1204974,115514] 0 2024-04-20T10:59:26.497 INFO:tasks.workunit.client.0.smithi027.stdout:4/838: dwrite d0/d14/fc4 [0,4194304] 0 2024-04-20T10:59:26.497 INFO:tasks.workunit.client.0.smithi027.stdout:9/754: sync 2024-04-20T10:59:26.497 INFO:tasks.workunit.client.0.smithi027.stdout:7/847: sync 2024-04-20T10:59:26.497 INFO:tasks.workunit.client.0.smithi027.stdout:7/848: truncate d4/d5/d8/d21/d94/fff 375997 0 2024-04-20T10:59:26.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/822: symlink d19/d1b/d1c/d1e/da3/l10b 0 2024-04-20T10:59:26.499 INFO:tasks.workunit.client.0.smithi027.stdout:4/839: link d0/d14/d6b/da3/d53/c5a d0/d48/d123/d33/d5c/c129 0 2024-04-20T10:59:26.499 INFO:tasks.workunit.client.0.smithi027.stdout:4/840: chown d0/d48/d68/d76/l103 1351838896 1 2024-04-20T10:59:26.500 INFO:tasks.workunit.client.0.smithi027.stdout:7/849: stat d4/d5/d8/d45/cba 0 2024-04-20T10:59:26.500 INFO:tasks.workunit.client.0.smithi027.stdout:7/850: readlink d4/d1b/l1c 0 2024-04-20T10:59:26.500 INFO:tasks.workunit.client.0.smithi027.stdout:5/823: truncate d19/d1b/d1c/d1e/d3c/f74 2121897 0 2024-04-20T10:59:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:4/841: unlink d0/d48/d123/f9c 0 2024-04-20T10:59:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:4/842: chown d0/d14/d6b/da3/d53/d6a/ff3 38918377 1 2024-04-20T10:59:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:4/843: write d0/d48/d123/d2f/f82 [1926133,73555] 0 2024-04-20T10:59:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:4/844: readlink d0/d14/d6b/da3/d18/d10a/l99 0 2024-04-20T10:59:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/755: write d4/f1e [1280441,104028] 0 2024-04-20T10:59:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/756: chown d4/d1b/d1f/d45/d8a/db2 196601 1 2024-04-20T10:59:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/757: chown d4/d1b/d50/d5d/fd7 75 1 2024-04-20T10:59:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/758: truncate d4/d1b/d1f/d5a/fc9 341144 0 2024-04-20T10:59:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:7/851: mknod d4/d5/d8/d46/d77/c113 0 2024-04-20T10:59:26.504 INFO:tasks.workunit.client.0.smithi027.stdout:5/824: creat d19/d1b/d1c/d1e/d22/d68/d75/f10c x:0 0 0 2024-04-20T10:59:26.506 INFO:tasks.workunit.client.0.smithi027.stdout:9/759: stat d4/d2a/lb9 0 2024-04-20T10:59:26.508 INFO:tasks.workunit.client.0.smithi027.stdout:4/845: link d0/d14/d6b/da3/d1a/c88 d0/d14/d6b/da3/d53/d116/c12a 0 2024-04-20T10:59:26.509 INFO:tasks.workunit.client.0.smithi027.stdout:9/760: unlink d4/f1e 0 2024-04-20T10:59:26.510 INFO:tasks.workunit.client.0.smithi027.stdout:4/846: dread d0/d14/d6b/da3/d53/d6a/faa [0,4194304] 0 2024-04-20T10:59:26.515 INFO:tasks.workunit.client.0.smithi027.stdout:2/816: dwrite d5/dc/d16/d37/f46 [0,4194304] 0 2024-04-20T10:59:26.527 INFO:tasks.workunit.client.0.smithi027.stdout:9/761: dwrite d4/d1b/d31/d23/f47 [0,4194304] 0 2024-04-20T10:59:26.533 INFO:tasks.workunit.client.0.smithi027.stdout:2/817: dwrite d5/dc/d16/d2a/f9f [0,4194304] 0 2024-04-20T10:59:26.535 INFO:tasks.workunit.client.0.smithi027.stdout:9/762: unlink d4/d2a/l4d 0 2024-04-20T10:59:26.535 INFO:tasks.workunit.client.0.smithi027.stdout:9/763: chown d4/fc8 1535 1 2024-04-20T10:59:26.539 INFO:tasks.workunit.client.0.smithi027.stdout:6/903: sync 2024-04-20T10:59:26.539 INFO:tasks.workunit.client.0.smithi027.stdout:6/904: rename da/dcc/d108/d6b to da/dcc/d108/d6b/d10d/d116/d12a 22 2024-04-20T10:59:26.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/818: mkdir d5/dc/d10d 0 2024-04-20T10:59:26.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/819: dread d5/dc/fff [0,4194304] 0 2024-04-20T10:59:26.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/820: chown d5/d26/d47/fdc 1206208 1 2024-04-20T10:59:26.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/821: fsync d5/d26/d47/fea 0 2024-04-20T10:59:26.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/822: readlink d5/dc/d16/l2f 0 2024-04-20T10:59:26.542 INFO:tasks.workunit.client.0.smithi027.stdout:3/922: sync 2024-04-20T10:59:26.543 INFO:tasks.workunit.client.0.smithi027.stdout:3/923: chown d2/d18/d2c/d4c/d67/d6c 148810528 1 2024-04-20T10:59:26.547 INFO:tasks.workunit.client.0.smithi027.stdout:6/905: rmdir da/d21/dab 39 2024-04-20T10:59:26.547 INFO:tasks.workunit.client.0.smithi027.stdout:6/906: chown da/d21/d28/d2d/d53/dae 1717 1 2024-04-20T10:59:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/823: dread d5/d26/d78/f7b [0,4194304] 0 2024-04-20T10:59:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/824: stat d5/dc/d16/d2a/f75 0 2024-04-20T10:59:26.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/825: chown d5/dc/d16/d2a/d59/da4/de1 998 1 2024-04-20T10:59:26.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/764: mknod d4/ce2 0 2024-04-20T10:59:26.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/924: dread d2/dd/f19 [0,4194304] 0 2024-04-20T10:59:26.553 INFO:tasks.workunit.client.0.smithi027.stdout:6/907: rename da/dd/d3d/db9/d127/cf7 to da/dcc/d108/d6b/de0/c12b 0 2024-04-20T10:59:26.553 INFO:tasks.workunit.client.0.smithi027.stdout:6/908: chown da/l6c 35549213 1 2024-04-20T10:59:26.553 INFO:tasks.workunit.client.0.smithi027.stdout:2/826: getdents d5/dc/d16/d37/de2/dac/d8f 0 2024-04-20T10:59:26.558 INFO:tasks.workunit.client.0.smithi027.stdout:9/765: write d4/d74/d72/f98 [3839566,97692] 0 2024-04-20T10:59:26.559 INFO:tasks.workunit.client.0.smithi027.stdout:3/925: truncate d2/dd/d57/d8d/fd2 40918 0 2024-04-20T10:59:26.560 INFO:tasks.workunit.client.0.smithi027.stdout:6/909: mknod da/dd/d3c/d9e/c12c 0 2024-04-20T10:59:26.565 INFO:tasks.workunit.client.0.smithi027.stdout:7/852: sync 2024-04-20T10:59:26.566 INFO:tasks.workunit.client.0.smithi027.stdout:3/926: symlink d2/d18/d2c/d4c/d67/d6c/dd0/dd5/l122 0 2024-04-20T10:59:26.571 INFO:tasks.workunit.client.0.smithi027.stdout:2/827: dwrite d5/dc/d16/d2a/f77 [0,4194304] 0 2024-04-20T10:59:26.572 INFO:tasks.workunit.client.0.smithi027.stdout:6/910: getdents da/d21/d28/d54 0 2024-04-20T10:59:26.573 INFO:tasks.workunit.client.0.smithi027.stdout:7/853: rename d4/d4d/f4f to d4/d5/d6f/d102/f114 0 2024-04-20T10:59:26.573 INFO:tasks.workunit.client.0.smithi027.stdout:7/854: chown d4/d5b 52 1 2024-04-20T10:59:26.588 INFO:tasks.workunit.client.0.smithi027.stdout:2/828: dwrite d5/dc/d16/db1/fe5 [0,4194304] 0 2024-04-20T10:59:26.589 INFO:tasks.workunit.client.0.smithi027.stdout:6/911: symlink da/dd/d3d/db9/l12d 0 2024-04-20T10:59:26.589 INFO:tasks.workunit.client.0.smithi027.stdout:7/855: mknod d4/d5/d8/d45/c115 0 2024-04-20T10:59:26.591 INFO:tasks.workunit.client.0.smithi027.stdout:2/829: truncate d5/d26/fc0 475494 0 2024-04-20T10:59:26.591 INFO:tasks.workunit.client.0.smithi027.stdout:2/830: write d5/dc/d16/d2a/f84 [92457,94485] 0 2024-04-20T10:59:26.592 INFO:tasks.workunit.client.0.smithi027.stdout:6/912: symlink da/dd/d3d/db9/d127/de8/l12e 0 2024-04-20T10:59:26.593 INFO:tasks.workunit.client.0.smithi027.stdout:7/856: link d4/d1b/l1c d4/d5b/l116 0 2024-04-20T10:59:26.593 INFO:tasks.workunit.client.0.smithi027.stdout:4/847: sync 2024-04-20T10:59:26.597 INFO:tasks.workunit.client.0.smithi027.stdout:4/848: dread d0/d48/d123/f61 [0,4194304] 0 2024-04-20T10:59:26.597 INFO:tasks.workunit.client.0.smithi027.stdout:4/849: write d0/d5e/fba [485501,23496] 0 2024-04-20T10:59:26.603 INFO:tasks.workunit.client.0.smithi027.stdout:4/850: write d0/d14/d6b/da3/d18/f4b [3937939,79936] 0 2024-04-20T10:59:26.604 INFO:tasks.workunit.client.0.smithi027.stdout:2/831: creat d5/dc/d16/d32/dab/db9/f10e x:0 0 0 2024-04-20T10:59:26.605 INFO:tasks.workunit.client.0.smithi027.stdout:6/913: mkdir da/d21/dab/d1f/dbf/d12f 0 2024-04-20T10:59:26.606 INFO:tasks.workunit.client.0.smithi027.stdout:4/851: truncate d0/d48/d123/d2f/ff6 3690739 0 2024-04-20T10:59:26.607 INFO:tasks.workunit.client.0.smithi027.stdout:2/832: symlink d5/d26/l10f 0 2024-04-20T10:59:26.607 INFO:tasks.workunit.client.0.smithi027.stdout:2/833: chown d5/dc/d16/d37/de2/dac/d8f/dfd/cf7 67 1 2024-04-20T10:59:26.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/914: creat da/d21/dab/d7e/d86/da3/f130 x:0 0 0 2024-04-20T10:59:26.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/915: chown da/dd/d3c/ld6 128492 1 2024-04-20T10:59:26.609 INFO:tasks.workunit.client.0.smithi027.stdout:7/857: dwrite d4/d1b/d4a/d62/fa0 [0,4194304] 0 2024-04-20T10:59:26.609 INFO:tasks.workunit.client.0.smithi027.stdout:6/916: dread da/d21/dab/f11 [0,4194304] 0 2024-04-20T10:59:26.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/834: getdents d5/dc/d16/d2a/d59 0 2024-04-20T10:59:26.612 INFO:tasks.workunit.client.0.smithi027.stdout:7/858: unlink d4/d5b/f9e 0 2024-04-20T10:59:26.612 INFO:tasks.workunit.client.0.smithi027.stdout:7/859: write d4/d1b/d4a/f44 [2965219,12042] 0 2024-04-20T10:59:26.613 INFO:tasks.workunit.client.0.smithi027.stdout:6/917: mkdir da/dcc/d131 0 2024-04-20T10:59:26.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/852: link d0/d48/d123/d33/d5c/fd2 d0/d14/f12b 0 2024-04-20T10:59:26.614 INFO:tasks.workunit.client.0.smithi027.stdout:4/853: readlink d0/d14/d6b/d92/dcd/led 0 2024-04-20T10:59:26.615 INFO:tasks.workunit.client.0.smithi027.stdout:2/835: symlink d5/l110 0 2024-04-20T10:59:26.615 INFO:tasks.workunit.client.0.smithi027.stdout:8/941: dread d9/de/d1e/d4a/d6d/db7/f138 [0,4194304] 0 2024-04-20T10:59:26.615 INFO:tasks.workunit.client.0.smithi027.stdout:8/942: read - d9/de/d1e/d4a/d6d/d79/fba zero size 2024-04-20T10:59:26.616 INFO:tasks.workunit.client.0.smithi027.stdout:7/860: mkdir d4/da2/dd1/dfb/d104/d117 0 2024-04-20T10:59:26.619 INFO:tasks.workunit.client.0.smithi027.stdout:2/836: unlink d5/d26/d78/cdf 0 2024-04-20T10:59:26.619 INFO:tasks.workunit.client.0.smithi027.stdout:8/943: symlink d9/de/d1e/d4a/d6d/d75/l142 0 2024-04-20T10:59:26.619 INFO:tasks.workunit.client.0.smithi027.stdout:8/944: chown d9/de/d1e/d4a/d6d/dea/d59/d74/de8/c130 62199 1 2024-04-20T10:59:26.626 INFO:tasks.workunit.client.0.smithi027.stdout:5/825: dread d19/d1b/d1c/d1e/d58/f7b [0,4194304] 0 2024-04-20T10:59:26.627 INFO:tasks.workunit.client.0.smithi027.stdout:7/861: link d4/c3a d4/c118 0 2024-04-20T10:59:26.627 INFO:tasks.workunit.client.0.smithi027.stdout:7/862: write d4/d5/d8/d46/d106/f109 [718540,98347] 0 2024-04-20T10:59:26.630 INFO:tasks.workunit.client.0.smithi027.stdout:8/945: write d9/de/d1e/d4a/d6d/d75/f83 [481290,45029] 0 2024-04-20T10:59:26.637 INFO:tasks.workunit.client.0.smithi027.stdout:6/918: mknod da/c132 0 2024-04-20T10:59:26.638 INFO:tasks.workunit.client.0.smithi027.stdout:9/766: sync 2024-04-20T10:59:26.638 INFO:tasks.workunit.client.0.smithi027.stdout:3/927: sync 2024-04-20T10:59:26.638 INFO:tasks.workunit.client.0.smithi027.stdout:3/928: readlink d2/dd/d57/d8d/dc6/de1/lf8 0 2024-04-20T10:59:26.638 INFO:tasks.workunit.client.0.smithi027.stdout:9/767: write d4/d1b/d1f/d45/d8a/f6e [2286682,28720] 0 2024-04-20T10:59:26.638 INFO:tasks.workunit.client.0.smithi027.stdout:5/826: mkdir d19/d10d 0 2024-04-20T10:59:26.639 INFO:tasks.workunit.client.0.smithi027.stdout:6/919: rename da/dcc/d108/fb4 to da/dcc/d108/d6b/f133 0 2024-04-20T10:59:26.640 INFO:tasks.workunit.client.0.smithi027.stdout:3/929: rmdir d2/d18/d2c/d4c/d67/d6c 39 2024-04-20T10:59:26.642 INFO:tasks.workunit.client.0.smithi027.stdout:9/768: truncate d4/d1b/d50/d5d/fb1 4168949 0 2024-04-20T10:59:26.643 INFO:tasks.workunit.client.0.smithi027.stdout:5/827: symlink d19/d1b/d1c/d1e/d22/dca/l10e 0 2024-04-20T10:59:26.644 INFO:tasks.workunit.client.0.smithi027.stdout:5/828: creat d19/d1b/f10f x:0 0 0 2024-04-20T10:59:26.647 INFO:tasks.workunit.client.0.smithi027.stdout:5/829: read f0 [2155879,35622] 0 2024-04-20T10:59:26.648 INFO:tasks.workunit.client.0.smithi027.stdout:5/830: truncate f3 1014281 0 2024-04-20T10:59:26.650 INFO:tasks.workunit.client.0.smithi027.stdout:5/831: rename d19/l2d to d19/d1b/d1c/d1e/da3/db9/l110 0 2024-04-20T10:59:26.651 INFO:tasks.workunit.client.0.smithi027.stdout:5/832: symlink d19/d1b/d1c/d1e/l111 0 2024-04-20T10:59:26.655 INFO:tasks.workunit.client.0.smithi027.stdout:7/863: dwrite d4/d5/d8/f82 [0,4194304] 0 2024-04-20T10:59:26.655 INFO:tasks.workunit.client.0.smithi027.stdout:7/864: write d4/da2/dd1/f103 [31088,39214] 0 2024-04-20T10:59:26.656 INFO:tasks.workunit.client.0.smithi027.stdout:7/865: mkdir d4/da2/dd1/dfb/d104/d117/d119 0 2024-04-20T10:59:26.656 INFO:tasks.workunit.client.0.smithi027.stdout:3/930: write d2/dd/f1a [5911343,4234] 0 2024-04-20T10:59:26.657 INFO:tasks.workunit.client.0.smithi027.stdout:3/931: dread d2/dd/d57/d8c/ff7 [0,4194304] 0 2024-04-20T10:59:26.658 INFO:tasks.workunit.client.0.smithi027.stdout:6/920: dwrite da/d21/dab/d7e/d86/da3/f130 [0,4194304] 0 2024-04-20T10:59:26.658 INFO:tasks.workunit.client.0.smithi027.stdout:6/921: fdatasync da/d21/dab/d19/f124 0 2024-04-20T10:59:26.671 INFO:tasks.workunit.client.0.smithi027.stdout:2/837: dread d5/dc/d16/d37/f3f [0,4194304] 0 2024-04-20T10:59:26.671 INFO:tasks.workunit.client.0.smithi027.stdout:2/838: chown d5/dc/d16/d2a 0 1 2024-04-20T10:59:26.677 INFO:tasks.workunit.client.0.smithi027.stdout:6/922: dwrite da/dcc/d108/d6b/f133 [0,4194304] 0 2024-04-20T10:59:26.682 INFO:tasks.workunit.client.0.smithi027.stdout:6/923: write da/d21/dab/d19/f6a [1657,35778] 0 2024-04-20T10:59:26.913 INFO:tasks.workunit.client.0.smithi027.stdout:7/866: creat d4/d5/d81/f11a x:0 0 0 2024-04-20T10:59:26.914 INFO:tasks.workunit.client.0.smithi027.stdout:2/839: rmdir d5/dc 39 2024-04-20T10:59:26.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/924: link da/d21/d28/d2d/d53/d9a/lcb da/dd/d3d/db9/d127/l134 0 2024-04-20T10:59:26.918 INFO:tasks.workunit.client.0.smithi027.stdout:2/840: mknod d5/d26/d30/c111 0 2024-04-20T10:59:26.919 INFO:tasks.workunit.client.0.smithi027.stdout:6/925: creat da/d21/dab/d7e/d86/dcd/f135 x:0 0 0 2024-04-20T10:59:26.919 INFO:tasks.workunit.client.0.smithi027.stdout:6/926: chown da/dd/d3d/db9/f103 316 1 2024-04-20T10:59:26.920 INFO:tasks.workunit.client.0.smithi027.stdout:6/927: write da/dcc/d108/d6b/de0/f113 [381134,120283] 0 2024-04-20T10:59:26.920 INFO:tasks.workunit.client.0.smithi027.stdout:6/928: symlink da/dcc/d108/d32/l136 0 2024-04-20T10:59:26.922 INFO:tasks.workunit.client.0.smithi027.stdout:2/841: dread d5/dc/d16/d2a/f75 [0,4194304] 0 2024-04-20T10:59:26.931 INFO:tasks.workunit.client.0.smithi027.stdout:2/842: write d5/dc/d16/db1/fe5 [823267,23541] 0 2024-04-20T10:59:26.931 INFO:tasks.workunit.client.0.smithi027.stdout:2/843: dread - d5/dc/d16/d37/de2/dac/fe6 zero size 2024-04-20T10:59:26.931 INFO:tasks.workunit.client.0.smithi027.stdout:2/844: stat d5/dc/d16/d37/de2/f101 0 2024-04-20T10:59:26.932 INFO:tasks.workunit.client.0.smithi027.stdout:2/845: rmdir d5/dc/d16/d32/dab 39 2024-04-20T10:59:26.932 INFO:tasks.workunit.client.0.smithi027.stdout:2/846: readlink d5/d26/d78/lb6 0 2024-04-20T10:59:26.932 INFO:tasks.workunit.client.0.smithi027.stdout:2/847: creat d5/f112 x:0 0 0 2024-04-20T10:59:26.933 INFO:tasks.workunit.client.0.smithi027.stdout:2/848: mkdir d5/dc/d16/d32/d109/d113 0 2024-04-20T10:59:26.933 INFO:tasks.workunit.client.0.smithi027.stdout:2/849: fsync d5/d44/d7c/f7e 0 2024-04-20T10:59:26.940 INFO:tasks.workunit.client.0.smithi027.stdout:7/867: dwrite d4/d5/d8/d3b/f3e [0,4194304] 0 2024-04-20T10:59:26.940 INFO:tasks.workunit.client.0.smithi027.stdout:7/868: fdatasync d4/d5/d8/d21/d94/fbd 0 2024-04-20T10:59:26.940 INFO:tasks.workunit.client.0.smithi027.stdout:8/946: dwrite d9/de/d1e/d4a/d6d/db7/f138 [0,4194304] 0 2024-04-20T10:59:26.941 INFO:tasks.workunit.client.0.smithi027.stdout:7/869: link d4/da2/fb5 d4/d5/d81/f11b 0 2024-04-20T10:59:26.941 INFO:tasks.workunit.client.0.smithi027.stdout:7/870: write d4/da2/dd1/f103 [59052,79775] 0 2024-04-20T10:59:26.943 INFO:tasks.workunit.client.0.smithi027.stdout:8/947: creat d9/de/d1e/d4a/d6d/db7/f143 x:0 0 0 2024-04-20T10:59:26.943 INFO:tasks.workunit.client.0.smithi027.stdout:8/948: read d9/de/da3/d3c/f9c [236696,32800] 0 2024-04-20T10:59:26.943 INFO:tasks.workunit.client.0.smithi027.stdout:2/850: write d5/dc/d16/d37/fc8 [3181653,66099] 0 2024-04-20T10:59:26.948 INFO:tasks.workunit.client.0.smithi027.stdout:7/871: unlink d4/d5/d8/d21/d94/fa8 0 2024-04-20T10:59:26.948 INFO:tasks.workunit.client.0.smithi027.stdout:7/872: chown d4/cb 3125286 1 2024-04-20T10:59:26.949 INFO:tasks.workunit.client.0.smithi027.stdout:7/873: fsync d4/d5/d8/d46/d77/fcf 0 2024-04-20T10:59:26.949 INFO:tasks.workunit.client.0.smithi027.stdout:7/874: fdatasync d4/d5/f6c 0 2024-04-20T10:59:26.949 INFO:tasks.workunit.client.0.smithi027.stdout:7/875: fsync d4/d5/d8/d3b/f3e 0 2024-04-20T10:59:26.949 INFO:tasks.workunit.client.0.smithi027.stdout:7/876: readlink d4/d5/d8/d3b/l51 0 2024-04-20T10:59:26.949 INFO:tasks.workunit.client.0.smithi027.stdout:7/877: write d4/d1b/f88 [829281,57374] 0 2024-04-20T10:59:26.949 INFO:tasks.workunit.client.0.smithi027.stdout:2/851: mknod d5/dc/d16/d2a/d59/da4/c114 0 2024-04-20T10:59:26.952 INFO:tasks.workunit.client.0.smithi027.stdout:7/878: mknod d4/d5b/c11c 0 2024-04-20T10:59:26.953 INFO:tasks.workunit.client.0.smithi027.stdout:2/852: getdents d5/dc/d16/d37/de2/dac 0 2024-04-20T10:59:26.953 INFO:tasks.workunit.client.0.smithi027.stdout:2/853: chown d5/d26/d30 1149 1 2024-04-20T10:59:26.953 INFO:tasks.workunit.client.0.smithi027.stdout:2/854: stat d5/d26/d78/lb6 0 2024-04-20T10:59:26.954 INFO:tasks.workunit.client.0.smithi027.stdout:2/855: rmdir d5/d26 39 2024-04-20T10:59:26.955 INFO:tasks.workunit.client.0.smithi027.stdout:2/856: truncate d5/d44/f90 811535 0 2024-04-20T10:59:26.961 INFO:tasks.workunit.client.0.smithi027.stdout:3/932: dwrite d2/f21 [8388608,4194304] 0 2024-04-20T10:59:26.962 INFO:tasks.workunit.client.0.smithi027.stdout:7/879: write d4/d5b/fae [543864,65234] 0 2024-04-20T10:59:26.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/933: creat d2/dd/d57/de7/dff/d100/f123 x:0 0 0 2024-04-20T10:59:26.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/934: write d2/dd/df/d23/fef [113210,65566] 0 2024-04-20T10:59:26.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/935: stat d2/dd/d57/de7/f113 0 2024-04-20T10:59:26.969 INFO:tasks.workunit.client.0.smithi027.stdout:7/880: getdents d4/d5/d8/d21/d94/da7 0 2024-04-20T10:59:26.970 INFO:tasks.workunit.client.0.smithi027.stdout:3/936: mkdir d2/dd/d57/de7/dff/d124 0 2024-04-20T10:59:26.970 INFO:tasks.workunit.client.0.smithi027.stdout:3/937: write d2/d18/d2c/f4f [1841825,666] 0 2024-04-20T10:59:26.972 INFO:tasks.workunit.client.0.smithi027.stdout:7/881: rename d4/d5/d8/d46/d106/f10f to d4/d5/d8/d46/d77/d7c/f11d 0 2024-04-20T10:59:26.973 INFO:tasks.workunit.client.0.smithi027.stdout:7/882: mkdir d4/d5/d8/d45/d11e 0 2024-04-20T10:59:26.975 INFO:tasks.workunit.client.0.smithi027.stdout:4/854: sync 2024-04-20T10:59:26.975 INFO:tasks.workunit.client.0.smithi027.stdout:5/833: sync 2024-04-20T10:59:26.975 INFO:tasks.workunit.client.0.smithi027.stdout:5/834: write d19/d1b/d1c/d1e/d25/d40/d82/f108 [507427,120860] 0 2024-04-20T10:59:26.975 INFO:tasks.workunit.client.0.smithi027.stdout:5/835: write d19/d1b/d1c/d1e/d22/d68/d75/f77 [1106352,70057] 0 2024-04-20T10:59:26.976 INFO:tasks.workunit.client.0.smithi027.stdout:7/883: unlink d4/d1b/d4a/d7a/d93/fde 0 2024-04-20T10:59:26.976 INFO:tasks.workunit.client.0.smithi027.stdout:7/884: fdatasync d4/f70 0 2024-04-20T10:59:26.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/769: sync 2024-04-20T10:59:26.976 INFO:tasks.workunit.client.0.smithi027.stdout:2/857: dwrite d5/d26/d47/fdc [0,4194304] 0 2024-04-20T10:59:26.976 INFO:tasks.workunit.client.0.smithi027.stdout:2/858: dread - d5/dc/d16/d32/fb7 zero size 2024-04-20T10:59:26.978 INFO:tasks.workunit.client.0.smithi027.stdout:5/836: symlink d19/d1b/d1c/d1e/d22/db1/dcf/l112 0 2024-04-20T10:59:26.978 INFO:tasks.workunit.client.0.smithi027.stdout:7/885: rename d4/f70 to d4/d5/d8/d45/d11e/f11f 0 2024-04-20T10:59:26.978 INFO:tasks.workunit.client.0.smithi027.stdout:9/770: mknod d4/d1b/d31/d51/ce3 0 2024-04-20T10:59:26.979 INFO:tasks.workunit.client.0.smithi027.stdout:5/837: creat d19/d1b/d1c/d1e/f113 x:0 0 0 2024-04-20T10:59:26.980 INFO:tasks.workunit.client.0.smithi027.stdout:7/886: symlink d4/da2/l120 0 2024-04-20T10:59:26.980 INFO:tasks.workunit.client.0.smithi027.stdout:3/938: write d2/d18/d2c/f78 [3891188,92045] 0 2024-04-20T10:59:26.980 INFO:tasks.workunit.client.0.smithi027.stdout:9/771: symlink d4/d1b/d1f/d45/d9e/le4 0 2024-04-20T10:59:26.983 INFO:tasks.workunit.client.0.smithi027.stdout:5/838: rmdir d19/d1b/d1c/d1e/d22/dec 0 2024-04-20T10:59:26.984 INFO:tasks.workunit.client.0.smithi027.stdout:9/772: rename d4/d1b/d50/d5d/fd7 to d4/d1b/d1f/d45/d9e/fe5 0 2024-04-20T10:59:26.984 INFO:tasks.workunit.client.0.smithi027.stdout:9/773: chown d4/d1b/d1f/d5a/fc9 484055426 1 2024-04-20T10:59:26.986 INFO:tasks.workunit.client.0.smithi027.stdout:5/839: dread d19/d1b/d1c/d1e/d25/d40/fe7 [0,4194304] 0 2024-04-20T10:59:26.987 INFO:tasks.workunit.client.0.smithi027.stdout:5/840: symlink d19/d1b/d1c/d1e/d58/l114 0 2024-04-20T10:59:26.988 INFO:tasks.workunit.client.0.smithi027.stdout:2/859: dread d5/dc/d16/d2a/f77 [0,4194304] 0 2024-04-20T10:59:26.989 INFO:tasks.workunit.client.0.smithi027.stdout:5/841: creat d19/d1b/d1c/d1e/d22/db1/dcf/d91/f115 x:0 0 0 2024-04-20T10:59:26.990 INFO:tasks.workunit.client.0.smithi027.stdout:2/860: mknod d5/dc/d16/d32/c115 0 2024-04-20T10:59:26.990 INFO:tasks.workunit.client.0.smithi027.stdout:3/939: write d2/dd/d22/fae [4763279,25038] 0 2024-04-20T10:59:26.991 INFO:tasks.workunit.client.0.smithi027.stdout:5/842: truncate d19/d1b/d1c/d1e/d22/f89 19170 0 2024-04-20T10:59:26.993 INFO:tasks.workunit.client.0.smithi027.stdout:3/940: creat d2/dd/df/d23/f125 x:0 0 0 2024-04-20T10:59:26.994 INFO:tasks.workunit.client.0.smithi027.stdout:3/941: rmdir d2/dd/d57/d8d/dc6/de1 39 2024-04-20T10:59:26.994 INFO:tasks.workunit.client.0.smithi027.stdout:3/942: fsync d2/dd/f39 0 2024-04-20T10:59:26.995 INFO:tasks.workunit.client.0.smithi027.stdout:3/943: rename d2/dd/df/f3a to d2/dd/d57/d8d/d9d/f126 0 2024-04-20T10:59:26.995 INFO:tasks.workunit.client.0.smithi027.stdout:3/944: readlink d2/dd/df/d23/d68/l9b 0 2024-04-20T10:59:26.995 INFO:tasks.workunit.client.0.smithi027.stdout:3/945: chown d2/d18/c9e 0 1 2024-04-20T10:59:26.997 INFO:tasks.workunit.client.0.smithi027.stdout:5/843: dread d19/d1b/f86 [0,4194304] 0 2024-04-20T10:59:26.997 INFO:tasks.workunit.client.0.smithi027.stdout:5/844: rmdir d19/d1b/d1c/d1e/d22/dca 39 2024-04-20T10:59:26.999 INFO:tasks.workunit.client.0.smithi027.stdout:5/845: mkdir d19/d1b/d1c/d1e/d116 0 2024-04-20T10:59:26.999 INFO:tasks.workunit.client.0.smithi027.stdout:7/887: dwrite d4/d5/d8/d21/f96 [0,4194304] 0 2024-04-20T10:59:27.000 INFO:tasks.workunit.client.0.smithi027.stdout:4/855: dwrite d0/d48/ff5 [0,4194304] 0 2024-04-20T10:59:27.002 INFO:tasks.workunit.client.0.smithi027.stdout:5/846: link d19/d1b/d1c/d1e/d22/d68/d75/f10c d19/d1b/d1c/d1e/d58/db4/f117 0 2024-04-20T10:59:27.002 INFO:tasks.workunit.client.0.smithi027.stdout:7/888: creat d4/da2/dd1/dfb/d104/f121 x:0 0 0 2024-04-20T10:59:27.002 INFO:tasks.workunit.client.0.smithi027.stdout:7/889: dread - d4/d1b/ff6 zero size 2024-04-20T10:59:27.005 INFO:tasks.workunit.client.0.smithi027.stdout:4/856: link d0/d14/d6b/da3/fe0 d0/d14/d6b/d92/d11e/f12c 0 2024-04-20T10:59:27.005 INFO:tasks.workunit.client.0.smithi027.stdout:4/857: write d0/d14/d6b/d92/fc7 [414851,2784] 0 2024-04-20T10:59:27.009 INFO:tasks.workunit.client.0.smithi027.stdout:7/890: mkdir d4/d5/d6f/d102/d122 0 2024-04-20T10:59:27.009 INFO:tasks.workunit.client.0.smithi027.stdout:4/858: rename d0/d48/d123/c77 to d0/d48/d123/d33/d5c/d6f/dd1/c12d 0 2024-04-20T10:59:27.011 INFO:tasks.workunit.client.0.smithi027.stdout:2/861: dwrite d5/d26/d47/ffa [0,4194304] 0 2024-04-20T10:59:27.015 INFO:tasks.workunit.client.0.smithi027.stdout:2/862: symlink d5/dc/d16/d32/dab/l116 0 2024-04-20T10:59:27.017 INFO:tasks.workunit.client.0.smithi027.stdout:2/863: link d5/dc/d16/d37/de2/d8e/fb5 d5/dc/d16/dd0/f117 0 2024-04-20T10:59:27.017 INFO:tasks.workunit.client.0.smithi027.stdout:2/864: dread - d5/d26/ffb zero size 2024-04-20T10:59:27.019 INFO:tasks.workunit.client.0.smithi027.stdout:3/946: dwrite d2/dd/d57/d8d/fba [0,4194304] 0 2024-04-20T10:59:27.022 INFO:tasks.workunit.client.0.smithi027.stdout:3/947: symlink d2/d18/d2c/d4c/d10b/l127 0 2024-04-20T10:59:27.024 INFO:tasks.workunit.client.0.smithi027.stdout:3/948: link d2/dd/d57/d8d/fe9 d2/dd/d22/f128 0 2024-04-20T10:59:27.025 INFO:tasks.workunit.client.0.smithi027.stdout:3/949: creat d2/f129 x:0 0 0 2024-04-20T10:59:27.026 INFO:tasks.workunit.client.0.smithi027.stdout:3/950: mkdir d2/dd/d57/d8d/dc6/d12a 0 2024-04-20T10:59:27.026 INFO:tasks.workunit.client.0.smithi027.stdout:3/951: chown d2/d18/d2c/d4c/d67/d6c/c82 0 1 2024-04-20T10:59:27.028 INFO:tasks.workunit.client.0.smithi027.stdout:3/952: rename d2/dd/df/d23/d68/c114 to d2/d94/c12b 0 2024-04-20T10:59:27.028 INFO:tasks.workunit.client.0.smithi027.stdout:3/953: chown d2/dd/df/d23/l5f 856540 1 2024-04-20T10:59:27.028 INFO:tasks.workunit.client.0.smithi027.stdout:3/954: truncate d2/dd/df/d23/df0/f120 451992 0 2024-04-20T10:59:27.028 INFO:tasks.workunit.client.0.smithi027.stdout:5/847: dwrite f0 [0,4194304] 0 2024-04-20T10:59:27.028 INFO:tasks.workunit.client.0.smithi027.stdout:5/848: write d19/d1b/d1c/d1e/d25/f34 [17646662,83383] 0 2024-04-20T10:59:27.028 INFO:tasks.workunit.client.0.smithi027.stdout:7/891: dwrite d4/da2/dd1/dfb/d104/f121 [0,4194304] 0 2024-04-20T10:59:27.031 INFO:tasks.workunit.client.0.smithi027.stdout:5/849: rename d19/d1b/d1c/d1e/d22/f5a to d19/d1b/d1c/d1e/d58/db4/f118 0 2024-04-20T10:59:27.032 INFO:tasks.workunit.client.0.smithi027.stdout:6/929: dread da/dd/d79/fa5 [0,4194304] 0 2024-04-20T10:59:27.032 INFO:tasks.workunit.client.0.smithi027.stdout:7/892: write d4/d5/d8/d45/d11e/f11f [623565,9187] 0 2024-04-20T10:59:27.034 INFO:tasks.workunit.client.0.smithi027.stdout:5/850: mkdir d19/d1b/d1c/d1e/da3/db9/d119 0 2024-04-20T10:59:27.034 INFO:tasks.workunit.client.0.smithi027.stdout:5/851: dread - d19/d1b/d1c/d1e/d3c/fba zero size 2024-04-20T10:59:27.034 INFO:tasks.workunit.client.0.smithi027.stdout:6/930: symlink da/d21/d28/d2d/d53/l137 0 2024-04-20T10:59:27.038 INFO:tasks.workunit.client.0.smithi027.stdout:5/852: creat d19/d1b/d1c/d1e/d83/f11a x:0 0 0 2024-04-20T10:59:27.038 INFO:tasks.workunit.client.0.smithi027.stdout:5/853: write d19/d1b/d1c/dbd/dbe/fe6 [3891435,9250] 0 2024-04-20T10:59:27.040 INFO:tasks.workunit.client.0.smithi027.stdout:6/931: creat da/dcc/d108/d32/f138 x:0 0 0 2024-04-20T10:59:27.040 INFO:tasks.workunit.client.0.smithi027.stdout:6/932: chown da/d21/d28/d54/c8d 10 1 2024-04-20T10:59:27.040 INFO:tasks.workunit.client.0.smithi027.stdout:6/933: chown da/dcc/d108/cd1 911485 1 2024-04-20T10:59:27.040 INFO:tasks.workunit.client.0.smithi027.stdout:6/934: dread - da/d21/dab/d7e/d86/fd5 zero size 2024-04-20T10:59:27.040 INFO:tasks.workunit.client.0.smithi027.stdout:6/935: write da/d21/d28/f3a [100093,190] 0 2024-04-20T10:59:27.040 INFO:tasks.workunit.client.0.smithi027.stdout:6/936: write da/d21/f8f [1676831,127055] 0 2024-04-20T10:59:27.040 INFO:tasks.workunit.client.0.smithi027.stdout:6/937: write da/d21/f128 [343671,44211] 0 2024-04-20T10:59:27.041 INFO:tasks.workunit.client.0.smithi027.stdout:7/893: symlink d4/da2/dd1/dfb/d104/l123 0 2024-04-20T10:59:27.046 INFO:tasks.workunit.client.0.smithi027.stdout:6/938: getdents da/d21 0 2024-04-20T10:59:27.046 INFO:tasks.workunit.client.0.smithi027.stdout:6/939: write da/dcc/d108/df1/f120 [1024110,14730] 0 2024-04-20T10:59:27.059 INFO:tasks.workunit.client.0.smithi027.stdout:5/854: dwrite d19/d1b/d1c/d1e/d25/fc0 [0,4194304] 0 2024-04-20T10:59:27.063 INFO:tasks.workunit.client.0.smithi027.stdout:5/855: mknod d19/d1b/d1c/d1e/da3/db9/d119/c11b 0 2024-04-20T10:59:27.063 INFO:tasks.workunit.client.0.smithi027.stdout:5/856: chown d19/d1b/d1c/d1e/c6e 1 1 2024-04-20T10:59:27.064 INFO:tasks.workunit.client.0.smithi027.stdout:5/857: write d19/d1b/d1c/d1e/d22/db1/f6b [2552815,66988] 0 2024-04-20T10:59:27.070 INFO:tasks.workunit.client.0.smithi027.stdout:4/859: dwrite d0/d14/d6b/da3/d18/f20 [0,4194304] 0 2024-04-20T10:59:27.070 INFO:tasks.workunit.client.0.smithi027.stdout:8/949: dwrite d9/de/d1e/d4a/d6d/d75/fee [0,4194304] 0 2024-04-20T10:59:27.071 INFO:tasks.workunit.client.0.smithi027.stdout:4/860: mknod d0/d14/d71/dfd/c12e 0 2024-04-20T10:59:27.073 INFO:tasks.workunit.client.0.smithi027.stdout:8/950: rename d9/de/d1e/d4a/d6d/db7/d97/cf0 to d9/de/d1e/dc1/dd1/c144 0 2024-04-20T10:59:27.073 INFO:tasks.workunit.client.0.smithi027.stdout:8/951: write d9/de/d1e/d4a/d6d/d75/f12b [901315,41808] 0 2024-04-20T10:59:27.073 INFO:tasks.workunit.client.0.smithi027.stdout:8/952: getdents d9/de/d20/dfd/d122/d126 0 2024-04-20T10:59:27.073 INFO:tasks.workunit.client.0.smithi027.stdout:8/953: chown d9/de/d18 13261836 1 2024-04-20T10:59:27.073 INFO:tasks.workunit.client.0.smithi027.stdout:7/894: dwrite d4/d1b/f53 [4194304,4194304] 0 2024-04-20T10:59:27.075 INFO:tasks.workunit.client.0.smithi027.stdout:7/895: creat d4/d5/d6f/d110/f124 x:0 0 0 2024-04-20T10:59:27.075 INFO:tasks.workunit.client.0.smithi027.stdout:3/955: dwrite d2/dd/df/f85 [0,4194304] 0 2024-04-20T10:59:27.075 INFO:tasks.workunit.client.0.smithi027.stdout:3/956: dread - d2/d18/f63 zero size 2024-04-20T10:59:27.079 INFO:tasks.workunit.client.0.smithi027.stdout:3/957: creat d2/d18/d2c/f12c x:0 0 0 2024-04-20T10:59:27.083 INFO:tasks.workunit.client.0.smithi027.stdout:4/861: write d0/d14/d6b/f125 [2449699,41160] 0 2024-04-20T10:59:27.083 INFO:tasks.workunit.client.0.smithi027.stdout:9/774: sync 2024-04-20T10:59:27.085 INFO:tasks.workunit.client.0.smithi027.stdout:4/862: truncate d0/d14/d6b/da3/d18/d10a/f89 88331 0 2024-04-20T10:59:27.086 INFO:tasks.workunit.client.0.smithi027.stdout:9/775: creat d4/d1b/d31/dd0/dda/fe6 x:0 0 0 2024-04-20T10:59:27.088 INFO:tasks.workunit.client.0.smithi027.stdout:4/863: rmdir d0/d48/d68/d11a 0 2024-04-20T10:59:27.088 INFO:tasks.workunit.client.0.smithi027.stdout:4/864: readlink d0/d14/d6b/da3/d18/dbf/ldb 0 2024-04-20T10:59:27.088 INFO:tasks.workunit.client.0.smithi027.stdout:4/865: chown d0/f108 16500895 1 2024-04-20T10:59:27.088 INFO:tasks.workunit.client.0.smithi027.stdout:4/866: write d0/d48/d123/d33/f35 [2756254,64850] 0 2024-04-20T10:59:27.088 INFO:tasks.workunit.client.0.smithi027.stdout:4/867: read - d0/d48/d123/fa5 zero size 2024-04-20T10:59:27.088 INFO:tasks.workunit.client.0.smithi027.stdout:9/776: symlink d4/d1b/d31/dd0/le7 0 2024-04-20T10:59:27.089 INFO:tasks.workunit.client.0.smithi027.stdout:4/868: mknod d0/d48/d68/d76/dfc/c12f 0 2024-04-20T10:59:27.089 INFO:tasks.workunit.client.0.smithi027.stdout:4/869: stat d0/d48/d123/d2f/f3f 0 2024-04-20T10:59:27.090 INFO:tasks.workunit.client.0.smithi027.stdout:9/777: rmdir d4/d1b/d1f/d45/d8a/db2 39 2024-04-20T10:59:27.091 INFO:tasks.workunit.client.0.smithi027.stdout:4/870: dread d0/d14/d6b/da3/d53/d6a/faa [0,4194304] 0 2024-04-20T10:59:27.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/778: creat d4/d1b/d1f/d45/d8a/fe8 x:0 0 0 2024-04-20T10:59:27.094 INFO:tasks.workunit.client.0.smithi027.stdout:9/779: stat d4/d1b/d8f/la6 0 2024-04-20T10:59:27.095 INFO:tasks.workunit.client.0.smithi027.stdout:9/780: truncate d4/d1b/d8f/f90 4800226 0 2024-04-20T10:59:27.095 INFO:tasks.workunit.client.0.smithi027.stdout:9/781: fsync d4/d14/d43/f52 0 2024-04-20T10:59:27.096 INFO:tasks.workunit.client.0.smithi027.stdout:9/782: rmdir d4/d74/dae 39 2024-04-20T10:59:27.096 INFO:tasks.workunit.client.0.smithi027.stdout:9/783: readlink d4/d1b/d1f/d45/l40 0 2024-04-20T10:59:27.096 INFO:tasks.workunit.client.0.smithi027.stdout:9/784: read - d4/d1b/d31/dd0/dda/fe6 zero size 2024-04-20T10:59:27.097 INFO:tasks.workunit.client.0.smithi027.stdout:9/785: creat d4/d14/d43/d91/fe9 x:0 0 0 2024-04-20T10:59:27.098 INFO:tasks.workunit.client.0.smithi027.stdout:9/786: chown d4/f88 22 1 2024-04-20T10:59:27.113 INFO:tasks.workunit.client.0.smithi027.stdout:8/954: dwrite d9/de/d1e/d4a/d6d/d75/f83 [4194304,4194304] 0 2024-04-20T10:59:27.114 INFO:tasks.workunit.client.0.smithi027.stdout:9/787: dwrite d4/d1b/d1f/d45/d8a/fd9 [0,4194304] 0 2024-04-20T10:59:27.116 INFO:tasks.workunit.client.0.smithi027.stdout:7/896: dwrite d4/d5/f33 [0,4194304] 0 2024-04-20T10:59:27.116 INFO:tasks.workunit.client.0.smithi027.stdout:3/958: dwrite d2/fe [4194304,4194304] 0 2024-04-20T10:59:27.117 INFO:tasks.workunit.client.0.smithi027.stdout:9/788: rmdir d4/d1b/d1f/d45/ddb 0 2024-04-20T10:59:27.118 INFO:tasks.workunit.client.0.smithi027.stdout:9/789: truncate d4/d14/f44 4451400 0 2024-04-20T10:59:27.137 INFO:tasks.workunit.client.0.smithi027.stdout:8/955: dwrite d9/de/d1e/d4a/d6d/dea/db3/f101 [0,4194304] 0 2024-04-20T10:59:27.137 INFO:tasks.workunit.client.0.smithi027.stdout:8/956: write d9/de/d20/d53/f77 [2205969,13282] 0 2024-04-20T10:59:27.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/959: dwrite d2/dd/d22/fae [0,4194304] 0 2024-04-20T10:59:27.139 INFO:tasks.workunit.client.0.smithi027.stdout:8/957: rename d9/de/da3/d3c to d9/de/d1e/d4a/d6d/d78/d145 0 2024-04-20T10:59:27.140 INFO:tasks.workunit.client.0.smithi027.stdout:8/958: mknod d9/de/d1e/d4a/d6d/dea/d62/c146 0 2024-04-20T10:59:27.141 INFO:tasks.workunit.client.0.smithi027.stdout:8/959: mknod d9/de/d20/dfd/c147 0 2024-04-20T10:59:27.144 INFO:tasks.workunit.client.0.smithi027.stdout:9/790: dwrite d4/d1b/d31/d23/d29/d7f/fad [0,4194304] 0 2024-04-20T10:59:27.155 INFO:tasks.workunit.client.0.smithi027.stdout:3/960: dwrite d2/dd/d57/d8d/d9d/f11d [0,4194304] 0 2024-04-20T10:59:27.158 INFO:tasks.workunit.client.0.smithi027.stdout:3/961: unlink d2/dd/d106/f10f 0 2024-04-20T10:59:27.159 INFO:tasks.workunit.client.0.smithi027.stdout:3/962: mkdir d2/d18/d2c/d4c/d101/d12d 0 2024-04-20T10:59:27.160 INFO:tasks.workunit.client.0.smithi027.stdout:3/963: rename d2/d94/dc4 to d2/dd/d57/d12e 0 2024-04-20T10:59:27.160 INFO:tasks.workunit.client.0.smithi027.stdout:3/964: chown d2/dd/d57/de7/f113 1 1 2024-04-20T10:59:27.160 INFO:tasks.workunit.client.0.smithi027.stdout:3/965: stat d2/d18/d2c/d4c/d67/l84 0 2024-04-20T10:59:27.162 INFO:tasks.workunit.client.0.smithi027.stdout:8/960: dwrite d9/de/d1e/d4a/d6d/db7/d97/fdf [4194304,4194304] 0 2024-04-20T10:59:27.174 INFO:tasks.workunit.client.0.smithi027.stdout:3/966: dwrite d2/f17 [8388608,4194304] 0 2024-04-20T10:59:27.174 INFO:tasks.workunit.client.0.smithi027.stdout:3/967: getdents d2/d18/d2c/d4c/d67 0 2024-04-20T10:59:27.174 INFO:tasks.workunit.client.0.smithi027.stdout:3/968: truncate d2/dd/d22/de2/d11b/f11c 635759 0 2024-04-20T10:59:27.175 INFO:tasks.workunit.client.0.smithi027.stdout:3/969: unlink d2/dd/d57/d8c/f90 0 2024-04-20T10:59:27.175 INFO:tasks.workunit.client.0.smithi027.stdout:3/970: stat d2/dd/d57/d8d/d9d/f11d 0 2024-04-20T10:59:27.175 INFO:tasks.workunit.client.0.smithi027.stdout:3/971: fsync d2/dd/d57/d8c/fb8 0 2024-04-20T10:59:27.176 INFO:tasks.workunit.client.0.smithi027.stdout:3/972: link d2/d80/f108 d2/dd/d22/de2/d11b/f12f 0 2024-04-20T10:59:27.182 INFO:tasks.workunit.client.0.smithi027.stdout:8/961: dwrite d9/de/d1e/d4a/d6d/d79/fed [0,4194304] 0 2024-04-20T10:59:27.319 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:27.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.422+0000 7f2b71f96640 1 Processor -- start 2024-04-20T10:59:27.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.422+0000 7f2b71f96640 1 -- start start 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b71f96640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b6c155950 0x7f2b6c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b71f96640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0cc330 0x7f2b6c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b71f96640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b6c0b5ea0 con 0x7f2b6c0b5a30 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b71f96640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f2b6c15b9c0 con 0x7f2b6c155950 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b71f96640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c15bca0 con 0x7f2b6c0cc330 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b6b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2b6c155950 0x7f2b6c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b6bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2b6c0b5a30 0x7f2b6c0d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52448/0 (socket says 172.21.15.27:52448) 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b6bfff640 1 -- 172.21.15.27:0/2104016312 learned_addr learned my addr 172.21.15.27:0/2104016312 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b63fff640 1 --2- 172.21.15.27:0/2104016312 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0cc330 0x7f2b6c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b6b7fe640 1 -- 172.21.15.27:0/2104016312 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0cc330 msgr2=0x7f2b6c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b6b7fe640 1 --2- 172.21.15.27:0/2104016312 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0cc330 0x7f2b6c0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b6b7fe640 1 -- 172.21.15.27:0/2104016312 >> v1:172.21.15.27:6789/0 conn(0x7f2b6c0b5a30 legacy=0x7f2b6c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b6b7fe640 1 -- 172.21.15.27:0/2104016312 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c15bf80 con 0x7f2b6c155950 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.423+0000 7f2b6affd640 1 -- 172.21.15.27:0/2104016312 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 701290355 0 0) 0x7f2b6c0b5ea0 con 0x7f2b6c0b5a30 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.424+0000 7f2b6b7fe640 1 --2- 172.21.15.27:0/2104016312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c155950 0x7f2b6c0cbdf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0097a0 tx=0x7f2b5c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c24ad64276d26e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.424+0000 7f2b6affd640 1 -- 172.21.15.27:0/2104016312 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c034070 con 0x7f2b6c155950 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.424+0000 7f2b6affd640 1 -- 172.21.15.27:0/2104016312 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5c030610 con 0x7f2b6c155950 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.424+0000 7f2b6affd640 1 -- 172.21.15.27:0/2104016312 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c0309e0 con 0x7f2b6c155950 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.425+0000 7f2b71f96640 1 -- 172.21.15.27:0/2104016312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c155950 msgr2=0x7f2b6c0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.425+0000 7f2b71f96640 1 --2- 172.21.15.27:0/2104016312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c155950 0x7f2b6c0cbdf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0097a0 tx=0x7f2b5c02b960 comp rx=0 tx=0).stop 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.425+0000 7f2b71f96640 1 -- 172.21.15.27:0/2104016312 shutdown_connections 2024-04-20T10:59:27.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.425+0000 7f2b71f96640 1 --2- 172.21.15.27:0/2104016312 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0cc330 0x7f2b6c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.425+0000 7f2b71f96640 1 --2- 172.21.15.27:0/2104016312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c155950 0x7f2b6c0cbdf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.425+0000 7f2b71f96640 1 -- 172.21.15.27:0/2104016312 >> 172.21.15.27:0/2104016312 conn(0x7f2b6c0676c0 msgr2=0x7f2b6c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:27.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.425+0000 7f2b71f96640 1 -- 172.21.15.27:0/2104016312 shutdown_connections 2024-04-20T10:59:27.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.425+0000 7f2b71f96640 1 -- 172.21.15.27:0/2104016312 wait complete. 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.426+0000 7f2b71f96640 1 Processor -- start 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.426+0000 7f2b71f96640 1 -- start start 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.426+0000 7f2b71f96640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0b5a30 0x7f2b6c150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.426+0000 7f2b71f96640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c0cc330 0x7f2b6c1e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b71f96640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b6c155950 0x7f2b6c1f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b71f96640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c15fb90 con 0x7f2b6c0cc330 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b71f96640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c15c540 con 0x7f2b6c155950 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b71f96640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c15fd10 con 0x7f2b6c0b5a30 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b63fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c0cc330 0x7f2b6c1e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:27.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b63fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c0cc330 0x7f2b6c1e7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39462/0 (socket says 172.21.15.27:39462) 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b63fff640 1 -- 172.21.15.27:0/330585992 learned_addr learned my addr 172.21.15.27:0/330585992 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b6b7fe640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0b5a30 0x7f2b6c150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b6bfff640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b6c155950 0x7f2b6c1f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b63fff640 1 -- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0b5a30 msgr2=0x7f2b6c150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b63fff640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0b5a30 0x7f2b6c150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b63fff640 1 -- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b6c155950 msgr2=0x7f2b6c1f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b63fff640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b6c155950 0x7f2b6c1f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b63fff640 1 -- 172.21.15.27:0/330585992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c150940 con 0x7f2b6c0cc330 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b6b7fe640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0b5a30 0x7f2b6c150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b63fff640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c0cc330 0x7f2b6c1e7100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400e8a0 tx=0x7f2b5400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b68ff9640 1 -- 172.21.15.27:0/330585992 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54017070 con 0x7f2b6c0cc330 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b68ff9640 1 -- 172.21.15.27:0/330585992 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5400c940 con 0x7f2b6c0cc330 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c150c20 con 0x7f2b6c0cc330 2024-04-20T10:59:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b68ff9640 1 -- 172.21.15.27:0/330585992 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5400cd30 con 0x7f2b6c0cc330 2024-04-20T10:59:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.427+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c1f5100 con 0x7f2b6c0cc330 2024-04-20T10:59:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.428+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6c04fcf0 con 0x7f2b6c0cc330 2024-04-20T10:59:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.429+0000 7f2b68ff9640 1 -- 172.21.15.27:0/330585992 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b5401a070 con 0x7f2b6c0cc330 2024-04-20T10:59:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.429+0000 7f2b68ff9640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b4c06c920 0x7f2b4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.429+0000 7f2b68ff9640 1 -- 172.21.15.27:0/330585992 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f2b5408be80 con 0x7f2b6c0cc330 2024-04-20T10:59:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.429+0000 7f2b6b7fe640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b4c06c920 0x7f2b4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.430+0000 7f2b6b7fe640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b4c06c920 0x7f2b4c06ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c036000 tx=0x7f2b5c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:27.433 INFO:tasks.workunit.client.0.smithi027.stdout:3/973: rmdir d2/d18/d2c/d4c 39 2024-04-20T10:59:27.433 INFO:tasks.workunit.client.0.smithi027.stdout:3/974: chown d2/dd/d57/ffa 7989 1 2024-04-20T10:59:27.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.434+0000 7f2b68ff9640 1 -- 172.21.15.27:0/330585992 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2b5405a180 con 0x7f2b6c0cc330 2024-04-20T10:59:27.434 INFO:tasks.workunit.client.0.smithi027.stdout:8/962: getdents d9/de/d1e/d4a/d6d/dea/d59/d74 0 2024-04-20T10:59:27.435 INFO:tasks.workunit.client.0.smithi027.stdout:3/975: creat d2/dd/df/d33/dfb/f130 x:0 0 0 2024-04-20T10:59:27.439 INFO:tasks.workunit.client.0.smithi027.stdout:3/976: write d2/d18/f29 [2242808,4780] 0 2024-04-20T10:59:27.440 INFO:tasks.workunit.client.0.smithi027.stdout:3/977: creat d2/d18/d2c/f131 x:0 0 0 2024-04-20T10:59:27.440 INFO:tasks.workunit.client.0.smithi027.stdout:3/978: truncate d2/d18/d2c/f131 363810 0 2024-04-20T10:59:27.443 INFO:tasks.workunit.client.0.smithi027.stdout:3/979: mkdir d2/dd/df/d132 0 2024-04-20T10:59:27.444 INFO:tasks.workunit.client.0.smithi027.stdout:3/980: creat d2/d18/d2c/d4c/d10b/f133 x:0 0 0 2024-04-20T10:59:27.448 INFO:tasks.workunit.client.0.smithi027.stdout:2/865: sync 2024-04-20T10:59:27.448 INFO:tasks.workunit.client.0.smithi027.stdout:6/940: sync 2024-04-20T10:59:27.449 INFO:tasks.workunit.client.0.smithi027.stdout:8/963: write d9/de/d1e/d4a/d6d/db7/d5b/fa8 [427623,55535] 0 2024-04-20T10:59:27.450 INFO:tasks.workunit.client.0.smithi027.stdout:6/941: mkdir da/d21/dab/d19/d139 0 2024-04-20T10:59:27.451 INFO:tasks.workunit.client.0.smithi027.stdout:8/964: rmdir d9/de/d1e/d4a/d6d/dea/d62 39 2024-04-20T10:59:27.453 INFO:tasks.workunit.client.0.smithi027.stdout:6/942: rename da/d21/dab/f4d to da/d21/dab/d1f/dbf/f13a 0 2024-04-20T10:59:27.453 INFO:tasks.workunit.client.0.smithi027.stdout:6/943: readlink da/db8/lc5 0 2024-04-20T10:59:27.454 INFO:tasks.workunit.client.0.smithi027.stdout:6/944: chown da/d21/d28/d54/lb6 1130905 1 2024-04-20T10:59:27.454 INFO:tasks.workunit.client.0.smithi027.stdout:8/965: truncate d9/f15 520785 0 2024-04-20T10:59:27.455 INFO:tasks.workunit.client.0.smithi027.stdout:6/945: rmdir da/d21/d28/d2d/d53 39 2024-04-20T10:59:27.455 INFO:tasks.workunit.client.0.smithi027.stdout:6/946: chown da/ffa 237264269 1 2024-04-20T10:59:27.455 INFO:tasks.workunit.client.0.smithi027.stdout:6/947: fsync da/dd/d3c/fa8 0 2024-04-20T10:59:27.457 INFO:tasks.workunit.client.0.smithi027.stdout:6/948: rename da/ffa to da/db8/f13b 0 2024-04-20T10:59:27.458 INFO:tasks.workunit.client.0.smithi027.stdout:6/949: creat da/dcc/d108/f13c x:0 0 0 2024-04-20T10:59:27.460 INFO:tasks.workunit.client.0.smithi027.stdout:6/950: creat da/d21/d28/d2d/d53/dae/f13d x:0 0 0 2024-04-20T10:59:27.460 INFO:tasks.workunit.client.0.smithi027.stdout:6/951: write da/dcc/d108/d6b/f62 [1070278,76136] 0 2024-04-20T10:59:27.462 INFO:tasks.workunit.client.0.smithi027.stdout:6/952: unlink da/d21/dab/c23 0 2024-04-20T10:59:27.464 INFO:tasks.workunit.client.0.smithi027.stdout:6/953: creat da/d21/d28/d2d/d53/dae/ddf/f13e x:0 0 0 2024-04-20T10:59:27.464 INFO:tasks.workunit.client.0.smithi027.stdout:2/866: dread d5/f17 [4194304,4194304] 0 2024-04-20T10:59:27.465 INFO:tasks.workunit.client.0.smithi027.stdout:3/981: dwrite d2/dd/d57/d8d/dc6/de1/fe8 [0,4194304] 0 2024-04-20T10:59:27.470 INFO:tasks.workunit.client.0.smithi027.stdout:6/954: dread da/d21/d28/fbb [0,4194304] 0 2024-04-20T10:59:27.472 INFO:tasks.workunit.client.0.smithi027.stdout:6/955: truncate da/dd/f25 239272 0 2024-04-20T10:59:27.473 INFO:tasks.workunit.client.0.smithi027.stdout:3/982: dread d2/d94/f95 [0,4194304] 0 2024-04-20T10:59:27.474 INFO:tasks.workunit.client.0.smithi027.stdout:2/867: dread d5/d44/f65 [0,4194304] 0 2024-04-20T10:59:27.474 INFO:tasks.workunit.client.0.smithi027.stdout:2/868: fdatasync d5/d26/d30/f100 0 2024-04-20T10:59:27.474 INFO:tasks.workunit.client.0.smithi027.stdout:2/869: truncate d5/dc/d16/d2a/d59/da4/de1/dcc/faa 4163155 0 2024-04-20T10:59:27.475 INFO:tasks.workunit.client.0.smithi027.stdout:6/956: mknod da/dcc/dff/c13f 0 2024-04-20T10:59:27.476 INFO:tasks.workunit.client.0.smithi027.stdout:6/957: readlink da/d21/d28/d2d/d53/laa 0 2024-04-20T10:59:27.476 INFO:tasks.workunit.client.0.smithi027.stdout:6/958: dread - da/d21/dab/d1f/dbf/fc4 zero size 2024-04-20T10:59:27.477 INFO:tasks.workunit.client.0.smithi027.stdout:3/983: rename d2/dd/df/d23/df0 to d2/d18/d2c/d4c/d67/d6c/dd0/dd5/d134 0 2024-04-20T10:59:27.477 INFO:tasks.workunit.client.0.smithi027.stdout:3/984: getdents d2/dd/d57/d12e/d104 0 2024-04-20T10:59:27.477 INFO:tasks.workunit.client.0.smithi027.stdout:6/959: stat da/d21/dab/d19/c78 0 2024-04-20T10:59:27.477 INFO:tasks.workunit.client.0.smithi027.stdout:6/960: read da/d21/dab/f10 [1867805,92970] 0 2024-04-20T10:59:27.477 INFO:tasks.workunit.client.0.smithi027.stdout:6/961: readlink da/dcc/d108/d32/l7a 0 2024-04-20T10:59:27.478 INFO:tasks.workunit.client.0.smithi027.stdout:6/962: mknod da/d21/dab/d19/d139/c140 0 2024-04-20T10:59:27.478 INFO:tasks.workunit.client.0.smithi027.stdout:6/963: write da/f2a [4723346,114085] 0 2024-04-20T10:59:27.479 INFO:tasks.workunit.client.0.smithi027.stdout:6/964: dread da/d21/dab/f29 [0,4194304] 0 2024-04-20T10:59:27.480 INFO:tasks.workunit.client.0.smithi027.stdout:6/965: getdents da/d21/d96 0 2024-04-20T10:59:27.481 INFO:tasks.workunit.client.0.smithi027.stdout:6/966: creat da/d21/dab/f141 x:0 0 0 2024-04-20T10:59:27.481 INFO:tasks.workunit.client.0.smithi027.stdout:6/967: chown da/d21/dab/d7e/d86/da3/fac 19625 1 2024-04-20T10:59:27.482 INFO:tasks.workunit.client.0.smithi027.stdout:2/870: dread d5/dc/d16/d37/de2/dac/f67 [0,4194304] 0 2024-04-20T10:59:27.482 INFO:tasks.workunit.client.0.smithi027.stdout:2/871: fdatasync d5/d26/d47/fea 0 2024-04-20T10:59:27.482 INFO:tasks.workunit.client.0.smithi027.stdout:6/968: rename da/dd/d3c/d9e to da/dcc/d131/d142 0 2024-04-20T10:59:27.483 INFO:tasks.workunit.client.0.smithi027.stdout:6/969: readlink da/dcc/d108/d32/l7a 0 2024-04-20T10:59:27.483 INFO:tasks.workunit.client.0.smithi027.stdout:2/872: getdents d5/dc/d16/d32 0 2024-04-20T10:59:27.483 INFO:tasks.workunit.client.0.smithi027.stdout:6/970: mkdir da/dcc/d108/d32/d143 0 2024-04-20T10:59:27.484 INFO:tasks.workunit.client.0.smithi027.stdout:6/971: fdatasync da/d21/d96/fa4 0 2024-04-20T10:59:27.498 INFO:tasks.workunit.client.0.smithi027.stdout:3/985: dwrite d2/d18/d2c/f78 [0,4194304] 0 2024-04-20T10:59:27.499 INFO:tasks.workunit.client.0.smithi027.stdout:2/873: dwrite d5/dc/d16/d37/f3f [4194304,4194304] 0 2024-04-20T10:59:27.499 INFO:tasks.workunit.client.0.smithi027.stdout:3/986: mknod d2/dd/df/d33/c135 0 2024-04-20T10:59:27.500 INFO:tasks.workunit.client.0.smithi027.stdout:2/874: mknod d5/dc/d16/d37/de2/dac/c118 0 2024-04-20T10:59:27.501 INFO:tasks.workunit.client.0.smithi027.stdout:2/875: symlink d5/dc/d16/d32/l119 0 2024-04-20T10:59:27.502 INFO:tasks.workunit.client.0.smithi027.stdout:2/876: mknod d5/d26/c11a 0 2024-04-20T10:59:27.504 INFO:tasks.workunit.client.0.smithi027.stdout:2/877: link d5/l110 d5/dc/d16/d2a/d59/d66/l11b 0 2024-04-20T10:59:27.505 INFO:tasks.workunit.client.0.smithi027.stdout:2/878: creat d5/d26/d30/f11c x:0 0 0 2024-04-20T10:59:27.506 INFO:tasks.workunit.client.0.smithi027.stdout:2/879: creat d5/dc/d16/dd0/dd5/f11d x:0 0 0 2024-04-20T10:59:27.516 INFO:tasks.workunit.client.0.smithi027.stdout:3/987: dwrite d2/fcb [0,4194304] 0 2024-04-20T10:59:27.520 INFO:tasks.workunit.client.0.smithi027.stdout:2/880: dwrite d5/dc/fff [0,4194304] 0 2024-04-20T10:59:27.523 INFO:tasks.workunit.client.0.smithi027.stdout:2/881: creat d5/dc/d16/f11e x:0 0 0 2024-04-20T10:59:27.536 INFO:tasks.workunit.client.0.smithi027.stdout:6/972: dwrite da/d21/f128 [0,4194304] 0 2024-04-20T10:59:27.538 INFO:tasks.workunit.client.0.smithi027.stdout:6/973: getdents da/dd/d79 0 2024-04-20T10:59:27.539 INFO:tasks.workunit.client.0.smithi027.stdout:6/974: mknod da/d21/d28/d2d/d53/dae/ddf/c144 0 2024-04-20T10:59:27.540 INFO:tasks.workunit.client.0.smithi027.stdout:6/975: mkdir da/d21/d28/d145 0 2024-04-20T10:59:27.541 INFO:tasks.workunit.client.0.smithi027.stdout:6/976: rename da/dd/d3c/f126 to da/d21/dab/d7e/daf/f146 0 2024-04-20T10:59:27.541 INFO:tasks.workunit.client.0.smithi027.stdout:3/988: dwrite d2/d18/f7b [0,4194304] 0 2024-04-20T10:59:27.542 INFO:tasks.workunit.client.0.smithi027.stdout:6/977: creat da/dcc/d131/f147 x:0 0 0 2024-04-20T10:59:27.544 INFO:tasks.workunit.client.0.smithi027.stdout:6/978: creat da/d21/dab/d7e/daf/f148 x:0 0 0 2024-04-20T10:59:27.549 INFO:tasks.workunit.client.0.smithi027.stdout:6/979: unlink da/d21/dab/d19/ce6 0 2024-04-20T10:59:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:4/871: sync 2024-04-20T10:59:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/791: sync 2024-04-20T10:59:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:5/858: sync 2024-04-20T10:59:27.550 INFO:tasks.workunit.client.0.smithi027.stdout:7/897: sync 2024-04-20T10:59:27.552 INFO:tasks.workunit.client.0.smithi027.stdout:7/898: link d4/d5/d8/c15 d4/d1b/c125 0 2024-04-20T10:59:27.552 INFO:tasks.workunit.client.0.smithi027.stdout:7/899: write d4/d1b/d4a/d7a/f10a [648671,54453] 0 2024-04-20T10:59:27.552 INFO:tasks.workunit.client.0.smithi027.stdout:9/792: getdents d4/d1b/d31/d23/d29/d7f 0 2024-04-20T10:59:27.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/793: creat d4/d74/dae/fea x:0 0 0 2024-04-20T10:59:27.554 INFO:tasks.workunit.client.0.smithi027.stdout:9/794: mknod d4/d74/ceb 0 2024-04-20T10:59:27.565 INFO:tasks.workunit.client.0.smithi027.stdout:5/859: write d19/d1b/d1c/d1e/d58/f5e [2429976,101994] 0 2024-04-20T10:59:27.566 INFO:tasks.workunit.client.0.smithi027.stdout:5/860: mkdir d19/d1b/d1c/d1e/d22/d68/d75/de9/d11c 0 2024-04-20T10:59:27.569 INFO:tasks.workunit.client.0.smithi027.stdout:6/980: dwrite da/d21/dab/fc7 [0,4194304] 0 2024-04-20T10:59:27.569 INFO:tasks.workunit.client.0.smithi027.stdout:6/981: truncate da/d21/d28/d2d/d53/dae/f13d 874115 0 2024-04-20T10:59:27.569 INFO:tasks.workunit.client.0.smithi027.stdout:4/872: dwrite d0/d14/d6b/da3/d18/db9/fd6 [0,4194304] 0 2024-04-20T10:59:27.570 INFO:tasks.workunit.client.0.smithi027.stdout:4/873: fdatasync d0/d5e/d6c/fb2 0 2024-04-20T10:59:27.570 INFO:tasks.workunit.client.0.smithi027.stdout:7/900: fsync d4/d1b/d4a/d7a/f10a 0 2024-04-20T10:59:27.571 INFO:tasks.workunit.client.0.smithi027.stdout:6/982: link da/dd/d3d/ce9 da/d21/d28/d54/c149 0 2024-04-20T10:59:27.571 INFO:tasks.workunit.client.0.smithi027.stdout:6/983: chown da/d21/f66 64 1 2024-04-20T10:59:27.571 INFO:tasks.workunit.client.0.smithi027.stdout:6/984: chown da/l5a 2 1 2024-04-20T10:59:27.574 INFO:tasks.workunit.client.0.smithi027.stdout:6/985: symlink da/dcc/d131/d142/l14a 0 2024-04-20T10:59:27.576 INFO:tasks.workunit.client.0.smithi027.stdout:7/901: dread d4/d1b/fb0 [0,4194304] 0 2024-04-20T10:59:27.578 INFO:tasks.workunit.client.0.smithi027.stdout:7/902: creat d4/d5/d8/d21/d94/da7/dc2/f126 x:0 0 0 2024-04-20T10:59:27.579 INFO:tasks.workunit.client.0.smithi027.stdout:7/903: fsync d4/d4d/fbc 0 2024-04-20T10:59:27.579 INFO:tasks.workunit.client.0.smithi027.stdout:7/904: write d4/d1b/d4a/f9c [650100,30589] 0 2024-04-20T10:59:27.581 INFO:tasks.workunit.client.0.smithi027.stdout:6/986: dread da/dcc/d108/d32/f75 [0,4194304] 0 2024-04-20T10:59:27.583 INFO:tasks.workunit.client.0.smithi027.stdout:6/987: rename da/dd/d3d/db9 to da/d21/d28/d2d/d53/d9a/d14b 0 2024-04-20T10:59:27.583 INFO:tasks.workunit.client.0.smithi027.stdout:6/988: write da/d21/d28/d2d/d53/f5e [26704,47946] 0 2024-04-20T10:59:27.584 INFO:tasks.workunit.client.0.smithi027.stdout:6/989: link da/d21/d28/d2d/d53/d9a/d14b/d127/l117 da/dcc/d108/d6b/d10d/l14c 0 2024-04-20T10:59:27.584 INFO:tasks.workunit.client.0.smithi027.stdout:6/990: chown da/dcc/d108/d6b/c48 82815166 1 2024-04-20T10:59:27.584 INFO:tasks.workunit.client.0.smithi027.stdout:6/991: read - da/d21/dab/d19/ffc zero size 2024-04-20T10:59:27.587 INFO:tasks.workunit.client.0.smithi027.stdout:5/861: dwrite d19/d1b/d1c/ff1 [0,4194304] 0 2024-04-20T10:59:27.588 INFO:tasks.workunit.client.0.smithi027.stdout:5/862: creat d19/d1b/d1c/d1e/da3/db9/f11d x:0 0 0 2024-04-20T10:59:27.593 INFO:tasks.workunit.client.0.smithi027.stdout:5/863: dread d19/f4d [0,4194304] 0 2024-04-20T10:59:27.594 INFO:tasks.workunit.client.0.smithi027.stdout:5/864: symlink d19/d1b/d1c/d1e/d22/d68/l11e 0 2024-04-20T10:59:27.595 INFO:tasks.workunit.client.0.smithi027.stdout:5/865: symlink d19/d1b/d1c/d1e/d116/l11f 0 2024-04-20T10:59:27.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/866: rename d19/d1b/d1c/d1e/d25/d40/d82/fbc to d19/d1b/d1c/d1e/d83/f120 0 2024-04-20T10:59:27.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/867: chown d19/d1b/d1c/d1e/d22 4208 1 2024-04-20T10:59:27.597 INFO:tasks.workunit.client.0.smithi027.stdout:5/868: rmdir d19/d1b/d1c/d1e/d22/d68/d75/de9/d11c 0 2024-04-20T10:59:27.598 INFO:tasks.workunit.client.0.smithi027.stdout:6/992: dwrite da/d21/dab/f11 [0,4194304] 0 2024-04-20T10:59:27.599 INFO:tasks.workunit.client.0.smithi027.stdout:5/869: creat d19/d1b/d1c/d1e/d3c/f121 x:0 0 0 2024-04-20T10:59:27.601 INFO:tasks.workunit.client.0.smithi027.stdout:5/870: rename d19/d1b/d1c/d1e/d22/db1/dcf/c106 to d19/d1b/d1c/d1e/d3c/c122 0 2024-04-20T10:59:27.602 INFO:tasks.workunit.client.0.smithi027.stdout:5/871: creat d19/d1b/d1c/d1e/d22/db1/dcf/d91/f123 x:0 0 0 2024-04-20T10:59:27.605 INFO:tasks.workunit.client.0.smithi027.stdout:6/993: read da/d21/dab/d19/f74 [100509,11306] 0 2024-04-20T10:59:27.606 INFO:tasks.workunit.client.0.smithi027.stdout:6/994: readlink da/dd/d3d/l40 0 2024-04-20T10:59:27.606 INFO:tasks.workunit.client.0.smithi027.stdout:6/995: dread da/d21/d28/d54/fde [0,4194304] 0 2024-04-20T10:59:27.606 INFO:tasks.workunit.client.0.smithi027.stdout:6/996: write da/f110 [414124,51497] 0 2024-04-20T10:59:27.606 INFO:tasks.workunit.client.0.smithi027.stdout:6/997: dread - da/d21/d28/d2d/d53/d9a/d14b/d127/fbc zero size 2024-04-20T10:59:27.607 INFO:tasks.workunit.client.0.smithi027.stdout:6/998: mkdir da/d21/dab/d1f/d14d 0 2024-04-20T10:59:27.609 INFO:tasks.workunit.client.0.smithi027.stdout:6/999: creat da/dcc/d108/d6b/d10d/d11e/f14e x:0 0 0 2024-04-20T10:59:27.612 INFO:tasks.workunit.client.0.smithi027.stdout:5/872: getdents d19/d1b/d1c/d1e/da3/db9 0 2024-04-20T10:59:27.612 INFO:tasks.workunit.client.0.smithi027.stdout:5/873: chown d19/d1b/d1c/d1e/d25/d40/fd5 29874 1 2024-04-20T10:59:27.614 INFO:tasks.workunit.client.0.smithi027.stdout:3/989: dread d2/d18/d2c/d4c/d67/d6c/dd0/dd5/d134/ff9 [0,4194304] 0 2024-04-20T10:59:27.614 INFO:tasks.workunit.client.0.smithi027.stdout:3/990: write d2/dd/d57/de7/dff/f10c [591015,46885] 0 2024-04-20T10:59:27.615 INFO:tasks.workunit.client.0.smithi027.stdout:3/991: creat d2/dd/d57/de7/dff/d100/f136 x:0 0 0 2024-04-20T10:59:27.620 INFO:tasks.workunit.client.0.smithi027.stdout:5/874: dread d19/d1b/f86 [0,4194304] 0 2024-04-20T10:59:27.621 INFO:tasks.workunit.client.0.smithi027.stdout:5/875: write d19/d1b/d1c/d1e/d25/f2f [841247,123794] 0 2024-04-20T10:59:27.622 INFO:tasks.workunit.client.0.smithi027.stdout:3/992: dread d2/dd/df/d23/f50 [0,4194304] 0 2024-04-20T10:59:27.623 INFO:tasks.workunit.client.0.smithi027.stdout:5/876: unlink d19/d1b/d1c/d1e/d22/db1/dcf/lb2 0 2024-04-20T10:59:27.624 INFO:tasks.workunit.client.0.smithi027.stdout:3/993: mkdir d2/d80/d137 0 2024-04-20T10:59:27.626 INFO:tasks.workunit.client.0.smithi027.stdout:5/877: unlink d19/d1b/c53 0 2024-04-20T10:59:27.626 INFO:tasks.workunit.client.0.smithi027.stdout:5/878: dread - d19/d1b/d1c/d1e/d22/ff5 zero size 2024-04-20T10:59:27.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/994: mknod d2/d80/c138 0 2024-04-20T10:59:27.627 INFO:tasks.workunit.client.0.smithi027.stdout:5/879: creat d19/d1b/d1c/d1e/d22/db1/f124 x:0 0 0 2024-04-20T10:59:27.628 INFO:tasks.workunit.client.0.smithi027.stdout:3/995: link d2/dd/df/lc1 d2/d18/d2c/d4c/d67/d6c/dd0/l139 0 2024-04-20T10:59:27.628 INFO:tasks.workunit.client.0.smithi027.stdout:5/880: rmdir d19/d1b/d1c/dbd/dbe 39 2024-04-20T10:59:27.629 INFO:tasks.workunit.client.0.smithi027.stdout:5/881: stat d19/d1b/d1c/d1e/d22/d68/d75/db7/l104 0 2024-04-20T10:59:27.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/996: dwrite d2/d18/d2e/f75 [0,4194304] 0 2024-04-20T10:59:27.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/997: dread - d2/d18/d2c/d4c/d67/d6c/fdc zero size 2024-04-20T10:59:27.647 INFO:tasks.workunit.client.0.smithi027.stdout:3/998: creat d2/dd/d57/de7/dff/d124/f13a x:0 0 0 2024-04-20T10:59:27.665 INFO:tasks.workunit.client.0.smithi027.stdout:3/999: dwrite d2/dd/df/d23/f125 [0,4194304] 0 2024-04-20T10:59:27.670 INFO:tasks.workunit.client.0.smithi027.stdout:8/966: sync 2024-04-20T10:59:27.670 INFO:tasks.workunit.client.0.smithi027.stdout:8/967: getdents d9/de/d1e/d11b 0 2024-04-20T10:59:27.671 INFO:tasks.workunit.client.0.smithi027.stdout:8/968: chown d9/de/d20/dfd 55952392 1 2024-04-20T10:59:27.690 INFO:tasks.workunit.client.0.smithi027.stdout:8/969: dwrite d9/de/d1e/d4a/d6d/dea/d59/ffc [0,4194304] 0 2024-04-20T10:59:27.692 INFO:tasks.workunit.client.0.smithi027.stdout:8/970: rmdir d9/de/d1e/d4a/d6d/d7d/dd6 39 2024-04-20T10:59:27.693 INFO:tasks.workunit.client.0.smithi027.stdout:8/971: mknod d9/de/d1e/d4a/d6d/dea/d59/d74/de8/d118/c148 0 2024-04-20T10:59:27.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.694+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b6c14f110 con 0x7f2b6c0cc330 2024-04-20T10:59:27.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.695+0000 7f2b68ff9640 1 -- 172.21.15.27:0/330585992 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6933 (secure 0 0 0) 0x7f2b54059a80 con 0x7f2b6c0cc330 2024-04-20T10:59:27.695 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:27.695 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5029,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:59:27.695 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-04-20T10:59:27.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b4c06c920 msgr2=0x7f2b4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:27.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b4c06c920 0x7f2b4c06ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c036000 tx=0x7f2b5c03b040 comp rx=0 tx=0).stop 2024-04-20T10:59:27.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c0cc330 msgr2=0x7f2b6c1e7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:27.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c0cc330 0x7f2b6c1e7100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400e8a0 tx=0x7f2b5400eca0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 shutdown_connections 2024-04-20T10:59:27.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2b4c06c920 0x7f2b4c06ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b6c155950 0x7f2b6c1f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b6c0cc330 0x7f2b6c1e7100 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 --2- 172.21.15.27:0/330585992 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2b6c0b5a30 0x7f2b6c150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:27.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 >> 172.21.15.27:0/330585992 conn(0x7f2b6c0676c0 msgr2=0x7f2b6c0b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:27.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 shutdown_connections 2024-04-20T10:59:27.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:27.697+0000 7f2b71f96640 1 -- 172.21.15.27:0/330585992 wait complete. 2024-04-20T10:59:27.701 INFO:tasks.workunit.client.0.smithi027.stdout:8/972: dread d9/de/d1e/d4a/d6d/db7/d97/ff2 [0,4194304] 0 2024-04-20T10:59:27.701 INFO:tasks.workunit.client.0.smithi027.stdout:8/973: readlink d9/de/d1e/lb2 0 2024-04-20T10:59:27.701 INFO:tasks.workunit.client.0.smithi027.stdout:8/974: truncate d9/de/d1e/d107/f11d 897934 0 2024-04-20T10:59:27.707 INFO:tasks.workunit.client.0.smithi027.stdout:8/975: read d9/de/d1e/d4a/d6d/d78/d145/f61 [152063,81362] 0 2024-04-20T10:59:27.708 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T10:59:27.709 INFO:tasks.workunit.client.0.smithi027.stdout:8/976: write d9/de/d1e/d4a/d6d/d78/d145/f9c [174206,119084] 0 2024-04-20T10:59:27.709 INFO:tasks.workunit.client.0.smithi027.stdout:8/977: truncate d9/de/d1e/d11b/dbd/f129 1023640 0 2024-04-20T10:59:27.709 INFO:tasks.workunit.client.0.smithi027.stdout:8/978: symlink d9/de/d1e/d4a/l149 0 2024-04-20T10:59:27.709 INFO:tasks.workunit.client.0.smithi027.stdout:8/979: fsync d9/de/d1e/d4a/d6d/d75/f11e 0 2024-04-20T10:59:27.727 INFO:tasks.workunit.client.0.smithi027.stdout:8/980: dwrite d9/de/d1e/d4a/d6d/d78/d145/f90 [0,4194304] 0 2024-04-20T10:59:27.727 INFO:tasks.workunit.client.0.smithi027.stdout:8/981: write d9/de/d1e/d11b/dbd/f129 [2619,94068] 0 2024-04-20T10:59:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:8/982: link f3 d9/de/d1e/d11b/f14a 0 2024-04-20T10:59:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:8/983: chown d9/de/d64/d6c 3245 1 2024-04-20T10:59:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:8/984: mknod d9/de/d18/c14b 0 2024-04-20T10:59:27.729 INFO:tasks.workunit.client.0.smithi027.stdout:8/985: chown d9/f15 123 1 2024-04-20T10:59:27.730 INFO:tasks.workunit.client.0.smithi027.stdout:8/986: creat d9/de/d1e/d4a/d6d/dea/d62/f14c x:0 0 0 2024-04-20T10:59:27.998 INFO:tasks.workunit.client.0.smithi027.stdout:9/795: sync 2024-04-20T10:59:27.998 INFO:tasks.workunit.client.0.smithi027.stdout:2/882: sync 2024-04-20T10:59:27.999 INFO:tasks.workunit.client.0.smithi027.stdout:2/883: readlink d5/d1e/d21/l49 0 2024-04-20T10:59:27.999 INFO:tasks.workunit.client.0.smithi027.stdout:2/884: truncate d5/d1e/d9d/fe9 641572 0 2024-04-20T10:59:27.999 INFO:tasks.workunit.client.0.smithi027.stdout:2/885: dread d5/d1e/d9d/fe9 [0,4194304] 0 2024-04-20T10:59:27.999 INFO:tasks.workunit.client.0.smithi027.stdout:2/886: write d5/d26/d30/f100 [675783,24470] 0 2024-04-20T10:59:28.003 INFO:tasks.workunit.client.0.smithi027.stdout:2/887: creat d5/d26/d78/f11f x:0 0 0 2024-04-20T10:59:28.010 INFO:tasks.workunit.client.0.smithi027.stdout:2/888: dread d5/f58 [4194304,4194304] 0 2024-04-20T10:59:28.010 INFO:tasks.workunit.client.0.smithi027.stdout:2/889: stat d5/d44/d7c 0 2024-04-20T10:59:28.010 INFO:tasks.workunit.client.0.smithi027.stdout:2/890: creat d5/dbc/f120 x:0 0 0 2024-04-20T10:59:28.011 INFO:tasks.workunit.client.0.smithi027.stdout:2/891: fsync d5/dc/d16/d32/d109/f104 0 2024-04-20T10:59:28.013 INFO:tasks.workunit.client.0.smithi027.stdout:9/796: dwrite d4/d1b/d1f/d45/d8a/f6e [0,4194304] 0 2024-04-20T10:59:28.021 INFO:tasks.workunit.client.0.smithi027.stdout:7/905: sync 2024-04-20T10:59:28.021 INFO:tasks.workunit.client.0.smithi027.stdout:8/987: sync 2024-04-20T10:59:28.021 INFO:tasks.workunit.client.0.smithi027.stdout:4/874: sync 2024-04-20T10:59:28.021 INFO:tasks.workunit.client.0.smithi027.stdout:5/882: sync 2024-04-20T10:59:28.022 INFO:tasks.workunit.client.0.smithi027.stdout:8/988: truncate d9/de/d1e/d4a/d6d/dea/d59/fe1 104414 0 2024-04-20T10:59:28.022 INFO:tasks.workunit.client.0.smithi027.stdout:9/797: dread d4/f6b [0,4194304] 0 2024-04-20T10:59:28.022 INFO:tasks.workunit.client.0.smithi027.stdout:4/875: mknod d0/d48/d123/d2f/c130 0 2024-04-20T10:59:28.023 INFO:tasks.workunit.client.0.smithi027.stdout:7/906: mkdir d4/d5/d8/d127 0 2024-04-20T10:59:28.023 INFO:tasks.workunit.client.0.smithi027.stdout:8/989: getdents d9/de/d1e/d4a/d6d/d7d 0 2024-04-20T10:59:28.024 INFO:tasks.workunit.client.0.smithi027.stdout:9/798: getdents d4/d1b/d31 0 2024-04-20T10:59:28.024 INFO:tasks.workunit.client.0.smithi027.stdout:9/799: fsync d4/d1b/d1f/d45/d8a/fd9 0 2024-04-20T10:59:28.024 INFO:tasks.workunit.client.0.smithi027.stdout:7/907: creat d4/d1b/d4a/d7a/f128 x:0 0 0 2024-04-20T10:59:28.025 INFO:tasks.workunit.client.0.smithi027.stdout:9/800: mkdir d4/d1b/d31/d23/d29/d7f/dec 0 2024-04-20T10:59:28.026 INFO:tasks.workunit.client.0.smithi027.stdout:9/801: creat d4/d1b/d1f/d45/d8a/fed x:0 0 0 2024-04-20T10:59:28.027 INFO:tasks.workunit.client.0.smithi027.stdout:2/892: dwrite d5/dc/d16/d2a/f93 [0,4194304] 0 2024-04-20T10:59:28.027 INFO:tasks.workunit.client.0.smithi027.stdout:5/883: dread d19/d1b/d1c/d1e/f57 [4194304,4194304] 0 2024-04-20T10:59:28.027 INFO:tasks.workunit.client.0.smithi027.stdout:5/884: read - d19/d1b/d1c/d1e/d58/db4/ffe zero size 2024-04-20T10:59:28.028 INFO:tasks.workunit.client.0.smithi027.stdout:9/802: rename d4/d1b/cac to d4/d1b/d1f/d45/cee 0 2024-04-20T10:59:28.029 INFO:tasks.workunit.client.0.smithi027.stdout:2/893: unlink d5/dc/d16/d37/de2/d8e/fb5 0 2024-04-20T10:59:28.029 INFO:tasks.workunit.client.0.smithi027.stdout:8/990: dread d9/de/d1e/d4a/d6d/db7/d5b/d104/f69 [0,4194304] 0 2024-04-20T10:59:28.030 INFO:tasks.workunit.client.0.smithi027.stdout:9/803: creat d4/d1b/d31/d51/fef x:0 0 0 2024-04-20T10:59:28.030 INFO:tasks.workunit.client.0.smithi027.stdout:9/804: fsync d4/d1b/d31/d23/d29/d7f/fad 0 2024-04-20T10:59:28.032 INFO:tasks.workunit.client.0.smithi027.stdout:5/885: read d19/d1b/d1c/d1e/d3c/f74 [1819914,15138] 0 2024-04-20T10:59:28.032 INFO:tasks.workunit.client.0.smithi027.stdout:2/894: mkdir d5/d44/d121 0 2024-04-20T10:59:28.034 INFO:tasks.workunit.client.0.smithi027.stdout:9/805: rename d4/d1b/d31/d51/c5b to d4/d1b/d8f/cf0 0 2024-04-20T10:59:28.035 INFO:tasks.workunit.client.0.smithi027.stdout:2/895: creat d5/dc/d10d/f122 x:0 0 0 2024-04-20T10:59:28.036 INFO:tasks.workunit.client.0.smithi027.stdout:9/806: creat d4/d74/d72/ff1 x:0 0 0 2024-04-20T10:59:28.042 INFO:tasks.workunit.client.0.smithi027.stdout:2/896: write d5/dc/d16/d32/d109/fd8 [2796499,57747] 0 2024-04-20T10:59:28.042 INFO:tasks.workunit.client.0.smithi027.stdout:2/897: dread - d5/d44/ffe zero size 2024-04-20T10:59:28.043 INFO:tasks.workunit.client.0.smithi027.stdout:7/908: dwrite d4/d5/fd3 [0,4194304] 0 2024-04-20T10:59:28.043 INFO:tasks.workunit.client.0.smithi027.stdout:7/909: chown d4/d1b/d4a/d62/d87/f9b 1 1 2024-04-20T10:59:28.051 INFO:tasks.workunit.client.0.smithi027.stdout:7/910: read d4/d1b/d4a/d62/f9f [1523827,130062] 0 2024-04-20T10:59:28.051 INFO:tasks.workunit.client.0.smithi027.stdout:7/911: dread - d4/d1b/ff6 zero size 2024-04-20T10:59:28.051 INFO:tasks.workunit.client.0.smithi027.stdout:7/912: write d4/d5/d8/d46/d77/fd0 [623023,106601] 0 2024-04-20T10:59:28.051 INFO:tasks.workunit.client.0.smithi027.stdout:2/898: dread d5/dc/f1c [0,4194304] 0 2024-04-20T10:59:28.051 INFO:tasks.workunit.client.0.smithi027.stdout:2/899: write d5/dc/d16/f11e [1530,74175] 0 2024-04-20T10:59:28.054 INFO:tasks.workunit.client.0.smithi027.stdout:2/900: dread d5/dc/f1c [0,4194304] 0 2024-04-20T10:59:28.054 INFO:tasks.workunit.client.0.smithi027.stdout:7/913: creat d4/d5/d8/d3b/f129 x:0 0 0 2024-04-20T10:59:28.056 INFO:tasks.workunit.client.0.smithi027.stdout:2/901: symlink d5/d26/l123 0 2024-04-20T10:59:28.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/886: dwrite d19/d1b/d1c/dbd/dbe/f9e [0,4194304] 0 2024-04-20T10:59:28.066 INFO:tasks.workunit.client.0.smithi027.stdout:8/991: dwrite d9/f15 [0,4194304] 0 2024-04-20T10:59:28.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/807: dwrite d4/d14/f27 [0,4194304] 0 2024-04-20T10:59:28.068 INFO:tasks.workunit.client.0.smithi027.stdout:5/887: creat d19/d1b/d1c/d1e/d25/d40/f125 x:0 0 0 2024-04-20T10:59:28.069 INFO:tasks.workunit.client.0.smithi027.stdout:7/914: dread d4/d1b/d4a/d62/fa0 [0,4194304] 0 2024-04-20T10:59:28.071 INFO:tasks.workunit.client.0.smithi027.stdout:9/808: mknod d4/d1b/d31/dd0/cf2 0 2024-04-20T10:59:28.072 INFO:tasks.workunit.client.0.smithi027.stdout:5/888: link d19/d1b/d1c/d1e/d22/c3e d19/d1b/d1c/c126 0 2024-04-20T10:59:28.072 INFO:tasks.workunit.client.0.smithi027.stdout:5/889: chown d19/d1b/d1c/dbd/dbe/cf8 70436 1 2024-04-20T10:59:28.072 INFO:tasks.workunit.client.0.smithi027.stdout:7/915: chown d4/d5b/fe1 3582315 1 2024-04-20T10:59:28.073 INFO:tasks.workunit.client.0.smithi027.stdout:9/809: creat d4/d1b/ff3 x:0 0 0 2024-04-20T10:59:28.073 INFO:tasks.workunit.client.0.smithi027.stdout:9/810: chown d4/d1b/d8f/la6 703 1 2024-04-20T10:59:28.074 INFO:tasks.workunit.client.0.smithi027.stdout:9/811: unlink d4/d14/f21 0 2024-04-20T10:59:28.078 INFO:tasks.workunit.client.0.smithi027.stdout:8/992: dread d9/de/d1e/d4a/d6d/d79/f92 [0,4194304] 0 2024-04-20T10:59:28.079 INFO:tasks.workunit.client.0.smithi027.stdout:8/993: fdatasync d9/ff 0 2024-04-20T10:59:28.079 INFO:tasks.workunit.client.0.smithi027.stdout:2/902: dwrite d5/dc/d16/d32/d109/ff9 [0,4194304] 0 2024-04-20T10:59:28.079 INFO:tasks.workunit.client.0.smithi027.stdout:2/903: write d5/d26/d30/f79 [1327040,112229] 0 2024-04-20T10:59:28.080 INFO:tasks.workunit.client.0.smithi027.stdout:8/994: creat d9/de/d20/f14d x:0 0 0 2024-04-20T10:59:28.084 INFO:tasks.workunit.client.0.smithi027.stdout:8/995: creat d9/de/d64/d6a/f14e x:0 0 0 2024-04-20T10:59:28.085 INFO:tasks.workunit.client.0.smithi027.stdout:9/812: write d4/f1c [10587693,115705] 0 2024-04-20T10:59:28.085 INFO:tasks.workunit.client.0.smithi027.stdout:7/916: write d4/d5/fa [3958574,73557] 0 2024-04-20T10:59:28.086 INFO:tasks.workunit.client.0.smithi027.stdout:9/813: getdents d4/d14/da8 0 2024-04-20T10:59:28.089 INFO:tasks.workunit.client.0.smithi027.stdout:9/814: rename d4/d1b/d1f/d5a/l6f to d4/d1b/d1f/d45/d9e/lf4 0 2024-04-20T10:59:28.091 INFO:tasks.workunit.client.0.smithi027.stdout:7/917: unlink d4/fb4 0 2024-04-20T10:59:28.091 INFO:tasks.workunit.client.0.smithi027.stdout:7/918: write d4/d5/d81/fe8 [79406,74166] 0 2024-04-20T10:59:28.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/815: symlink d4/d1b/d1f/d5a/lf5 0 2024-04-20T10:59:28.092 INFO:tasks.workunit.client.0.smithi027.stdout:9/816: fsync d4/d1b/f22 0 2024-04-20T10:59:28.094 INFO:tasks.workunit.client.0.smithi027.stdout:7/919: creat d4/da2/dd1/dfb/d104/f12a x:0 0 0 2024-04-20T10:59:28.096 INFO:tasks.workunit.client.0.smithi027.stdout:9/817: unlink d4/d14/d43/c95 0 2024-04-20T10:59:28.102 INFO:tasks.workunit.client.0.smithi027.stdout:2/904: dwrite d5/dc/d16/d37/de2/d8e/fc6 [0,4194304] 0 2024-04-20T10:59:28.102 INFO:tasks.workunit.client.0.smithi027.stdout:2/905: readlink d5/d26/l10f 0 2024-04-20T10:59:28.102 INFO:tasks.workunit.client.0.smithi027.stdout:5/890: dwrite d19/d1b/d1c/d1e/d25/fa0 [0,4194304] 0 2024-04-20T10:59:28.112 INFO:tasks.workunit.client.0.smithi027.stdout:2/906: dread d5/d1e/f9c [0,4194304] 0 2024-04-20T10:59:28.116 INFO:tasks.workunit.client.0.smithi027.stdout:8/996: dwrite d9/de/d1e/fcb [0,4194304] 0 2024-04-20T10:59:28.117 INFO:tasks.workunit.client.0.smithi027.stdout:7/920: dwrite d4/da2/fb5 [0,4194304] 0 2024-04-20T10:59:28.117 INFO:tasks.workunit.client.0.smithi027.stdout:9/818: dwrite d4/d1b/d31/f75 [4194304,4194304] 0 2024-04-20T10:59:28.118 INFO:tasks.workunit.client.0.smithi027.stdout:9/819: chown d4/d1b/d31/d23/d29/d7f 26 1 2024-04-20T10:59:28.118 INFO:tasks.workunit.client.0.smithi027.stdout:9/820: write d4/d1b/d1f/d45/d8a/fe8 [1042488,130230] 0 2024-04-20T10:59:28.128 INFO:tasks.workunit.client.0.smithi027.stdout:5/891: symlink d19/d1b/d1c/d1e/d22/d68/d75/db7/l127 0 2024-04-20T10:59:28.129 INFO:tasks.workunit.client.0.smithi027.stdout:7/921: mkdir d4/d5/d6f/d110/d12b 0 2024-04-20T10:59:28.130 INFO:tasks.workunit.client.0.smithi027.stdout:9/821: getdents d4/d14/d43 0 2024-04-20T10:59:28.130 INFO:tasks.workunit.client.0.smithi027.stdout:9/822: stat d4/d14/l37 0 2024-04-20T10:59:28.133 INFO:tasks.workunit.client.0.smithi027.stdout:7/922: rename d4/d1b/d4a/fb3 to d4/d5/d8/d21/d94/da7/f12c 0 2024-04-20T10:59:28.134 INFO:tasks.workunit.client.0.smithi027.stdout:9/823: creat d4/d1b/d50/ff6 x:0 0 0 2024-04-20T10:59:28.137 INFO:tasks.workunit.client.0.smithi027.stdout:9/824: unlink d4/d1b/c6d 0 2024-04-20T10:59:28.137 INFO:tasks.workunit.client.0.smithi027.stdout:9/825: truncate d4/d1b/d8f/fd3 849296 0 2024-04-20T10:59:28.137 INFO:tasks.workunit.client.0.smithi027.stdout:9/826: chown d4/d14/c17 953149215 1 2024-04-20T10:59:28.137 INFO:tasks.workunit.client.0.smithi027.stdout:9/827: write d4/f1c [1475375,34379] 0 2024-04-20T10:59:28.137 INFO:tasks.workunit.client.0.smithi027.stdout:8/997: dwrite d9/de/d1e/d107/f11d [0,4194304] 0 2024-04-20T10:59:28.138 INFO:tasks.workunit.client.0.smithi027.stdout:8/998: readlink d9/de/d1e/d4a/d6d/db7/d5b/lae 0 2024-04-20T10:59:28.139 INFO:tasks.workunit.client.0.smithi027.stdout:7/923: unlink d4/l50 0 2024-04-20T10:59:28.141 INFO:tasks.workunit.client.0.smithi027.stdout:8/999: getdents d9/de/d1e/d49 0 2024-04-20T10:59:28.143 INFO:tasks.workunit.client.0.smithi027.stdout:7/924: creat d4/d5/d8/d127/f12d x:0 0 0 2024-04-20T10:59:28.145 INFO:tasks.workunit.client.0.smithi027.stdout:2/907: dwrite d5/dc/d16/d2a/f93 [0,4194304] 0 2024-04-20T10:59:28.145 INFO:tasks.workunit.client.0.smithi027.stdout:9/828: dread d4/d1b/d31/d23/f47 [0,4194304] 0 2024-04-20T10:59:28.150 INFO:tasks.workunit.client.0.smithi027.stdout:7/925: creat d4/da2/dd1/dfb/d104/d117/d119/f12e x:0 0 0 2024-04-20T10:59:28.151 INFO:tasks.workunit.client.0.smithi027.stdout:9/829: unlink d4/d74/d72/f73 0 2024-04-20T10:59:28.151 INFO:tasks.workunit.client.0.smithi027.stdout:9/830: readlink d4/d1b/d31/l8b 0 2024-04-20T10:59:28.152 INFO:tasks.workunit.client.0.smithi027.stdout:9/831: dread - d4/d1b/d31/dd0/dda/fe6 zero size 2024-04-20T10:59:28.157 INFO:tasks.workunit.client.0.smithi027.stdout:4/876: sync 2024-04-20T10:59:28.157 INFO:tasks.workunit.client.0.smithi027.stdout:4/877: write d0/d14/d6b/da3/fe5 [1067737,81889] 0 2024-04-20T10:59:28.157 INFO:tasks.workunit.client.0.smithi027.stdout:4/878: dread - d0/d14/d6b/da3/d53/d6a/da1/f118 zero size 2024-04-20T10:59:28.162 INFO:tasks.workunit.client.0.smithi027.stdout:2/908: creat d5/dc/d16/f124 x:0 0 0 2024-04-20T10:59:28.162 INFO:tasks.workunit.client.0.smithi027.stdout:7/926: dread d4/d1b/fb6 [0,4194304] 0 2024-04-20T10:59:28.162 INFO:tasks.workunit.client.0.smithi027.stdout:7/927: write d4/d5/d8/d45/ff7 [514823,88601] 0 2024-04-20T10:59:28.180 INFO:tasks.workunit.client.0.smithi027.stdout:9/832: dwrite d4/fc8 [0,4194304] 0 2024-04-20T10:59:28.180 INFO:tasks.workunit.client.0.smithi027.stdout:2/909: dwrite d5/d44/f106 [0,4194304] 0 2024-04-20T10:59:28.180 INFO:tasks.workunit.client.0.smithi027.stdout:2/910: chown d5/d26/d47 494613 1 2024-04-20T10:59:28.180 INFO:tasks.workunit.client.0.smithi027.stdout:4/879: rmdir d0/d14 39 2024-04-20T10:59:28.181 INFO:tasks.workunit.client.0.smithi027.stdout:4/880: truncate d0/d14/fc4 4360844 0 2024-04-20T10:59:28.181 INFO:tasks.workunit.client.0.smithi027.stdout:7/928: mknod d4/d1b/d4a/d62/d87/c12f 0 2024-04-20T10:59:28.182 INFO:tasks.workunit.client.0.smithi027.stdout:7/929: truncate d4/d1b/ff 4989230 0 2024-04-20T10:59:28.183 INFO:tasks.workunit.client.0.smithi027.stdout:9/833: rmdir d4/d1b/d50/d5d 39 2024-04-20T10:59:28.184 INFO:tasks.workunit.client.0.smithi027.stdout:7/930: dread d4/d5/d8/d46/d106/f109 [0,4194304] 0 2024-04-20T10:59:28.184 INFO:tasks.workunit.client.0.smithi027.stdout:4/881: symlink d0/d48/d123/d33/l131 0 2024-04-20T10:59:28.184 INFO:tasks.workunit.client.0.smithi027.stdout:4/882: read d0/d14/d6b/f125 [3228554,9246] 0 2024-04-20T10:59:28.185 INFO:tasks.workunit.client.0.smithi027.stdout:9/834: creat d4/d14/da8/ff7 x:0 0 0 2024-04-20T10:59:28.186 INFO:tasks.workunit.client.0.smithi027.stdout:7/931: truncate d4/d5/d8/d46/d77/fcf 1752888 0 2024-04-20T10:59:28.218 INFO:tasks.workunit.client.0.smithi027.stdout:4/883: mkdir d0/d48/d68/d132 0 2024-04-20T10:59:28.234 INFO:tasks.workunit.client.0.smithi027.stdout:4/884: mknod d0/d48/d123/d33/d5c/d6f/c133 0 2024-04-20T10:59:28.234 INFO:tasks.workunit.client.0.smithi027.stdout:4/885: readlink d0/d48/d123/lcc 0 2024-04-20T10:59:28.250 INFO:tasks.workunit.client.0.smithi027.stdout:4/886: dwrite d0/d14/d6b/f125 [0,4194304] 0 2024-04-20T10:59:28.250 INFO:tasks.workunit.client.0.smithi027.stdout:4/887: truncate d0/d14/d6b/da3/d53/d6a/da1/f118 154498 0 2024-04-20T10:59:28.251 INFO:tasks.workunit.client.0.smithi027.stdout:9/835: link d4/d1b/d1f/d45/l40 d4/lf8 0 2024-04-20T10:59:28.254 INFO:tasks.workunit.client.0.smithi027.stdout:2/911: sync 2024-04-20T10:59:28.254 INFO:tasks.workunit.client.0.smithi027.stdout:7/932: sync 2024-04-20T10:59:28.254 INFO:tasks.workunit.client.0.smithi027.stdout:5/892: sync 2024-04-20T10:59:28.255 INFO:tasks.workunit.client.0.smithi027.stdout:4/888: creat d0/d14/d6b/da3/d53/d6a/da1/de1/f134 x:0 0 0 2024-04-20T10:59:28.256 INFO:tasks.workunit.client.0.smithi027.stdout:9/836: rename d4/lf to d4/d1b/d31/d23/d29/d7f/dec/lf9 0 2024-04-20T10:59:28.256 INFO:tasks.workunit.client.0.smithi027.stdout:7/933: rmdir d4/d5/d8/d21 39 2024-04-20T10:59:28.257 INFO:tasks.workunit.client.0.smithi027.stdout:2/912: link d5/d26/d47/d8a/ldd d5/dc/d16/db1/l125 0 2024-04-20T10:59:28.258 INFO:tasks.workunit.client.0.smithi027.stdout:2/913: chown d5/d1e/d21/f33 249 1 2024-04-20T10:59:28.258 INFO:tasks.workunit.client.0.smithi027.stdout:2/914: fdatasync d5/f17 0 2024-04-20T10:59:28.258 INFO:tasks.workunit.client.0.smithi027.stdout:4/889: creat d0/d48/d123/d33/d5c/f135 x:0 0 0 2024-04-20T10:59:28.258 INFO:tasks.workunit.client.0.smithi027.stdout:4/890: dread - d0/d48/d123/d33/d5c/d6f/fa9 zero size 2024-04-20T10:59:28.259 INFO:tasks.workunit.client.0.smithi027.stdout:4/891: readlink d0/d14/d6b/d92/dcd/led 0 2024-04-20T10:59:28.259 INFO:tasks.workunit.client.0.smithi027.stdout:5/893: dread d19/d1b/d1c/d1e/d58/f7b [0,4194304] 0 2024-04-20T10:59:28.260 INFO:tasks.workunit.client.0.smithi027.stdout:9/837: mknod d4/d1b/d1f/d45/d8a/db2/cfa 0 2024-04-20T10:59:28.260 INFO:tasks.workunit.client.0.smithi027.stdout:9/838: readlink d4/d1b/d31/dd0/le7 0 2024-04-20T10:59:28.260 INFO:tasks.workunit.client.0.smithi027.stdout:9/839: read - d4/d1b/d50/fe0 zero size 2024-04-20T10:59:28.260 INFO:tasks.workunit.client.0.smithi027.stdout:9/840: fdatasync d4/d1b/d31/d23/d29/f82 0 2024-04-20T10:59:28.260 INFO:tasks.workunit.client.0.smithi027.stdout:9/841: dread - d4/d14/d43/fc7 zero size 2024-04-20T10:59:28.261 INFO:tasks.workunit.client.0.smithi027.stdout:2/915: unlink d5/d26/fb3 0 2024-04-20T10:59:28.261 INFO:tasks.workunit.client.0.smithi027.stdout:2/916: write d5/dc/d16/d2a/d59/fad [411724,122435] 0 2024-04-20T10:59:28.262 INFO:tasks.workunit.client.0.smithi027.stdout:5/894: creat d19/d1b/d1c/d1e/d22/d68/d75/f128 x:0 0 0 2024-04-20T10:59:28.262 INFO:tasks.workunit.client.0.smithi027.stdout:5/895: chown d19/d1b/d1c/d1e/d58/db4/fe4 71 1 2024-04-20T10:59:28.262 INFO:tasks.workunit.client.0.smithi027.stdout:4/892: dread d0/d48/d123/d2f/ff6 [0,4194304] 0 2024-04-20T10:59:28.262 INFO:tasks.workunit.client.0.smithi027.stdout:9/842: creat d4/d14/d43/ffb x:0 0 0 2024-04-20T10:59:28.265 INFO:tasks.workunit.client.0.smithi027.stdout:2/917: link d5/d26/fd9 d5/dc/d16/d37/de2/dac/f126 0 2024-04-20T10:59:28.266 INFO:tasks.workunit.client.0.smithi027.stdout:5/896: creat d19/d1b/d1c/dbd/dbe/f129 x:0 0 0 2024-04-20T10:59:28.266 INFO:tasks.workunit.client.0.smithi027.stdout:9/843: creat d4/d1b/ffc x:0 0 0 2024-04-20T10:59:28.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/844: rename d4/d74/dae to d4/d1b/d8f/de1/dfd 0 2024-04-20T10:59:28.270 INFO:tasks.workunit.client.0.smithi027.stdout:5/897: creat d19/d1b/d1c/d1e/d83/dcc/f12a x:0 0 0 2024-04-20T10:59:28.274 INFO:tasks.workunit.client.0.smithi027.stdout:7/934: dwrite d4/d5/d6f/d110/f124 [0,4194304] 0 2024-04-20T10:59:28.278 INFO:tasks.workunit.client.0.smithi027.stdout:4/893: dwrite d0/f108 [0,4194304] 0 2024-04-20T10:59:28.278 INFO:tasks.workunit.client.0.smithi027.stdout:4/894: chown d0/d14/d6b/da3/d18/f2d 21 1 2024-04-20T10:59:28.289 INFO:tasks.workunit.client.0.smithi027.stdout:7/935: write d4/d5/f5c [3074666,52357] 0 2024-04-20T10:59:28.289 INFO:tasks.workunit.client.0.smithi027.stdout:7/936: read - d4/d5/d8/d127/f12d zero size 2024-04-20T10:59:28.290 INFO:tasks.workunit.client.0.smithi027.stdout:2/918: write d5/dc/d16/dd0/f117 [689568,26250] 0 2024-04-20T10:59:28.291 INFO:tasks.workunit.client.0.smithi027.stdout:2/919: chown d5/dc/d16/d2a/f40 515017 1 2024-04-20T10:59:28.291 INFO:tasks.workunit.client.0.smithi027.stdout:2/920: chown d5/dc/d16/d2a/d59/f95 110169380 1 2024-04-20T10:59:28.291 INFO:tasks.workunit.client.0.smithi027.stdout:9/845: dwrite d4/d1b/d1f/d5a/db7/fcf [0,4194304] 0 2024-04-20T10:59:28.291 INFO:tasks.workunit.client.0.smithi027.stdout:2/921: getdents d5/dc/d16/d32/dab/db9/d10c 0 2024-04-20T10:59:28.291 INFO:tasks.workunit.client.0.smithi027.stdout:5/898: dwrite d19/d1b/d1c/d1e/d25/d40/f81 [8388608,4194304] 0 2024-04-20T10:59:28.291 INFO:tasks.workunit.client.0.smithi027.stdout:4/895: truncate d0/d48/d123/f3d 613219 0 2024-04-20T10:59:28.292 INFO:tasks.workunit.client.0.smithi027.stdout:7/937: read - d4/f98 zero size 2024-04-20T10:59:28.292 INFO:tasks.workunit.client.0.smithi027.stdout:2/922: dread d5/dc/f1c [0,4194304] 0 2024-04-20T10:59:28.292 INFO:tasks.workunit.client.0.smithi027.stdout:9/846: creat d4/d1b/d1f/ffe x:0 0 0 2024-04-20T10:59:28.293 INFO:tasks.workunit.client.0.smithi027.stdout:5/899: symlink d19/d1b/d1c/d1e/d22/db1/dcf/l12b 0 2024-04-20T10:59:28.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/938: dread d4/d5/d8/d46/d77/d7c/f8c [0,4194304] 0 2024-04-20T10:59:28.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/939: write d4/d5/d8/d46/d77/d7c/f11d [736070,99283] 0 2024-04-20T10:59:28.293 INFO:tasks.workunit.client.0.smithi027.stdout:7/940: write d4/d5/d8/d21/d94/da7/dc2/f126 [122231,124061] 0 2024-04-20T10:59:28.294 INFO:tasks.workunit.client.0.smithi027.stdout:4/896: rename d0/d5e/d6c/fb2 to d0/d48/d68/d76/de2/f136 0 2024-04-20T10:59:28.295 INFO:tasks.workunit.client.0.smithi027.stdout:4/897: write d0/d48/d123/f21 [3854773,98396] 0 2024-04-20T10:59:28.295 INFO:tasks.workunit.client.0.smithi027.stdout:2/923: link d5/d26/d47/fdc d5/dc/d16/d32/d109/f127 0 2024-04-20T10:59:28.295 INFO:tasks.workunit.client.0.smithi027.stdout:2/924: fsync d5/d26/d30/f79 0 2024-04-20T10:59:28.296 INFO:tasks.workunit.client.0.smithi027.stdout:5/900: mknod d19/d1b/c12c 0 2024-04-20T10:59:28.297 INFO:tasks.workunit.client.0.smithi027.stdout:9/847: rename d4/d74/cce to d4/d1b/d8f/cff 0 2024-04-20T10:59:28.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/898: creat d0/d48/d123/d2f/de6/f137 x:0 0 0 2024-04-20T10:59:28.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/899: chown d0/l10 187212449 1 2024-04-20T10:59:28.302 INFO:tasks.workunit.client.0.smithi027.stdout:7/941: dread d4/d5/d8/d21/d94/fbd [0,4194304] 0 2024-04-20T10:59:28.304 INFO:tasks.workunit.client.0.smithi027.stdout:9/848: write d4/d1b/d31/f9a [325953,73791] 0 2024-04-20T10:59:28.306 INFO:tasks.workunit.client.0.smithi027.stdout:2/925: link d5/dc/d16/f124 d5/dc/d16/d37/de2/f128 0 2024-04-20T10:59:28.306 INFO:tasks.workunit.client.0.smithi027.stdout:2/926: write d5/d44/ffe [18127,34635] 0 2024-04-20T10:59:28.306 INFO:tasks.workunit.client.0.smithi027.stdout:2/927: fdatasync d5/d1e/d9d/fe9 0 2024-04-20T10:59:28.306 INFO:tasks.workunit.client.0.smithi027.stdout:5/901: getdents d19/d1b/d1c/d1e/da3 0 2024-04-20T10:59:28.307 INFO:tasks.workunit.client.0.smithi027.stdout:9/849: mkdir d4/d1b/d50/d100 0 2024-04-20T10:59:28.316 INFO:tasks.workunit.client.0.smithi027.stdout:7/942: write d4/d5/d8/d21/f48 [3862251,36700] 0 2024-04-20T10:59:28.316 INFO:tasks.workunit.client.0.smithi027.stdout:2/928: rmdir d5/d44 39 2024-04-20T10:59:28.317 INFO:tasks.workunit.client.0.smithi027.stdout:5/902: chown d19/d1b/d1c/d1e/d83/lf2 59145 1 2024-04-20T10:59:28.318 INFO:tasks.workunit.client.0.smithi027.stdout:4/900: dwrite d0/d14/d6b/d92/fc7 [0,4194304] 0 2024-04-20T10:59:28.320 INFO:tasks.workunit.client.0.smithi027.stdout:7/943: creat d4/d5/d8/d46/f130 x:0 0 0 2024-04-20T10:59:28.328 INFO:tasks.workunit.client.0.smithi027.stdout:9/850: dwrite d4/d1b/d1f/d5a/db7/fc5 [0,4194304] 0 2024-04-20T10:59:28.331 INFO:tasks.workunit.client.0.smithi027.stdout:7/944: symlink d4/d5/d8/d21/d94/da7/dc2/l131 0 2024-04-20T10:59:28.331 INFO:tasks.workunit.client.0.smithi027.stdout:7/945: read d4/d5b/fae [3747620,127089] 0 2024-04-20T10:59:28.331 INFO:tasks.workunit.client.0.smithi027.stdout:7/946: write d4/d1b/d58/fd2 [892298,11224] 0 2024-04-20T10:59:28.331 INFO:tasks.workunit.client.0.smithi027.stdout:7/947: dread - d4/d1b/d58/fe2 zero size 2024-04-20T10:59:28.351 INFO:tasks.workunit.client.0.smithi027.stdout:2/929: dwrite d5/f17 [0,4194304] 0 2024-04-20T10:59:28.353 INFO:tasks.workunit.client.0.smithi027.stdout:9/851: mknod d4/d1b/d31/c101 0 2024-04-20T10:59:28.353 INFO:tasks.workunit.client.0.smithi027.stdout:4/901: dwrite d0/d14/d6b/da3/d18/f2d [4194304,4194304] 0 2024-04-20T10:59:28.353 INFO:tasks.workunit.client.0.smithi027.stdout:4/902: chown d0/d14/d6b/da3/d53/d6a/lf9 98 1 2024-04-20T10:59:28.353 INFO:tasks.workunit.client.0.smithi027.stdout:7/948: dwrite d4/d5/f33 [4194304,4194304] 0 2024-04-20T10:59:28.354 INFO:tasks.workunit.client.0.smithi027.stdout:5/903: dwrite d19/d1b/d1c/d1e/d22/d68/d75/f128 [0,4194304] 0 2024-04-20T10:59:28.354 INFO:tasks.workunit.client.0.smithi027.stdout:5/904: fsync d19/d1b/d1c/d1e/d22/d68/d75/f128 0 2024-04-20T10:59:28.354 INFO:tasks.workunit.client.0.smithi027.stdout:2/930: rmdir d5/dc/d16/d2a 39 2024-04-20T10:59:28.354 INFO:tasks.workunit.client.0.smithi027.stdout:2/931: fdatasync d5/d26/f3a 0 2024-04-20T10:59:28.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/852: mknod d4/d1b/d31/d23/c102 0 2024-04-20T10:59:28.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/853: chown d4/f9f 13637769 1 2024-04-20T10:59:28.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/854: write d4/d1b/d50/d5d/f9d [52700,66965] 0 2024-04-20T10:59:28.356 INFO:tasks.workunit.client.0.smithi027.stdout:4/903: creat d0/d14/d6b/d92/dcd/f138 x:0 0 0 2024-04-20T10:59:28.356 INFO:tasks.workunit.client.0.smithi027.stdout:4/904: readlink d0/d48/d123/d33/l131 0 2024-04-20T10:59:28.357 INFO:tasks.workunit.client.0.smithi027.stdout:2/932: read d5/d26/d47/d8a/f9a [438137,87469] 0 2024-04-20T10:59:28.357 INFO:tasks.workunit.client.0.smithi027.stdout:2/933: write d5/f6c [46775,127257] 0 2024-04-20T10:59:28.358 INFO:tasks.workunit.client.0.smithi027.stdout:7/949: mknod d4/d5/c132 0 2024-04-20T10:59:28.358 INFO:tasks.workunit.client.0.smithi027.stdout:7/950: read - d4/f98 zero size 2024-04-20T10:59:28.359 INFO:tasks.workunit.client.0.smithi027.stdout:9/855: mkdir d4/d1b/daf/d103 0 2024-04-20T10:59:28.363 INFO:tasks.workunit.client.0.smithi027.stdout:5/905: dread d19/d1b/d1c/d1e/d25/fc0 [0,4194304] 0 2024-04-20T10:59:28.364 INFO:tasks.workunit.client.0.smithi027.stdout:4/905: getdents d0/d48/d123/d33 0 2024-04-20T10:59:28.366 INFO:tasks.workunit.client.0.smithi027.stdout:2/934: creat d5/dc/d16/d37/de2/dac/d8f/dfd/f129 x:0 0 0 2024-04-20T10:59:28.368 INFO:tasks.workunit.client.0.smithi027.stdout:5/906: mknod d19/d1b/d1c/d1e/d22/d68/d75/db7/c12d 0 2024-04-20T10:59:28.369 INFO:tasks.workunit.client.0.smithi027.stdout:4/906: creat d0/d14/d6b/da3/d53/d6a/da1/de1/f139 x:0 0 0 2024-04-20T10:59:28.370 INFO:tasks.workunit.client.0.smithi027.stdout:2/935: creat d5/dc/d16/d2a/d59/f12a x:0 0 0 2024-04-20T10:59:28.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/951: dwrite d4/d5/d6f/fed [0,4194304] 0 2024-04-20T10:59:28.373 INFO:tasks.workunit.client.0.smithi027.stdout:5/907: creat d19/d1b/d1c/d1e/d25/de5/f12e x:0 0 0 2024-04-20T10:59:28.375 INFO:tasks.workunit.client.0.smithi027.stdout:7/952: mkdir d4/d5/d8/d46/d106/d133 0 2024-04-20T10:59:28.376 INFO:tasks.workunit.client.0.smithi027.stdout:5/908: mknod d19/d1b/d1c/d1e/d58/db4/ded/c12f 0 2024-04-20T10:59:28.376 INFO:tasks.workunit.client.0.smithi027.stdout:5/909: chown f3 3293 1 2024-04-20T10:59:28.376 INFO:tasks.workunit.client.0.smithi027.stdout:5/910: write d19/d1b/d1c/f98 [378781,24940] 0 2024-04-20T10:59:28.377 INFO:tasks.workunit.client.0.smithi027.stdout:9/856: dwrite d4/fb8 [0,4194304] 0 2024-04-20T10:59:28.379 INFO:tasks.workunit.client.0.smithi027.stdout:4/907: rename d0/d48/l59 to d0/d14/l13a 0 2024-04-20T10:59:28.379 INFO:tasks.workunit.client.0.smithi027.stdout:7/953: mkdir d4/d5/d8/d127/d134 0 2024-04-20T10:59:28.382 INFO:tasks.workunit.client.0.smithi027.stdout:7/954: dread d4/d5/d8/fac [0,4194304] 0 2024-04-20T10:59:28.382 INFO:tasks.workunit.client.0.smithi027.stdout:7/955: dread - d4/d4d/fbc zero size 2024-04-20T10:59:28.382 INFO:tasks.workunit.client.0.smithi027.stdout:7/956: readlink d4/d5/d81/lc3 0 2024-04-20T10:59:28.382 INFO:tasks.workunit.client.0.smithi027.stdout:7/957: write d4/d5/d8/d45/d11e/f11f [923741,19943] 0 2024-04-20T10:59:28.383 INFO:tasks.workunit.client.0.smithi027.stdout:9/857: unlink d4/d14/f44 0 2024-04-20T10:59:28.383 INFO:tasks.workunit.client.0.smithi027.stdout:9/858: write d4/d1b/d1f/d5a/fb4 [141966,37809] 0 2024-04-20T10:59:28.384 INFO:tasks.workunit.client.0.smithi027.stdout:2/936: rename d5/d1e/d9d to d5/dc/d16/d2a/d12b 0 2024-04-20T10:59:28.385 INFO:tasks.workunit.client.0.smithi027.stdout:4/908: truncate d0/f108 3323371 0 2024-04-20T10:59:28.385 INFO:tasks.workunit.client.0.smithi027.stdout:4/909: fdatasync d0/d14/d6b/da3/d18/dbf/f119 0 2024-04-20T10:59:28.386 INFO:tasks.workunit.client.0.smithi027.stdout:5/911: dread d19/d1b/d1c/d1e/d22/d68/d75/f128 [0,4194304] 0 2024-04-20T10:59:28.387 INFO:tasks.workunit.client.0.smithi027.stdout:9/859: symlink d4/d1b/d8f/l104 0 2024-04-20T10:59:28.388 INFO:tasks.workunit.client.0.smithi027.stdout:2/937: truncate d5/d26/d47/d8a/fa3 2885247 0 2024-04-20T10:59:28.390 INFO:tasks.workunit.client.0.smithi027.stdout:4/910: rename d0/d14/d6b/da3/d53/d6a/da1/de1/f139 to d0/d48/d123/dc3/f13b 0 2024-04-20T10:59:28.390 INFO:tasks.workunit.client.0.smithi027.stdout:5/912: mknod d19/d1b/d1c/d1e/c130 0 2024-04-20T10:59:28.391 INFO:tasks.workunit.client.0.smithi027.stdout:7/958: symlink d4/d1b/d4a/d7a/d93/l135 0 2024-04-20T10:59:28.392 INFO:tasks.workunit.client.0.smithi027.stdout:7/959: chown d4/d1b/ff9 229 1 2024-04-20T10:59:28.393 INFO:tasks.workunit.client.0.smithi027.stdout:9/860: creat d4/d74/d72/f105 x:0 0 0 2024-04-20T10:59:28.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/938: creat d5/dc/d16/d32/d109/d113/f12c x:0 0 0 2024-04-20T10:59:28.402 INFO:tasks.workunit.client.0.smithi027.stdout:2/939: write d5/dc/d16/d37/de2/d8e/fc6 [1564706,55219] 0 2024-04-20T10:59:28.404 INFO:tasks.workunit.client.0.smithi027.stdout:5/913: creat d19/d1b/d1c/d1e/d25/f131 x:0 0 0 2024-04-20T10:59:28.404 INFO:tasks.workunit.client.0.smithi027.stdout:5/914: dread - d19/d1b/d1c/d1e/d3c/fba zero size 2024-04-20T10:59:28.404 INFO:tasks.workunit.client.0.smithi027.stdout:5/915: write d19/d1b/d1c/d1e/d22/ff5 [1030850,85554] 0 2024-04-20T10:59:28.404 INFO:tasks.workunit.client.0.smithi027.stdout:5/916: read - d19/d1b/d1c/d1e/da3/db9/f11d zero size 2024-04-20T10:59:28.405 INFO:tasks.workunit.client.0.smithi027.stdout:9/861: truncate d4/d1b/d1f/d45/d8a/f6e 1656181 0 2024-04-20T10:59:28.407 INFO:tasks.workunit.client.0.smithi027.stdout:2/940: creat d5/dc/d16/d2a/d12b/f12d x:0 0 0 2024-04-20T10:59:28.408 INFO:tasks.workunit.client.0.smithi027.stdout:9/862: mknod d4/d1b/d8f/de1/dfd/c106 0 2024-04-20T10:59:28.416 INFO:tasks.workunit.client.0.smithi027.stdout:4/911: dwrite d0/d14/d6b/da3/fe0 [0,4194304] 0 2024-04-20T10:59:28.416 INFO:tasks.workunit.client.0.smithi027.stdout:4/912: stat d0/d14/d6b/da3/d1a/d3b/l3c 0 2024-04-20T10:59:28.417 INFO:tasks.workunit.client.0.smithi027.stdout:7/960: dwrite d4/d5/d8/d21/f96 [0,4194304] 0 2024-04-20T10:59:28.418 INFO:tasks.workunit.client.0.smithi027.stdout:2/941: creat d5/dbc/f12e x:0 0 0 2024-04-20T10:59:28.418 INFO:tasks.workunit.client.0.smithi027.stdout:2/942: chown d5/dc/d16/d2a/f84 46346354 1 2024-04-20T10:59:28.418 INFO:tasks.workunit.client.0.smithi027.stdout:5/917: mknod d19/c132 0 2024-04-20T10:59:28.418 INFO:tasks.workunit.client.0.smithi027.stdout:5/918: dread - d19/d1b/d1c/d1e/d3c/f121 zero size 2024-04-20T10:59:28.421 INFO:tasks.workunit.client.0.smithi027.stdout:7/961: rmdir d4/d1b/d58 39 2024-04-20T10:59:28.425 INFO:tasks.workunit.client.0.smithi027.stdout:7/962: dread d4/d5/d8/d45/ff7 [0,4194304] 0 2024-04-20T10:59:28.425 INFO:tasks.workunit.client.0.smithi027.stdout:7/963: write d4/d5/d8/f4c [4383042,83183] 0 2024-04-20T10:59:28.425 INFO:tasks.workunit.client.0.smithi027.stdout:9/863: dwrite d4/fc8 [4194304,4194304] 0 2024-04-20T10:59:28.426 INFO:tasks.workunit.client.0.smithi027.stdout:2/943: mkdir d5/d44/d12f 0 2024-04-20T10:59:28.427 INFO:tasks.workunit.client.0.smithi027.stdout:5/919: mknod d19/d1b/d1c/d1e/da3/db9/c133 0 2024-04-20T10:59:28.427 INFO:tasks.workunit.client.0.smithi027.stdout:4/913: dread d0/d14/d6b/da3/f2e [0,4194304] 0 2024-04-20T10:59:28.428 INFO:tasks.workunit.client.0.smithi027.stdout:7/964: link d4/d5/d8/c15 d4/d1b/d4a/d62/dcd/c136 0 2024-04-20T10:59:28.428 INFO:tasks.workunit.client.0.smithi027.stdout:7/965: chown d4/d5/d8/d21/d94/da7/f12c 10 1 2024-04-20T10:59:28.430 INFO:tasks.workunit.client.0.smithi027.stdout:9/864: rename d4/d1b/ffc to d4/d1b/d1f/d5a/db7/f107 0 2024-04-20T10:59:28.431 INFO:tasks.workunit.client.0.smithi027.stdout:2/944: rmdir d5/dc/d10d 39 2024-04-20T10:59:28.431 INFO:tasks.workunit.client.0.smithi027.stdout:9/865: write d4/d1b/f53 [43564,35391] 0 2024-04-20T10:59:28.432 INFO:tasks.workunit.client.0.smithi027.stdout:4/914: creat d0/d5e/d6c/de8/f13c x:0 0 0 2024-04-20T10:59:28.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/915: fsync d0/d14/d6b/da3/d18/fa8 0 2024-04-20T10:59:28.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/916: write d0/d48/d123/f7f [4297332,81095] 0 2024-04-20T10:59:28.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/917: truncate d0/d14/d6b/da3/d53/f120 424611 0 2024-04-20T10:59:28.433 INFO:tasks.workunit.client.0.smithi027.stdout:5/920: creat d19/d1b/d1c/d1e/d22/d68/d75/de9/f134 x:0 0 0 2024-04-20T10:59:28.433 INFO:tasks.workunit.client.0.smithi027.stdout:5/921: chown d19/d1b/d1c/d1e/d22/dca/l10e 187653 1 2024-04-20T10:59:28.433 INFO:tasks.workunit.client.0.smithi027.stdout:5/922: chown d19/l97 671905959 1 2024-04-20T10:59:28.433 INFO:tasks.workunit.client.0.smithi027.stdout:7/966: mkdir d4/d5/d8/d21/d94/da7/dce/d137 0 2024-04-20T10:59:28.434 INFO:tasks.workunit.client.0.smithi027.stdout:9/866: rename d4/d1b/d31/d51/fef to d4/d1b/d1f/d45/d8a/f108 0 2024-04-20T10:59:28.436 INFO:tasks.workunit.client.0.smithi027.stdout:4/918: symlink d0/d14/d6b/da3/d53/d6a/da1/db0/l13d 0 2024-04-20T10:59:28.439 INFO:tasks.workunit.client.0.smithi027.stdout:5/923: rename d19/d1b/d1c/d1e/d22/fd6 to d19/d1b/d1c/d1e/d25/d40/deb/f135 0 2024-04-20T10:59:28.439 INFO:tasks.workunit.client.0.smithi027.stdout:9/867: symlink d4/d1b/d1f/d45/d9e/l109 0 2024-04-20T10:59:28.440 INFO:tasks.workunit.client.0.smithi027.stdout:4/919: getdents d0/d14/d6b/da3/d1a/d3b 0 2024-04-20T10:59:28.442 INFO:tasks.workunit.client.0.smithi027.stdout:4/920: dread d0/d14/d6b/da3/d53/d6a/fd5 [0,4194304] 0 2024-04-20T10:59:28.442 INFO:tasks.workunit.client.0.smithi027.stdout:4/921: write d0/d48/d123/fa5 [266626,8145] 0 2024-04-20T10:59:28.444 INFO:tasks.workunit.client.0.smithi027.stdout:7/967: dread d4/d1b/d58/fb8 [0,4194304] 0 2024-04-20T10:59:28.444 INFO:tasks.workunit.client.0.smithi027.stdout:7/968: fdatasync d4/d5/d6f/fe4 0 2024-04-20T10:59:28.444 INFO:tasks.workunit.client.0.smithi027.stdout:7/969: read d4/d5b/fae [2599303,115994] 0 2024-04-20T10:59:28.447 INFO:tasks.workunit.client.0.smithi027.stdout:9/868: dread d4/d1b/d50/d5d/fb1 [0,4194304] 0 2024-04-20T10:59:28.448 INFO:tasks.workunit.client.0.smithi027.stdout:5/924: getdents d19/d1b/d1c/d1e/d22/d68/d75 0 2024-04-20T10:59:28.449 INFO:tasks.workunit.client.0.smithi027.stdout:2/945: dwrite d5/d1e/f9c [0,4194304] 0 2024-04-20T10:59:28.449 INFO:tasks.workunit.client.0.smithi027.stdout:2/946: readlink d5/dc/d16/db1/ldb 0 2024-04-20T10:59:28.450 INFO:tasks.workunit.client.0.smithi027.stdout:4/922: link d0/d14/d6b/da3/d53/d6a/faa d0/d14/d71/f13e 0 2024-04-20T10:59:28.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/869: getdents d4/d14/d43/d91 0 2024-04-20T10:59:28.452 INFO:tasks.workunit.client.0.smithi027.stdout:5/925: unlink d19/d1b/d1c/d1e/d22/l6f 0 2024-04-20T10:59:28.456 INFO:tasks.workunit.client.0.smithi027.stdout:7/970: dread d4/d5/d8/d21/f48 [0,4194304] 0 2024-04-20T10:59:28.458 INFO:tasks.workunit.client.0.smithi027.stdout:4/923: mknod d0/d5e/d6c/dd8/c13f 0 2024-04-20T10:59:28.460 INFO:tasks.workunit.client.0.smithi027.stdout:7/971: mkdir d4/d5/d8/d127/d134/d138 0 2024-04-20T10:59:28.461 INFO:tasks.workunit.client.0.smithi027.stdout:9/870: write d4/d14/f27 [1590435,75054] 0 2024-04-20T10:59:28.461 INFO:tasks.workunit.client.0.smithi027.stdout:9/871: chown d4/d1b/d50/ff6 3441 1 2024-04-20T10:59:28.461 INFO:tasks.workunit.client.0.smithi027.stdout:2/947: mknod d5/dc/d16/dd0/dd5/c130 0 2024-04-20T10:59:28.463 INFO:tasks.workunit.client.0.smithi027.stdout:4/924: creat d0/d5e/d6c/de8/f140 x:0 0 0 2024-04-20T10:59:28.463 INFO:tasks.workunit.client.0.smithi027.stdout:4/925: readlink d0/d14/d6b/da3/d18/dbf/lde 0 2024-04-20T10:59:28.464 INFO:tasks.workunit.client.0.smithi027.stdout:7/972: mkdir d4/d5/d8/d21/d139 0 2024-04-20T10:59:28.469 INFO:tasks.workunit.client.0.smithi027.stdout:4/926: dread d0/d48/d123/d33/d5c/d6f/dd1/f102 [4194304,4194304] 0 2024-04-20T10:59:28.471 INFO:tasks.workunit.client.0.smithi027.stdout:9/872: truncate d4/f7 11832652 0 2024-04-20T10:59:28.471 INFO:tasks.workunit.client.0.smithi027.stdout:9/873: chown d4/d1b/d31/dd0/le7 3160983 1 2024-04-20T10:59:28.471 INFO:tasks.workunit.client.0.smithi027.stdout:9/874: write d4/d14/da8/ff7 [995941,92100] 0 2024-04-20T10:59:28.471 INFO:tasks.workunit.client.0.smithi027.stdout:9/875: stat d4/d1b/d31/dd0/dda/fe6 0 2024-04-20T10:59:28.476 INFO:tasks.workunit.client.0.smithi027.stdout:2/948: mkdir d5/d26/d47/d131 0 2024-04-20T10:59:28.476 INFO:tasks.workunit.client.0.smithi027.stdout:2/949: chown d5/d26/d78/f11f 82494922 1 2024-04-20T10:59:28.477 INFO:tasks.workunit.client.0.smithi027.stdout:7/973: creat d4/d5/d8/d46/d77/f13a x:0 0 0 2024-04-20T10:59:28.477 INFO:tasks.workunit.client.0.smithi027.stdout:7/974: write d4/d5/d8/f38 [153342,29931] 0 2024-04-20T10:59:28.479 INFO:tasks.workunit.client.0.smithi027.stdout:4/927: creat d0/d14/d6b/da3/f141 x:0 0 0 2024-04-20T10:59:28.479 INFO:tasks.workunit.client.0.smithi027.stdout:4/928: write d0/d48/d123/d33/fd0 [3127582,70813] 0 2024-04-20T10:59:28.490 INFO:tasks.workunit.client.0.smithi027.stdout:4/929: dread d0/d14/d6b/da3/d1a/d3b/fc8 [0,4194304] 0 2024-04-20T10:59:28.491 INFO:tasks.workunit.client.0.smithi027.stdout:2/950: link d5/d26/c11a d5/dc/d16/d2a/d59/c132 0 2024-04-20T10:59:28.491 INFO:tasks.workunit.client.0.smithi027.stdout:2/951: chown d5/dc/d16/d2a/f40 3454 1 2024-04-20T10:59:28.494 INFO:tasks.workunit.client.0.smithi027.stdout:7/975: dwrite d4/da2/dd1/dfb/d104/d117/d119/f12e [0,4194304] 0 2024-04-20T10:59:28.496 INFO:tasks.workunit.client.0.smithi027.stdout:4/930: unlink d0/d48/d123/dc3/dc5/c107 0 2024-04-20T10:59:28.506 INFO:tasks.workunit.client.0.smithi027.stdout:2/952: getdents d5/dc/d16/d32 0 2024-04-20T10:59:28.506 INFO:tasks.workunit.client.0.smithi027.stdout:4/931: read d0/d48/d123/f30 [3882494,15686] 0 2024-04-20T10:59:28.551 INFO:tasks.workunit.client.0.smithi027.stdout:2/953: getdents d5/dc/d16 0 2024-04-20T10:59:28.551 INFO:tasks.workunit.client.0.smithi027.stdout:2/954: dread - d5/dc/d16/d37/de2/dac/fe6 zero size 2024-04-20T10:59:28.552 INFO:tasks.workunit.client.0.smithi027.stdout:4/932: link d0/d48/d123/d33/c40 d0/d14/d6b/da3/d53/d6a/da1/de1/c142 0 2024-04-20T10:59:28.569 INFO:tasks.workunit.client.0.smithi027.stdout:4/933: dwrite d0/d14/d6b/da3/d18/f2d [0,4194304] 0 2024-04-20T10:59:28.569 INFO:tasks.workunit.client.0.smithi027.stdout:4/934: readlink d0/d14/d6b/d92/ldc 0 2024-04-20T10:59:28.585 INFO:tasks.workunit.client.0.smithi027.stdout:4/935: dwrite d0/d14/d6b/da3/d18/f2d [0,4194304] 0 2024-04-20T10:59:28.602 INFO:tasks.workunit.client.0.smithi027.stdout:4/936: dwrite d0/d48/f11c [0,4194304] 0 2024-04-20T10:59:28.611 INFO:tasks.workunit.client.0.smithi027.stdout:4/937: write d0/d14/fc4 [274567,103516] 0 2024-04-20T10:59:28.612 INFO:tasks.workunit.client.0.smithi027.stdout:2/955: rmdir d5/d44/d12f 0 2024-04-20T10:59:28.618 INFO:tasks.workunit.client.0.smithi027.stdout:2/956: dread d5/dc/fff [0,4194304] 0 2024-04-20T10:59:28.619 INFO:tasks.workunit.client.0.smithi027.stdout:2/957: fdatasync d5/dc/fff 0 2024-04-20T10:59:28.622 INFO:tasks.workunit.client.0.smithi027.stdout:5/926: sync 2024-04-20T10:59:28.622 INFO:tasks.workunit.client.0.smithi027.stdout:5/927: dread - d19/d1b/d1c/d1e/d22/d68/d75/db7/f102 zero size 2024-04-20T10:59:28.626 INFO:tasks.workunit.client.0.smithi027.stdout:9/876: sync 2024-04-20T10:59:28.626 INFO:tasks.workunit.client.0.smithi027.stdout:7/976: sync 2024-04-20T10:59:28.626 INFO:tasks.workunit.client.0.smithi027.stdout:7/977: fdatasync d4/d5/f33 0 2024-04-20T10:59:28.626 INFO:tasks.workunit.client.0.smithi027.stdout:7/978: fdatasync d4/d1b/d4a/d62/fdd 0 2024-04-20T10:59:28.626 INFO:tasks.workunit.client.0.smithi027.stdout:7/979: chown d4/d4d/f91 2 1 2024-04-20T10:59:28.627 INFO:tasks.workunit.client.0.smithi027.stdout:4/938: getdents d0/d48/d123/d2f/de6 0 2024-04-20T10:59:28.627 INFO:tasks.workunit.client.0.smithi027.stdout:4/939: readlink d0/d14/d6b/da3/d53/d6a/lbd 0 2024-04-20T10:59:28.637 INFO:tasks.workunit.client.0.smithi027.stdout:5/928: dwrite d19/d1b/d1c/d1e/d25/d40/fe7 [0,4194304] 0 2024-04-20T10:59:28.637 INFO:tasks.workunit.client.0.smithi027.stdout:5/929: write d19/d1b/f8d [2870264,48427] 0 2024-04-20T10:59:28.641 INFO:tasks.workunit.client.0.smithi027.stdout:9/877: rename d4/d14 to d4/d1b/daf/d10a 0 2024-04-20T10:59:28.641 INFO:tasks.workunit.client.0.smithi027.stdout:9/878: truncate d4/d1b/daf/d10a/d43/ffb 177559 0 2024-04-20T10:59:28.641 INFO:tasks.workunit.client.0.smithi027.stdout:9/879: chown d4/d2a/l2b 56004 1 2024-04-20T10:59:28.642 INFO:tasks.workunit.client.0.smithi027.stdout:4/940: creat d0/d48/d123/d33/d5c/d6f/dd1/f143 x:0 0 0 2024-04-20T10:59:28.645 INFO:tasks.workunit.client.0.smithi027.stdout:7/980: dwrite d4/d5/f1f [0,4194304] 0 2024-04-20T10:59:28.646 INFO:tasks.workunit.client.0.smithi027.stdout:2/958: read d5/d26/d30/f3c [1020364,8596] 0 2024-04-20T10:59:28.646 INFO:tasks.workunit.client.0.smithi027.stdout:2/959: chown d5/dc/d16/d32/c10a 1369950 1 2024-04-20T10:59:28.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/880: rename d4/d1b/fa5 to d4/d1b/daf/d10a/da8/f10b 0 2024-04-20T10:59:28.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/930: dwrite d19/d1b/d1c/d1e/d58/f94 [4194304,4194304] 0 2024-04-20T10:59:28.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/931: write d19/d1b/d1c/d1e/f79 [4161,40367] 0 2024-04-20T10:59:28.661 INFO:tasks.workunit.client.0.smithi027.stdout:4/941: rename d0/c11 to d0/d48/d123/d33/d5c/d6f/d7b/c144 0 2024-04-20T10:59:28.661 INFO:tasks.workunit.client.0.smithi027.stdout:9/881: dread d4/d1b/d1f/d45/d8a/fd9 [0,4194304] 0 2024-04-20T10:59:28.664 INFO:tasks.workunit.client.0.smithi027.stdout:9/882: dread d4/d1b/d31/d23/d29/f82 [0,4194304] 0 2024-04-20T10:59:28.672 INFO:tasks.workunit.client.0.smithi027.stdout:2/960: dwrite f1 [0,4194304] 0 2024-04-20T10:59:28.672 INFO:tasks.workunit.client.0.smithi027.stdout:2/961: chown d5/dc/d16/d2a/d59/da4/de1/dcc/faa 278806238 1 2024-04-20T10:59:28.673 INFO:tasks.workunit.client.0.smithi027.stdout:2/962: getdents d5/d44/d121 0 2024-04-20T10:59:28.674 INFO:tasks.workunit.client.0.smithi027.stdout:5/932: mknod d19/d1b/d1c/d1e/d22/c136 0 2024-04-20T10:59:28.675 INFO:tasks.workunit.client.0.smithi027.stdout:5/933: fdatasync d19/d1b/d1c/d1e/d22/db1/fb3 0 2024-04-20T10:59:28.675 INFO:tasks.workunit.client.0.smithi027.stdout:5/934: truncate d19/d1b/d1c/d1e/d83/dcc/f12a 750474 0 2024-04-20T10:59:28.676 INFO:tasks.workunit.client.0.smithi027.stdout:4/942: rename d0/d14/d6b/da3/d53/fb4 to d0/d14/d6b/da3/d1a/dae/f145 0 2024-04-20T10:59:28.680 INFO:tasks.workunit.client.0.smithi027.stdout:9/883: dwrite d4/d1b/d50/d5d/fb1 [0,4194304] 0 2024-04-20T10:59:28.681 INFO:tasks.workunit.client.0.smithi027.stdout:5/935: link d19/d1b/d1c/d1e/d58/d9d/fe0 d19/d1b/d1c/d1e/d22/db1/f137 0 2024-04-20T10:59:28.682 INFO:tasks.workunit.client.0.smithi027.stdout:2/963: write d5/d1e/d21/f33 [1701655,8636] 0 2024-04-20T10:59:28.682 INFO:tasks.workunit.client.0.smithi027.stdout:2/964: dread - d5/d26/d78/f11f zero size 2024-04-20T10:59:28.682 INFO:tasks.workunit.client.0.smithi027.stdout:2/965: fsync d5/d26/f3a 0 2024-04-20T10:59:28.682 INFO:tasks.workunit.client.0.smithi027.stdout:2/966: dread - d5/dc/d16/d37/de2/dac/fe6 zero size 2024-04-20T10:59:28.682 INFO:tasks.workunit.client.0.smithi027.stdout:9/884: creat d4/d1b/d31/dd0/f10c x:0 0 0 2024-04-20T10:59:28.682 INFO:tasks.workunit.client.0.smithi027.stdout:9/885: readlink d4/d1b/d31/d23/d29/d7f/dec/lf9 0 2024-04-20T10:59:28.683 INFO:tasks.workunit.client.0.smithi027.stdout:5/936: truncate d19/d1b/d1c/d1e/d22/fe8 846362 0 2024-04-20T10:59:28.683 INFO:tasks.workunit.client.0.smithi027.stdout:2/967: mknod d5/dc/d16/d37/de2/c133 0 2024-04-20T10:59:28.685 INFO:tasks.workunit.client.0.smithi027.stdout:5/937: dread d19/d1b/d1c/f6a [0,4194304] 0 2024-04-20T10:59:28.686 INFO:tasks.workunit.client.0.smithi027.stdout:9/886: rename d4/l59 to d4/d1b/d1f/d57/db0/l10d 0 2024-04-20T10:59:28.688 INFO:tasks.workunit.client.0.smithi027.stdout:5/938: mknod d19/d1b/d1c/d1e/d58/db4/c138 0 2024-04-20T10:59:28.688 INFO:tasks.workunit.client.0.smithi027.stdout:2/968: dread d5/dc/d16/d37/f6b [0,4194304] 0 2024-04-20T10:59:28.688 INFO:tasks.workunit.client.0.smithi027.stdout:2/969: chown d5/dc/d16/d2a 1559 1 2024-04-20T10:59:28.688 INFO:tasks.workunit.client.0.smithi027.stdout:9/887: truncate d4/d1b/d1f/d45/f71 729067 0 2024-04-20T10:59:28.689 INFO:tasks.workunit.client.0.smithi027.stdout:9/888: write d4/d1b/d1f/d5a/db7/f107 [482054,26178] 0 2024-04-20T10:59:28.690 INFO:tasks.workunit.client.0.smithi027.stdout:9/889: read d4/d1b/d50/d5d/f8c [256333,71618] 0 2024-04-20T10:59:28.690 INFO:tasks.workunit.client.0.smithi027.stdout:9/890: write d4/d1b/daf/d10a/d43/fc7 [332007,17940] 0 2024-04-20T10:59:28.698 INFO:tasks.workunit.client.0.smithi027.stdout:4/943: dwrite d0/d14/d6b/da3/f9e [0,4194304] 0 2024-04-20T10:59:28.698 INFO:tasks.workunit.client.0.smithi027.stdout:5/939: rmdir d19/d1b/d1c/d1e/d22/d68/d75/db7 39 2024-04-20T10:59:28.699 INFO:tasks.workunit.client.0.smithi027.stdout:4/944: stat d0/d5e/d6c/l80 0 2024-04-20T10:59:28.701 INFO:tasks.workunit.client.0.smithi027.stdout:4/945: mkdir d0/d48/d123/d33/d5c/d146 0 2024-04-20T10:59:28.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/940: link d19/d1b/d1c/dbd/dbe/fb6 d19/d1b/d1c/d1e/d83/f139 0 2024-04-20T10:59:28.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/941: read d19/d1b/d1c/fad [17092,33789] 0 2024-04-20T10:59:28.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/942: stat fb 0 2024-04-20T10:59:28.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/943: chown d19 4276186 1 2024-04-20T10:59:28.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/944: readlink d19/d1b/d1c/d1e/d22/d68/ld1 0 2024-04-20T10:59:28.703 INFO:tasks.workunit.client.0.smithi027.stdout:9/891: read d4/fc8 [5289258,9547] 0 2024-04-20T10:59:28.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/946: mknod d0/d48/d68/d132/c147 0 2024-04-20T10:59:28.704 INFO:tasks.workunit.client.0.smithi027.stdout:5/945: write d19/d1b/d1c/d1e/d22/ff5 [823293,96011] 0 2024-04-20T10:59:28.707 INFO:tasks.workunit.client.0.smithi027.stdout:5/946: dread d19/d1b/d1c/d1e/d58/f61 [0,4194304] 0 2024-04-20T10:59:28.707 INFO:tasks.workunit.client.0.smithi027.stdout:5/947: fdatasync d19/d1b/d1c/d1e/d22/db1/f137 0 2024-04-20T10:59:28.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/892: getdents d4/d1b/daf 0 2024-04-20T10:59:28.708 INFO:tasks.workunit.client.0.smithi027.stdout:9/893: fdatasync d4/d1b/f89 0 2024-04-20T10:59:28.709 INFO:tasks.workunit.client.0.smithi027.stdout:2/970: dwrite d5/d26/ffb [0,4194304] 0 2024-04-20T10:59:28.709 INFO:tasks.workunit.client.0.smithi027.stdout:5/948: creat d19/d1b/d1c/d1e/d22/db1/dcf/f13a x:0 0 0 2024-04-20T10:59:28.709 INFO:tasks.workunit.client.0.smithi027.stdout:2/971: fdatasync d5/d1e/f42 0 2024-04-20T10:59:28.710 INFO:tasks.workunit.client.0.smithi027.stdout:9/894: mknod d4/d1b/d1f/d57/db0/c10e 0 2024-04-20T10:59:28.717 INFO:tasks.workunit.client.0.smithi027.stdout:2/972: dread d5/dc/d16/d37/de2/d8e/fc6 [0,4194304] 0 2024-04-20T10:59:28.717 INFO:tasks.workunit.client.0.smithi027.stdout:2/973: write d5/dc/d16/d2a/d12b/fe9 [699417,29405] 0 2024-04-20T10:59:28.718 INFO:tasks.workunit.client.0.smithi027.stdout:9/895: creat d4/d1b/d50/d5d/f10f x:0 0 0 2024-04-20T10:59:28.720 INFO:tasks.workunit.client.0.smithi027.stdout:9/896: creat d4/d1b/d31/d23/d29/d7f/f110 x:0 0 0 2024-04-20T10:59:28.720 INFO:tasks.workunit.client.0.smithi027.stdout:9/897: fsync d4/fc8 0 2024-04-20T10:59:28.720 INFO:tasks.workunit.client.0.smithi027.stdout:9/898: write d4/d1b/d31/dd0/f10c [117029,54466] 0 2024-04-20T10:59:28.721 INFO:tasks.workunit.client.0.smithi027.stdout:2/974: mknod d5/dc/d16/dd0/c134 0 2024-04-20T10:59:28.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/981: sync 2024-04-20T10:59:28.722 INFO:tasks.workunit.client.0.smithi027.stdout:9/899: getdents d4/d1b/d31/dd0/dda 0 2024-04-20T10:59:28.722 INFO:tasks.workunit.client.0.smithi027.stdout:9/900: readlink d4/d1b/l9c 0 2024-04-20T10:59:28.722 INFO:tasks.workunit.client.0.smithi027.stdout:9/901: fdatasync d4/f7 0 2024-04-20T10:59:28.727 INFO:tasks.workunit.client.0.smithi027.stdout:5/949: dwrite f17 [8388608,4194304] 0 2024-04-20T10:59:28.739 INFO:tasks.workunit.client.0.smithi027.stdout:5/950: write d19/d1b/d1c/d1e/d25/d40/f81 [1802292,424] 0 2024-04-20T10:59:28.739 INFO:tasks.workunit.client.0.smithi027.stdout:5/951: truncate d19/d1b/d1c/d1e/d58/db4/f103 272856 0 2024-04-20T10:59:28.740 INFO:tasks.workunit.client.0.smithi027.stdout:7/982: truncate d4/d1b/d4a/d62/d87/f9b 3579985 0 2024-04-20T10:59:28.740 INFO:tasks.workunit.client.0.smithi027.stdout:5/952: link f2 d19/d1b/f13b 0 2024-04-20T10:59:28.741 INFO:tasks.workunit.client.0.smithi027.stdout:5/953: fdatasync d19/d1b/d1c/dbd/dbe/f129 0 2024-04-20T10:59:28.741 INFO:tasks.workunit.client.0.smithi027.stdout:2/975: dwrite d5/dc/d16/d37/de2/f128 [0,4194304] 0 2024-04-20T10:59:28.744 INFO:tasks.workunit.client.0.smithi027.stdout:2/976: symlink d5/dc/d16/d2a/d59/da4/l135 0 2024-04-20T10:59:28.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/954: rename d19/c44 to d19/d1b/d1c/d1e/d3c/c13c 0 2024-04-20T10:59:28.747 INFO:tasks.workunit.client.0.smithi027.stdout:9/902: dwrite d4/d1b/daf/d10a/d43/ffb [0,4194304] 0 2024-04-20T10:59:28.747 INFO:tasks.workunit.client.0.smithi027.stdout:2/977: mkdir d5/dc/d16/dd0/d136 0 2024-04-20T10:59:28.748 INFO:tasks.workunit.client.0.smithi027.stdout:5/955: mknod d19/d1b/d1c/d1e/d116/c13d 0 2024-04-20T10:59:28.752 INFO:tasks.workunit.client.0.smithi027.stdout:2/978: dread d5/d26/d47/f71 [0,4194304] 0 2024-04-20T10:59:28.752 INFO:tasks.workunit.client.0.smithi027.stdout:2/979: truncate d5/d26/f74 923849 0 2024-04-20T10:59:28.752 INFO:tasks.workunit.client.0.smithi027.stdout:2/980: chown d5/d44/f106 87 1 2024-04-20T10:59:28.752 INFO:tasks.workunit.client.0.smithi027.stdout:2/981: chown d5/dc/d16/d2a/d59/da4 4356 1 2024-04-20T10:59:28.752 INFO:tasks.workunit.client.0.smithi027.stdout:5/956: creat d19/d1b/d1c/d1e/d22/db1/dcf/f13e x:0 0 0 2024-04-20T10:59:28.754 INFO:tasks.workunit.client.0.smithi027.stdout:2/982: creat d5/d26/d78/f137 x:0 0 0 2024-04-20T10:59:28.755 INFO:tasks.workunit.client.0.smithi027.stdout:5/957: rename f3 to d19/d1b/d1c/dbd/f13f 0 2024-04-20T10:59:28.756 INFO:tasks.workunit.client.0.smithi027.stdout:2/983: rmdir d5/dc 39 2024-04-20T10:59:28.757 INFO:tasks.workunit.client.0.smithi027.stdout:5/958: mknod d19/d1b/d1c/d1e/da3/c140 0 2024-04-20T10:59:28.757 INFO:tasks.workunit.client.0.smithi027.stdout:5/959: write d19/d1b/d1c/d1e/d22/d68/d75/de9/f134 [866454,109206] 0 2024-04-20T10:59:28.757 INFO:tasks.workunit.client.0.smithi027.stdout:2/984: truncate d5/dbc/fd2 225002 0 2024-04-20T10:59:28.758 INFO:tasks.workunit.client.0.smithi027.stdout:5/960: creat d19/d1b/d1c/d1e/d25/de5/f141 x:0 0 0 2024-04-20T10:59:28.758 INFO:tasks.workunit.client.0.smithi027.stdout:5/961: chown d19/d1b/d1c/l2e 9976362 1 2024-04-20T10:59:28.774 INFO:tasks.workunit.client.0.smithi027.stdout:5/962: dwrite d19/d1b/d1c/d1e/d22/db1/dcf/d91/f123 [0,4194304] 0 2024-04-20T10:59:28.775 INFO:tasks.workunit.client.0.smithi027.stdout:2/985: mkdir d5/d138 0 2024-04-20T10:59:28.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/963: creat d19/d1b/d1c/dbd/f142 x:0 0 0 2024-04-20T10:59:28.779 INFO:tasks.workunit.client.0.smithi027.stdout:2/986: creat d5/d138/f139 x:0 0 0 2024-04-20T10:59:28.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/964: mkdir d19/d1b/d1c/d1e/d22/d68/d143 0 2024-04-20T10:59:28.780 INFO:tasks.workunit.client.0.smithi027.stdout:2/987: mkdir d5/dc/d16/d37/de2/dac/d8f/d13a 0 2024-04-20T10:59:28.784 INFO:tasks.workunit.client.0.smithi027.stdout:2/988: dread d5/d1e/f42 [0,4194304] 0 2024-04-20T10:59:28.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/989: dread d5/dc/d16/dd0/f117 [0,4194304] 0 2024-04-20T10:59:28.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/990: chown d5/dc/d16/d37/f3f 86 1 2024-04-20T10:59:28.787 INFO:tasks.workunit.client.0.smithi027.stdout:5/965: mkdir d19/d1b/d1c/d1e/d83/d144 0 2024-04-20T10:59:28.802 INFO:tasks.workunit.client.0.smithi027.stdout:5/966: dwrite d19/d1b/d1c/d1e/d22/d68/d75/de9/ff0 [0,4194304] 0 2024-04-20T10:59:28.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/967: truncate d19/d1b/d1c/d1e/d22/db1/f124 745496 0 2024-04-20T10:59:28.808 INFO:tasks.workunit.client.0.smithi027.stdout:2/991: mknod d5/d26/d30/c13b 0 2024-04-20T10:59:28.808 INFO:tasks.workunit.client.0.smithi027.stdout:4/947: sync 2024-04-20T10:59:28.810 INFO:tasks.workunit.client.0.smithi027.stdout:2/992: dread d5/dc/d16/d2a/d59/fad [0,4194304] 0 2024-04-20T10:59:28.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/993: write d5/d44/f106 [4701644,107993] 0 2024-04-20T10:59:28.819 INFO:tasks.workunit.client.0.smithi027.stdout:2/994: stat d5/d44/d7c/l108 0 2024-04-20T10:59:28.821 INFO:tasks.workunit.client.0.smithi027.stdout:5/968: dwrite d19/d1b/d1c/d1e/d22/d68/d75/db7/f102 [0,4194304] 0 2024-04-20T10:59:28.821 INFO:tasks.workunit.client.0.smithi027.stdout:5/969: truncate d19/d1b/d1c/d1e/d25/d40/d82/fae 1610715 0 2024-04-20T10:59:28.823 INFO:tasks.workunit.client.0.smithi027.stdout:2/995: rename d5/d26/d30/f3c to d5/dc/d16/d37/de2/dac/f13c 0 2024-04-20T10:59:28.827 INFO:tasks.workunit.client.0.smithi027.stdout:5/970: write d19/d1b/f13b [1081212,83007] 0 2024-04-20T10:59:28.828 INFO:tasks.workunit.client.0.smithi027.stdout:2/996: rename d5/cd1 to d5/dc/d16/d37/de2/dac/d8f/d13a/c13d 0 2024-04-20T10:59:28.834 INFO:tasks.workunit.client.0.smithi027.stdout:2/997: read d5/dc/d16/d32/d109/f104 [2771019,17242] 0 2024-04-20T10:59:28.835 INFO:tasks.workunit.client.0.smithi027.stdout:7/983: sync 2024-04-20T10:59:28.835 INFO:tasks.workunit.client.0.smithi027.stdout:7/984: write d4/d5/d8/d3b/f74 [2530099,112053] 0 2024-04-20T10:59:28.835 INFO:tasks.workunit.client.0.smithi027.stdout:7/985: chown d4/d5/d8/d21/fbb 2012149 1 2024-04-20T10:59:28.836 INFO:tasks.workunit.client.0.smithi027.stdout:5/971: mkdir d19/d1b/d1c/d145 0 2024-04-20T10:59:28.837 INFO:tasks.workunit.client.0.smithi027.stdout:7/986: rmdir d4/d5/d8/d46/d77/d7c 39 2024-04-20T10:59:28.838 INFO:tasks.workunit.client.0.smithi027.stdout:5/972: creat d19/d1b/d1c/d1e/d25/f146 x:0 0 0 2024-04-20T10:59:28.861 INFO:tasks.workunit.client.0.smithi027.stdout:7/987: rmdir d4/d5/d81 39 2024-04-20T10:59:28.862 INFO:tasks.workunit.client.0.smithi027.stdout:5/973: truncate d19/d1b/f8d 34798 0 2024-04-20T10:59:28.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/948: sync 2024-04-20T10:59:28.868 INFO:tasks.workunit.client.0.smithi027.stdout:9/903: sync 2024-04-20T10:59:28.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/949: write d0/d48/d123/d33/d5c/d6f/d7b/f104 [757142,127027] 0 2024-04-20T10:59:28.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/950: write d0/d48/d123/f30 [2732966,92671] 0 2024-04-20T10:59:28.869 INFO:tasks.workunit.client.0.smithi027.stdout:4/951: chown d0/d48/d123/d33/d5c/c129 17955875 1 2024-04-20T10:59:28.869 INFO:tasks.workunit.client.0.smithi027.stdout:7/988: mknod d4/d5/c13b 0 2024-04-20T10:59:28.875 INFO:tasks.workunit.client.0.smithi027.stdout:9/904: dread d4/d1b/d50/d5d/fb1 [0,4194304] 0 2024-04-20T10:59:28.875 INFO:tasks.workunit.client.0.smithi027.stdout:9/905: stat d4/d1b/daf/d10a/d43/ffb 0 2024-04-20T10:59:28.875 INFO:tasks.workunit.client.0.smithi027.stdout:9/906: write d4/d1b/d1f/d45/d8a/db2/fbb [1537477,116784] 0 2024-04-20T10:59:28.875 INFO:tasks.workunit.client.0.smithi027.stdout:9/907: chown d4/d2a/f69 121910199 1 2024-04-20T10:59:28.879 INFO:tasks.workunit.client.0.smithi027.stdout:7/989: write d4/d5/d8/d3b/f3e [4093572,82606] 0 2024-04-20T10:59:28.879 INFO:tasks.workunit.client.0.smithi027.stdout:7/990: chown d4/d5/d8/d46/d106/f112 3755280 1 2024-04-20T10:59:28.880 INFO:tasks.workunit.client.0.smithi027.stdout:5/974: symlink d19/d1b/d1c/d1e/d22/d68/d75/l147 0 2024-04-20T10:59:28.880 INFO:tasks.workunit.client.0.smithi027.stdout:4/952: rmdir d0/d48/d68/d76/dfc 39 2024-04-20T10:59:28.881 INFO:tasks.workunit.client.0.smithi027.stdout:7/991: getdents d4/d5/d8/d127 0 2024-04-20T10:59:28.894 INFO:tasks.workunit.client.0.smithi027.stdout:9/908: dwrite d4/d1b/d31/dd0/dda/fe6 [0,4194304] 0 2024-04-20T10:59:28.896 INFO:tasks.workunit.client.0.smithi027.stdout:9/909: dread d4/d1b/daf/d10a/d43/fc7 [0,4194304] 0 2024-04-20T10:59:28.897 INFO:tasks.workunit.client.0.smithi027.stdout:5/975: mkdir d19/d1b/d1c/d1e/d22/db1/dcf/d91/d148 0 2024-04-20T10:59:28.897 INFO:tasks.workunit.client.0.smithi027.stdout:4/953: symlink d0/d48/d123/dc3/d11f/l148 0 2024-04-20T10:59:28.897 INFO:tasks.workunit.client.0.smithi027.stdout:4/954: chown d0/d14/d71/c90 347787685 1 2024-04-20T10:59:28.897 INFO:tasks.workunit.client.0.smithi027.stdout:5/976: dread - d19/d1b/d1c/d1e/d22/dca/ffc zero size 2024-04-20T10:59:28.897 INFO:tasks.workunit.client.0.smithi027.stdout:5/977: chown d19/d1b/d1c/d1e/d25/d40/d82 503093 1 2024-04-20T10:59:28.909 INFO:tasks.workunit.client.0.smithi027.stdout:7/992: dwrite d4/d5/d8/d21/d94/da7/dc2/f126 [0,4194304] 0 2024-04-20T10:59:28.909 INFO:tasks.workunit.client.0.smithi027.stdout:7/993: fsync d4/d1b/d4a/d62/f10b 0 2024-04-20T10:59:28.909 INFO:tasks.workunit.client.0.smithi027.stdout:7/994: write d4/d5/d8/d46/f8f [5783074,11804] 0 2024-04-20T10:59:28.909 INFO:tasks.workunit.client.0.smithi027.stdout:7/995: chown d4/d5/d8/d3b/cc9 590350 1 2024-04-20T10:59:28.909 INFO:tasks.workunit.client.0.smithi027.stdout:7/996: readlink d4/d5/d8/l2d 0 2024-04-20T10:59:28.910 INFO:tasks.workunit.client.0.smithi027.stdout:4/955: mkdir d0/d14/d6b/da3/d53/d6a/d149 0 2024-04-20T10:59:28.912 INFO:tasks.workunit.client.0.smithi027.stdout:2/998: sync 2024-04-20T10:59:28.929 INFO:tasks.workunit.client.0.smithi027.stdout:9/910: dwrite d4/d1b/d1f/fc1 [0,4194304] 0 2024-04-20T10:59:28.929 INFO:tasks.workunit.client.0.smithi027.stdout:2/999: dwrite d5/dc/d16/d37/de2/dac/d8f/dfd/f107 [0,4194304] 0 2024-04-20T10:59:28.930 INFO:tasks.workunit.client.0.smithi027.stdout:7/997: creat d4/d5/d8/d21/d94/da7/dce/f13c x:0 0 0 2024-04-20T10:59:28.931 INFO:tasks.workunit.client.0.smithi027.stdout:4/956: unlink d0/d14/d6b/d92/dcd/f138 0 2024-04-20T10:59:28.934 INFO:tasks.workunit.client.0.smithi027.stdout:7/998: unlink d4/d1b/f72 0 2024-04-20T10:59:28.935 INFO:tasks.workunit.client.0.smithi027.stdout:4/957: mkdir d0/d48/d68/d76/dfc/d14a 0 2024-04-20T10:59:28.950 INFO:tasks.workunit.client.0.smithi027.stdout:4/958: dwrite d0/d48/d123/d33/d5c/d6f/d7b/f104 [0,4194304] 0 2024-04-20T10:59:28.951 INFO:tasks.workunit.client.0.smithi027.stdout:9/911: link d4/d1b/d1f/d45/d8a/fd9 d4/d2a/f111 0 2024-04-20T10:59:28.952 INFO:tasks.workunit.client.0.smithi027.stdout:7/999: link d4/d5/d8/d45/ff7 d4/d5/d8/d46/d106/f13d 0 2024-04-20T10:59:28.969 INFO:tasks.workunit.client.0.smithi027.stdout:4/959: mkdir d0/d14/d6b/d14b 0 2024-04-20T10:59:28.969 INFO:tasks.workunit.client.0.smithi027.stdout:4/960: dread - d0/d48/d123/d33/d5c/f135 zero size 2024-04-20T10:59:28.970 INFO:tasks.workunit.client.0.smithi027.stdout:9/912: creat d4/d1b/daf/d10a/d43/f112 x:0 0 0 2024-04-20T10:59:28.974 INFO:tasks.workunit.client.0.smithi027.stdout:5/978: sync 2024-04-20T10:59:28.974 INFO:tasks.workunit.client.0.smithi027.stdout:5/979: readlink d19/l8b 0 2024-04-20T10:59:28.975 INFO:tasks.workunit.client.0.smithi027.stdout:4/961: rename d0/d48/d123/d2f/f67 to d0/d14/d71/dfd/f14c 0 2024-04-20T10:59:28.975 INFO:tasks.workunit.client.0.smithi027.stdout:4/962: fdatasync d0/d5e/d6c/fbc 0 2024-04-20T10:59:28.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/913: mkdir d4/d1b/d1f/d45/d8a/d113 0 2024-04-20T10:59:28.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/914: fdatasync d4/d1b/d1f/d45/f71 0 2024-04-20T10:59:28.976 INFO:tasks.workunit.client.0.smithi027.stdout:5/980: mkdir d19/d10d/d149 0 2024-04-20T10:59:28.976 INFO:tasks.workunit.client.0.smithi027.stdout:5/981: write d19/d1b/d1c/d1e/d22/dca/f107 [145442,118553] 0 2024-04-20T10:59:28.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/915: write d4/d1b/daf/d10a/d43/fc7 [244351,74535] 0 2024-04-20T10:59:28.979 INFO:tasks.workunit.client.0.smithi027.stdout:4/963: symlink d0/d14/d71/dfd/l14d 0 2024-04-20T10:59:28.979 INFO:tasks.workunit.client.0.smithi027.stdout:4/964: fsync d0/d48/d123/d33/d5c/d6f/fa9 0 2024-04-20T10:59:28.979 INFO:tasks.workunit.client.0.smithi027.stdout:5/982: getdents d19/d1b/d1c/d1e 0 2024-04-20T10:59:28.979 INFO:tasks.workunit.client.0.smithi027.stdout:5/983: write d19/d1b/f13b [2019019,35125] 0 2024-04-20T10:59:28.980 INFO:tasks.workunit.client.0.smithi027.stdout:5/984: dread - d19/d1b/d1c/d1e/d25/f131 zero size 2024-04-20T10:59:28.998 INFO:tasks.workunit.client.0.smithi027.stdout:4/965: dwrite d0/d48/d123/d2f/f82 [0,4194304] 0 2024-04-20T10:59:29.036 INFO:tasks.workunit.client.0.smithi027.stdout:5/985: getdents d19/d1b/d1c/d1e/d83 0 2024-04-20T10:59:29.037 INFO:tasks.workunit.client.0.smithi027.stdout:4/966: truncate d0/d14/fc4 1117061 0 2024-04-20T10:59:29.037 INFO:tasks.workunit.client.0.smithi027.stdout:4/967: chown d0/d48/d123/d33/d5c/f98 2 1 2024-04-20T10:59:29.038 INFO:tasks.workunit.client.0.smithi027.stdout:5/986: read d19/d1b/d1c/d1e/d58/d9d/fce [120697,83814] 0 2024-04-20T10:59:29.043 INFO:tasks.workunit.client.0.smithi027.stdout:9/916: sync 2024-04-20T10:59:29.044 INFO:tasks.workunit.client.0.smithi027.stdout:4/968: write d0/d14/d71/dfd/f10f [1037037,60117] 0 2024-04-20T10:59:29.044 INFO:tasks.workunit.client.0.smithi027.stdout:4/969: read - d0/d5e/d6c/fec zero size 2024-04-20T10:59:29.044 INFO:tasks.workunit.client.0.smithi027.stdout:5/987: truncate d19/d1b/d1c/d1e/d3c/faa 749906 0 2024-04-20T10:59:29.045 INFO:tasks.workunit.client.0.smithi027.stdout:9/917: symlink d4/d1b/d1f/d45/d8a/db2/l114 0 2024-04-20T10:59:29.045 INFO:tasks.workunit.client.0.smithi027.stdout:9/918: dread - d4/d1b/d50/ff6 zero size 2024-04-20T10:59:29.046 INFO:tasks.workunit.client.0.smithi027.stdout:4/970: mknod d0/d14/d71/dfd/c14e 0 2024-04-20T10:59:29.046 INFO:tasks.workunit.client.0.smithi027.stdout:4/971: chown d0/l46 1 1 2024-04-20T10:59:29.046 INFO:tasks.workunit.client.0.smithi027.stdout:4/972: chown d0/d48/d123/d2f/l63 38766 1 2024-04-20T10:59:29.047 INFO:tasks.workunit.client.0.smithi027.stdout:5/988: getdents d19/d1b/d1c/d1e/d22/db1/dcf/d91 0 2024-04-20T10:59:29.048 INFO:tasks.workunit.client.0.smithi027.stdout:5/989: dread d19/d1b/d1c/d1e/f79 [0,4194304] 0 2024-04-20T10:59:29.048 INFO:tasks.workunit.client.0.smithi027.stdout:5/990: write d19/d1b/d1c/d1e/d22/db1/dcf/f13e [302451,62103] 0 2024-04-20T10:59:29.048 INFO:tasks.workunit.client.0.smithi027.stdout:9/919: mknod d4/d1b/d1f/d45/d9e/c115 0 2024-04-20T10:59:29.049 INFO:tasks.workunit.client.0.smithi027.stdout:4/973: rmdir d0/d14/d6b/d92/d11e 39 2024-04-20T10:59:29.049 INFO:tasks.workunit.client.0.smithi027.stdout:4/974: fsync d0/d14/d6b/da3/f2e 0 2024-04-20T10:59:29.065 INFO:tasks.workunit.client.0.smithi027.stdout:4/975: dwrite d0/d14/d6b/da3/d1a/feb [4194304,4194304] 0 2024-04-20T10:59:29.065 INFO:tasks.workunit.client.0.smithi027.stdout:4/976: chown d0/d14/d6b/f125 17176359 1 2024-04-20T10:59:29.066 INFO:tasks.workunit.client.0.smithi027.stdout:5/991: truncate d19/d1b/d1c/d1e/d25/f2f 245105 0 2024-04-20T10:59:29.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/920: symlink d4/d1b/daf/d10a/da8/l116 0 2024-04-20T10:59:29.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/921: truncate d4/d1b/d31/f67 2386055 0 2024-04-20T10:59:29.070 INFO:tasks.workunit.client.0.smithi027.stdout:5/992: symlink d19/d1b/d1c/d1e/d22/d68/l14a 0 2024-04-20T10:59:29.071 INFO:tasks.workunit.client.0.smithi027.stdout:9/922: mknod d4/d1b/d31/dd0/dda/c117 0 2024-04-20T10:59:29.071 INFO:tasks.workunit.client.0.smithi027.stdout:9/923: read - d4/d1b/d31/fdc zero size 2024-04-20T10:59:29.091 INFO:tasks.workunit.client.0.smithi027.stdout:9/924: dwrite d4/d1b/d1f/d45/d8a/f6e [0,4194304] 0 2024-04-20T10:59:29.091 INFO:tasks.workunit.client.0.smithi027.stdout:9/925: write d4/d1b/d31/f67 [2367808,36483] 0 2024-04-20T10:59:29.121 INFO:tasks.workunit.client.0.smithi027.stdout:5/993: rename d19/d1b/d1c/d1e/f88 to d19/d1b/d1c/d1e/d22/db1/dcf/d91/f14b 0 2024-04-20T10:59:29.121 INFO:tasks.workunit.client.0.smithi027.stdout:5/994: write d19/d1b/d1c/d1e/d25/f2f [346588,21366] 0 2024-04-20T10:59:29.126 INFO:tasks.workunit.client.0.smithi027.stdout:4/977: sync 2024-04-20T10:59:29.126 INFO:tasks.workunit.client.0.smithi027.stdout:4/978: truncate d0/d48/d123/d33/d5c/f135 775000 0 2024-04-20T10:59:29.126 INFO:tasks.workunit.client.0.smithi027.stdout:4/979: fdatasync d0/d48/d123/d2f/ff6 0 2024-04-20T10:59:29.126 INFO:tasks.workunit.client.0.smithi027.stdout:4/980: write d0/d48/d123/d33/d97/fca [843989,108016] 0 2024-04-20T10:59:29.131 INFO:tasks.workunit.client.0.smithi027.stdout:5/995: write d19/d1b/d1c/d1e/d3c/f31 [8020156,10033] 0 2024-04-20T10:59:29.131 INFO:tasks.workunit.client.0.smithi027.stdout:5/996: fsync d19/d1b/d1c/d1e/d22/d68/f109 0 2024-04-20T10:59:29.131 INFO:tasks.workunit.client.0.smithi027.stdout:5/997: truncate d19/d1b/d1c/d1e/d25/d40/d82/f108 990674 0 2024-04-20T10:59:29.131 INFO:tasks.workunit.client.0.smithi027.stdout:5/998: chown d19/d1b/d1c/d1e/d22/d68/d75/de9/f134 57 1 2024-04-20T10:59:29.132 INFO:tasks.workunit.client.0.smithi027.stdout:9/926: rename d4/d1b/d31/d51/ce3 to d4/d1b/daf/d10a/d43/c118 0 2024-04-20T10:59:29.145 INFO:tasks.workunit.client.0.smithi027.stdout:4/981: dwrite d0/d14/d6b/da3/d1a/dae/fb5 [0,4194304] 0 2024-04-20T10:59:29.146 INFO:tasks.workunit.client.0.smithi027.stdout:5/999: symlink d19/d1b/d1c/d1e/d22/d68/d143/l14c 0 2024-04-20T10:59:29.146 INFO:tasks.workunit.client.0.smithi027.stdout:9/927: creat d4/d1b/daf/d10a/da8/f119 x:0 0 0 2024-04-20T10:59:29.146 INFO:tasks.workunit.client.0.smithi027.stdout:9/928: chown d4/d1b/d31/d23/d29/f35 14455193 1 2024-04-20T10:59:29.150 INFO:tasks.workunit.client.0.smithi027.stdout:4/982: read d0/d14/d6b/da3/d1a/d3b/fc8 [3461007,62108] 0 2024-04-20T10:59:29.163 INFO:tasks.workunit.client.0.smithi027.stdout:9/929: dwrite d4/d1b/d1f/d45/f7d [0,4194304] 0 2024-04-20T10:59:29.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.821+0000 7f9374cba640 1 Processor -- start 2024-04-20T10:59:29.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.821+0000 7f9374cba640 1 -- start start 2024-04-20T10:59:29.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.822+0000 7f9374cba640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f93700b54f0 0x7f93700b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:29.822 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.822+0000 7f9374cba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93700d0c90 0x7f93700d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.822+0000 7f9374cba640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93700b5e70 con 0x7f9370155950 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.822+0000 7f9374cba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f937015b9c0 con 0x7f93700d0c90 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.822+0000 7f9374cba640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f937015bca0 con 0x7f93700b54f0 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.822+0000 7f936ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93700d0c90 0x7f93700d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.822+0000 7f936ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93700d0c90 0x7f93700d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39472/0 (socket says 172.21.15.27:39472) 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.822+0000 7f936ed9d640 1 -- 172.21.15.27:0/2457583539 learned_addr learned my addr 172.21.15.27:0/2457583539 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.822+0000 7f936dd9b640 1 --2- 172.21.15.27:0/2457583539 >> v2:172.21.15.137:3300/0 conn(0x7f93700b54f0 0x7f93700b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.823+0000 7f936ed9d640 1 -- 172.21.15.27:0/2457583539 >> v2:172.21.15.137:3300/0 conn(0x7f93700b54f0 msgr2=0x7f93700b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.823+0000 7f936ed9d640 1 --2- 172.21.15.27:0/2457583539 >> v2:172.21.15.137:3300/0 conn(0x7f93700b54f0 0x7f93700b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:29.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.823+0000 7f936ed9d640 1 -- 172.21.15.27:0/2457583539 >> v1:172.21.15.137:6789/0 conn(0x7f9370155950 legacy=0x7f93700cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:29.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.823+0000 7f936ed9d640 1 -- 172.21.15.27:0/2457583539 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937015bf80 con 0x7f93700d0c90 2024-04-20T10:59:29.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.823+0000 7f936dd9b640 1 --2- 172.21.15.27:0/2457583539 >> v2:172.21.15.137:3300/0 conn(0x7f93700b54f0 0x7f93700b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:59:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.823+0000 7f936ed9d640 1 --2- 172.21.15.27:0/2457583539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700d0c90 0x7f93700d10a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93700b44e0 tx=0x7f936002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cf9efb1a61cc3b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.823+0000 7f936d59a640 1 -- 172.21.15.27:0/2457583539 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936002cbb0 con 0x7f93700d0c90 2024-04-20T10:59:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f936d59a640 1 -- 172.21.15.27:0/2457583539 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936002cd50 con 0x7f93700d0c90 2024-04-20T10:59:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f936d59a640 1 -- 172.21.15.27:0/2457583539 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360034610 con 0x7f93700d0c90 2024-04-20T10:59:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f9374cba640 1 -- 172.21.15.27:0/2457583539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700d0c90 msgr2=0x7f93700d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:29.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f9374cba640 1 --2- 172.21.15.27:0/2457583539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700d0c90 0x7f93700d10a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93700b44e0 tx=0x7f936002c010 comp rx=0 tx=0).stop 2024-04-20T10:59:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f9374cba640 1 -- 172.21.15.27:0/2457583539 shutdown_connections 2024-04-20T10:59:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f9374cba640 1 --2- 172.21.15.27:0/2457583539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700d0c90 0x7f93700d10a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f9374cba640 1 --2- 172.21.15.27:0/2457583539 >> v2:172.21.15.137:3300/0 conn(0x7f93700b54f0 0x7f93700b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f9374cba640 1 -- 172.21.15.27:0/2457583539 >> 172.21.15.27:0/2457583539 conn(0x7f93700676c0 msgr2=0x7f9370067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f9374cba640 1 -- 172.21.15.27:0/2457583539 shutdown_connections 2024-04-20T10:59:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.824+0000 7f9374cba640 1 -- 172.21.15.27:0/2457583539 wait complete. 2024-04-20T10:59:29.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f9374cba640 1 Processor -- start 2024-04-20T10:59:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f9374cba640 1 -- start start 2024-04-20T10:59:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f9374cba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700b54f0 0x7f93701f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f9374cba640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93700d0c90 0x7f937015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f9374cba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9370155950 0x7f937014e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:29.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f9374cba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93700d15e0 con 0x7f93700b54f0 2024-04-20T10:59:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f9374cba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f937015c540 con 0x7f9370155950 2024-04-20T10:59:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f9374cba640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f937015fb90 con 0x7f93700d0c90 2024-04-20T10:59:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f936e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700b54f0 0x7f93701f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f936e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700b54f0 0x7f93701f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39480/0 (socket says 172.21.15.27:39480) 2024-04-20T10:59:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.826+0000 7f936e59c640 1 -- 172.21.15.27:0/1160691380 learned_addr learned my addr 172.21.15.27:0/1160691380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f936dd9b640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93700d0c90 0x7f937015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:29.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f936ed9d640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9370155950 0x7f937014e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f936e59c640 1 -- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93700d0c90 msgr2=0x7f937015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f936e59c640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93700d0c90 0x7f937015aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f936e59c640 1 -- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9370155950 msgr2=0x7f937014e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f936e59c640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9370155950 0x7f937014e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f936e59c640 1 -- 172.21.15.27:0/1160691380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937015b650 con 0x7f93700b54f0 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f936e59c640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700b54f0 0x7f93701f2180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f936800c430 tx=0x7f936800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f9366ffd640 1 -- 172.21.15.27:0/1160691380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93680052d0 con 0x7f93700b54f0 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f9366ffd640 1 -- 172.21.15.27:0/1160691380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9368005490 con 0x7f93700b54f0 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93701e86a0 con 0x7f93700b54f0 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f9366ffd640 1 -- 172.21.15.27:0/1160691380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368007260 con 0x7f93700b54f0 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.827+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937015cec0 con 0x7f93700b54f0 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.828+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93700cb350 con 0x7f93700b54f0 2024-04-20T10:59:29.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.829+0000 7f9366ffd640 1 -- 172.21.15.27:0/1160691380 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9368007400 con 0x7f93700b54f0 2024-04-20T10:59:29.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.829+0000 7f9366ffd640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f935006c920 0x7f935006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:29.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.829+0000 7f9366ffd640 1 -- 172.21.15.27:0/1160691380 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f936808b120 con 0x7f93700b54f0 2024-04-20T10:59:29.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.829+0000 7f936dd9b640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f935006c920 0x7f935006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:29.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.830+0000 7f936dd9b640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f935006c920 0x7f935006ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93700cfef0 tx=0x7f9358008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:29.834 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:29.833+0000 7f9366ffd640 1 -- 172.21.15.27:0/1160691380 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f93700cb350 con 0x7f93700b54f0 2024-04-20T10:59:30.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.059+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f937004fcf0 con 0x7f93700b54f0 2024-04-20T10:59:30.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.060+0000 7f9366ffd640 1 -- 172.21.15.27:0/1160691380 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6933 (secure 0 0 0) 0x7f9368058de0 con 0x7f93700b54f0 2024-04-20T10:59:30.060 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:30.060 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5029,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:59:30.060 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f935006c920 msgr2=0x7f935006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f935006c920 0x7f935006ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93700cfef0 tx=0x7f9358008040 comp rx=0 tx=0).stop 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700b54f0 msgr2=0x7f93701f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700b54f0 0x7f93701f2180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f936800c430 tx=0x7f936800c8f0 comp rx=0 tx=0).stop 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 shutdown_connections 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f935006c920 0x7f935006ede0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9370155950 0x7f937014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93700d0c90 0x7f937015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 --2- 172.21.15.27:0/1160691380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700b54f0 0x7f93701f2180 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 >> 172.21.15.27:0/1160691380 conn(0x7f93700676c0 msgr2=0x7f937006d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 shutdown_connections 2024-04-20T10:59:30.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:30.062+0000 7f9374cba640 1 -- 172.21.15.27:0/1160691380 wait complete. 2024-04-20T10:59:30.073 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T10:59:32.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:32.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.187+0000 7fb9b7843640 1 Processor -- start 2024-04-20T10:59:32.187 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.187+0000 7fb9b7843640 1 -- start start 2024-04-20T10:59:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b7843640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb9b015b580 0x7fb9b0070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b7843640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b00c9e60 con 0x7fb9b015f580 2024-04-20T10:59:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b7843640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b006cce0 con 0x7fb9b01556f0 2024-04-20T10:59:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b7843640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fb9b006ead0 con 0x7fb9b015b580 2024-04-20T10:59:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b4dde640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb9b015b580 0x7fb9b0070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b5de0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb9b015f580 0x7fb9b00cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52464/0 (socket says 172.21.15.27:52464) 2024-04-20T10:59:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b5de0640 1 -- 172.21.15.27:0/1433673597 learned_addr learned my addr 172.21.15.27:0/1433673597 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b4dde640 1 -- 172.21.15.27:0/1433673597 >> v1:172.21.15.137:6789/0 conn(0x7fb9b01556f0 legacy=0x7fb9b00714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b4dde640 1 -- 172.21.15.27:0/1433673597 >> v1:172.21.15.27:6789/0 conn(0x7fb9b015f580 legacy=0x7fb9b00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9a7fff640 1 -- 172.21.15.27:0/1433673597 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209732267 0 0) 0x7fb9b00c9e60 con 0x7fb9b015f580 2024-04-20T10:59:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.188+0000 7fb9b4dde640 1 -- 172.21.15.27:0/1433673597 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b00c98a0 con 0x7fb9b015b580 2024-04-20T10:59:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.189+0000 7fb9b4dde640 1 --2- 172.21.15.27:0/1433673597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 0x7fb9b0070ca0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00097a0 tx=0x7fb9a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb3050ed159e94b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.189+0000 7fb9a7fff640 1 -- 172.21.15.27:0/1433673597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0035020 con 0x7fb9b015b580 2024-04-20T10:59:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.189+0000 7fb9a7fff640 1 -- 172.21.15.27:0/1433673597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a0031830 con 0x7fb9b015b580 2024-04-20T10:59:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.190+0000 7fb9a7fff640 1 -- 172.21.15.27:0/1433673597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0031c00 con 0x7fb9b015b580 2024-04-20T10:59:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.190+0000 7fb9b7843640 1 -- 172.21.15.27:0/1433673597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 msgr2=0x7fb9b0070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.190+0000 7fb9b7843640 1 --2- 172.21.15.27:0/1433673597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 0x7fb9b0070ca0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00097a0 tx=0x7fb9a002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:59:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.190+0000 7fb9b7843640 1 -- 172.21.15.27:0/1433673597 shutdown_connections 2024-04-20T10:59:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.190+0000 7fb9b7843640 1 --2- 172.21.15.27:0/1433673597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 0x7fb9b0070ca0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.190+0000 7fb9b7843640 1 -- 172.21.15.27:0/1433673597 >> 172.21.15.27:0/1433673597 conn(0x7fb9b0067ae0 msgr2=0x7fb9b0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.190+0000 7fb9b7843640 1 -- 172.21.15.27:0/1433673597 shutdown_connections 2024-04-20T10:59:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.190+0000 7fb9b7843640 1 -- 172.21.15.27:0/1433673597 wait complete. 2024-04-20T10:59:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.191+0000 7fb9b7843640 1 Processor -- start 2024-04-20T10:59:32.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.191+0000 7fb9b7843640 1 -- start start 2024-04-20T10:59:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b7843640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b01556f0 0x7fb9b01531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b7843640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 0x7fb9b01536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b7843640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9b015f580 0x7fb9b014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b7843640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b006e480 con 0x7fb9b015b580 2024-04-20T10:59:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b7843640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b015b9c0 con 0x7fb9b01556f0 2024-04-20T10:59:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b7843640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b006e760 con 0x7fb9b015f580 2024-04-20T10:59:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b4dde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 0x7fb9b01536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b4dde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 0x7fb9b01536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39502/0 (socket says 172.21.15.27:39502) 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b4dde640 1 -- 172.21.15.27:0/1396004580 learned_addr learned my addr 172.21.15.27:0/1396004580 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b5de0640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9b015f580 0x7fb9b014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.192+0000 7fb9b55df640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b01556f0 0x7fb9b01531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9b4dde640 1 -- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9b015f580 msgr2=0x7fb9b014e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9b4dde640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9b015f580 0x7fb9b014e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9b4dde640 1 -- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b01556f0 msgr2=0x7fb9b01531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9b4dde640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b01556f0 0x7fb9b01531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9b4dde640 1 -- 172.21.15.27:0/1396004580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b0151db0 con 0x7fb9b015b580 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9b5de0640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9b015f580 0x7fb9b014e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9b4dde640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 0x7fb9b01536e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00098c0 tx=0x7fb9a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9a5ffb640 1 -- 172.21.15.27:0/1396004580 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0040070 con 0x7fb9b015b580 2024-04-20T10:59:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9a5ffb640 1 -- 172.21.15.27:0/1396004580 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a0039a00 con 0x7fb9b015b580 2024-04-20T10:59:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9a5ffb640 1 -- 172.21.15.27:0/1396004580 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0035040 con 0x7fb9b015b580 2024-04-20T10:59:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b0150460 con 0x7fb9b015b580 2024-04-20T10:59:32.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.193+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b0152490 con 0x7fb9b015b580 2024-04-20T10:59:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.194+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b00688e0 con 0x7fb9b015b580 2024-04-20T10:59:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.195+0000 7fb9a5ffb640 1 -- 172.21.15.27:0/1396004580 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9a0038020 con 0x7fb9b015b580 2024-04-20T10:59:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.195+0000 7fb9a5ffb640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb99406c920 0x7fb99406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.195+0000 7fb9a5ffb640 1 -- 172.21.15.27:0/1396004580 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7fb9a00ad680 con 0x7fb9b015b580 2024-04-20T10:59:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.195+0000 7fb9b55df640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb99406c920 0x7fb99406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:32.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.196+0000 7fb9b55df640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb99406c920 0x7fb99406ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac000f60 tx=0x7fb9ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:32.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.199+0000 7fb9a5ffb640 1 -- 172.21.15.27:0/1396004580 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb9a007b990 con 0x7fb9b015b580 2024-04-20T10:59:32.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.425+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9b004fcf0 con 0x7fb9b015b580 2024-04-20T10:59:32.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.426+0000 7fb9a5ffb640 1 -- 172.21.15.27:0/1396004580 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6933 (secure 0 0 0) 0x7fb9a007b290 con 0x7fb9b015b580 2024-04-20T10:59:32.426 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:32.426 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5029,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:59:32.427 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-04-20T10:59:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb99406c920 msgr2=0x7fb99406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb99406c920 0x7fb99406ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac000f60 tx=0x7fb9ac009040 comp rx=0 tx=0).stop 2024-04-20T10:59:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 msgr2=0x7fb9b01536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 0x7fb9b01536e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00098c0 tx=0x7fb9a00046c0 comp rx=0 tx=0).stop 2024-04-20T10:59:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 shutdown_connections 2024-04-20T10:59:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fb99406c920 0x7fb99406ede0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb9b015f580 0x7fb9b014e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9b015b580 0x7fb9b01536e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 --2- 172.21.15.27:0/1396004580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9b01556f0 0x7fb9b01531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 >> 172.21.15.27:0/1396004580 conn(0x7fb9b0067ae0 msgr2=0x7fb9b015e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 shutdown_connections 2024-04-20T10:59:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:32.428+0000 7fb9b7843640 1 -- 172.21.15.27:0/1396004580 wait complete. 2024-04-20T10:59:32.439 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T10:59:32.867 INFO:tasks.workunit.client.0.smithi027.stdout:4/983: sync 2024-04-20T10:59:32.867 INFO:tasks.workunit.client.0.smithi027.stdout:9/930: sync 2024-04-20T10:59:32.868 INFO:tasks.workunit.client.0.smithi027.stdout:4/984: truncate d0/d48/d123/f21 725719 0 2024-04-20T10:59:32.870 INFO:tasks.workunit.client.0.smithi027.stdout:9/931: creat d4/d1b/daf/f11a x:0 0 0 2024-04-20T10:59:32.872 INFO:tasks.workunit.client.0.smithi027.stdout:4/985: unlink d0/d48/d123/dc3/d11f/l148 0 2024-04-20T10:59:32.874 INFO:tasks.workunit.client.0.smithi027.stdout:4/986: unlink d0/d48/d123/f3d 0 2024-04-20T10:59:32.876 INFO:tasks.workunit.client.0.smithi027.stdout:4/987: unlink d0/d14/d6b/da3/d1a/dae/fb5 0 2024-04-20T10:59:32.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/988: dread d0/d14/d6b/da3/d18/f20 [0,4194304] 0 2024-04-20T10:59:32.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/989: fsync d0/d14/fc4 0 2024-04-20T10:59:32.925 INFO:tasks.workunit.client.0.smithi027.stdout:4/990: mkdir d0/d48/d123/dc3/d14f 0 2024-04-20T10:59:32.927 INFO:tasks.workunit.client.0.smithi027.stdout:4/991: mkdir d0/d48/d68/d76/de2/d150 0 2024-04-20T10:59:32.928 INFO:tasks.workunit.client.0.smithi027.stdout:4/992: symlink d0/d14/d6b/da3/d53/d6a/d149/l151 0 2024-04-20T10:59:32.929 INFO:tasks.workunit.client.0.smithi027.stdout:4/993: creat d0/d48/d123/d33/d97/f152 x:0 0 0 2024-04-20T10:59:32.948 INFO:tasks.workunit.client.0.smithi027.stdout:4/994: dwrite d0/d14/d6b/da3/d1a/f8a [0,4194304] 0 2024-04-20T10:59:32.950 INFO:tasks.workunit.client.0.smithi027.stdout:4/995: mkdir d0/d14/d6b/da3/d53/d116/d153 0 2024-04-20T10:59:32.952 INFO:tasks.workunit.client.0.smithi027.stdout:4/996: creat d0/d14/d6b/da3/d1a/d3b/dda/f154 x:0 0 0 2024-04-20T10:59:32.969 INFO:tasks.workunit.client.0.smithi027.stdout:4/997: dwrite d0/d48/d123/d2f/de6/f137 [0,4194304] 0 2024-04-20T10:59:32.972 INFO:tasks.workunit.client.0.smithi027.stdout:4/998: mknod d0/d14/d6b/da3/d1a/dae/c155 0 2024-04-20T10:59:32.973 INFO:tasks.workunit.client.0.smithi027.stdout:4/999: creat d0/d14/d6b/da3/d53/d6a/f156 x:0 0 0 2024-04-20T10:59:33.502 INFO:tasks.workunit.client.0.smithi027.stdout:9/932: sync 2024-04-20T10:59:33.502 INFO:tasks.workunit.client.0.smithi027.stdout:9/933: stat d4/d1b/d31/c101 0 2024-04-20T10:59:33.509 INFO:tasks.workunit.client.0.smithi027.stdout:9/934: dread d4/d1b/d1f/d5a/db7/fc5 [0,4194304] 0 2024-04-20T10:59:33.509 INFO:tasks.workunit.client.0.smithi027.stdout:9/935: chown d4/d74/ceb 336 1 2024-04-20T10:59:33.527 INFO:tasks.workunit.client.0.smithi027.stdout:9/936: dwrite d4/d74/d72/f105 [0,4194304] 0 2024-04-20T10:59:33.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/937: creat d4/d1b/d8f/de1/f11b x:0 0 0 2024-04-20T10:59:33.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/938: creat d4/d1b/d50/d100/f11c x:0 0 0 2024-04-20T10:59:33.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/939: symlink d4/d1b/d31/d51/l11d 0 2024-04-20T10:59:33.531 INFO:tasks.workunit.client.0.smithi027.stdout:9/940: truncate d4/fb8 156715 0 2024-04-20T10:59:33.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/941: creat d4/d1b/d31/d51/f11e x:0 0 0 2024-04-20T10:59:33.533 INFO:tasks.workunit.client.0.smithi027.stdout:9/942: rename d4/d1b/d1f/d45/d8a/db2 to d4/d1b/d31/d51/d11f 0 2024-04-20T10:59:33.534 INFO:tasks.workunit.client.0.smithi027.stdout:9/943: link d4/d1b/d8f/cff d4/d1b/d50/d100/c120 0 2024-04-20T10:59:33.543 INFO:tasks.workunit.client.0.smithi027.stdout:9/944: dread d4/f1c [4194304,4194304] 0 2024-04-20T10:59:33.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/945: creat d4/d1b/d1f/d45/d8a/f121 x:0 0 0 2024-04-20T10:59:33.545 INFO:tasks.workunit.client.0.smithi027.stdout:9/946: readlink d4/d1b/daf/d10a/da8/l116 0 2024-04-20T10:59:33.545 INFO:tasks.workunit.client.0.smithi027.stdout:9/947: chown d4/d1b/d31/dd0/dda/fdf 1853794 1 2024-04-20T10:59:33.545 INFO:tasks.workunit.client.0.smithi027.stdout:9/948: write d4/d74/d72/ff1 [811432,65920] 0 2024-04-20T10:59:33.548 INFO:tasks.workunit.client.0.smithi027.stdout:9/949: write d4/d1b/d1f/d5a/db7/fc5 [2326359,100714] 0 2024-04-20T10:59:33.549 INFO:tasks.workunit.client.0.smithi027.stdout:9/950: creat d4/d1b/daf/f122 x:0 0 0 2024-04-20T10:59:33.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/951: link d4/d1b/d31/d51/f94 d4/d1b/d1f/d5a/f123 0 2024-04-20T10:59:33.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/952: write d4/d1b/daf/d10a/d43/fd1 [5124231,5315] 0 2024-04-20T10:59:33.552 INFO:tasks.workunit.client.0.smithi027.stdout:9/953: creat d4/d1b/d31/d23/d29/d7f/dec/f124 x:0 0 0 2024-04-20T10:59:33.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/954: symlink d4/d1b/d1f/d45/d9e/l125 0 2024-04-20T10:59:33.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/955: fdatasync d4/d2a/fd2 0 2024-04-20T10:59:33.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/956: read - d4/d1b/d1f/d45/d8a/f108 zero size 2024-04-20T10:59:33.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/957: dread - d4/d1b/d1f/d57/db0/fd6 zero size 2024-04-20T10:59:33.639 INFO:tasks.workunit.client.0.smithi027.stdout:9/958: sync 2024-04-20T10:59:33.639 INFO:tasks.workunit.client.0.smithi027.stdout:9/959: chown d4/d1b/daf/f122 8 1 2024-04-20T10:59:33.639 INFO:tasks.workunit.client.0.smithi027.stdout:9/960: fsync d4/d1b/daf/d10a/da8/ff7 0 2024-04-20T10:59:33.640 INFO:tasks.workunit.client.0.smithi027.stdout:9/961: rename d4/d1b/d1f/d45/cee to d4/d1b/daf/d103/c126 0 2024-04-20T10:59:33.640 INFO:tasks.workunit.client.0.smithi027.stdout:9/962: write d4/d1b/d50/ff6 [492749,92279] 0 2024-04-20T10:59:33.642 INFO:tasks.workunit.client.0.smithi027.stdout:9/963: rename d4/d1b/daf/d10a/d43/fcc to d4/d1b/d50/d100/f127 0 2024-04-20T10:59:33.642 INFO:tasks.workunit.client.0.smithi027.stdout:9/964: truncate d4/d1b/d31/d51/f94 2844090 0 2024-04-20T10:59:33.642 INFO:tasks.workunit.client.0.smithi027.stdout:9/965: chown d4/d1b/d31/d51/d11f/fbb 662559548 1 2024-04-20T10:59:33.643 INFO:tasks.workunit.client.0.smithi027.stdout:9/966: mknod d4/d1b/d31/dd0/c128 0 2024-04-20T10:59:33.643 INFO:tasks.workunit.client.0.smithi027.stdout:9/967: write d4/d1b/daf/d10a/d43/f52 [3524913,26739] 0 2024-04-20T10:59:33.643 INFO:tasks.workunit.client.0.smithi027.stdout:9/968: truncate d4/d1b/daf/d10a/d43/d91/fe9 383912 0 2024-04-20T10:59:33.643 INFO:tasks.workunit.client.0.smithi027.stdout:9/969: fdatasync d4/d1b/d8f/f90 0 2024-04-20T10:59:33.646 INFO:tasks.workunit.client.0.smithi027.stdout:9/970: dread d4/d1b/d31/f9a [0,4194304] 0 2024-04-20T10:59:33.646 INFO:tasks.workunit.client.0.smithi027.stdout:9/971: read d4/d1b/daf/d10a/d43/fc7 [251656,109781] 0 2024-04-20T10:59:33.647 INFO:tasks.workunit.client.0.smithi027.stdout:9/972: rmdir d4/d1b/d1f/d45/d8a 39 2024-04-20T10:59:33.647 INFO:tasks.workunit.client.0.smithi027.stdout:9/973: write d4/d1b/d50/d5d/f10f [408940,36160] 0 2024-04-20T10:59:33.647 INFO:tasks.workunit.client.0.smithi027.stdout:9/974: chown d4/d2a/f69 1550607224 1 2024-04-20T10:59:33.647 INFO:tasks.workunit.client.0.smithi027.stdout:9/975: write d4/fc8 [7109844,36468] 0 2024-04-20T10:59:33.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/976: creat d4/d1b/d50/d100/f129 x:0 0 0 2024-04-20T10:59:33.649 INFO:tasks.workunit.client.0.smithi027.stdout:9/977: mknod d4/c12a 0 2024-04-20T10:59:33.650 INFO:tasks.workunit.client.0.smithi027.stdout:9/978: mkdir d4/d1b/daf/d10a/da8/d12b 0 2024-04-20T10:59:33.651 INFO:tasks.workunit.client.0.smithi027.stdout:9/979: mknod d4/d1b/d31/d51/c12c 0 2024-04-20T10:59:33.652 INFO:tasks.workunit.client.0.smithi027.stdout:9/980: rename d4/d1b/daf/d10a/l7e to d4/d1b/d8f/de1/dfd/l12d 0 2024-04-20T10:59:33.652 INFO:tasks.workunit.client.0.smithi027.stdout:9/981: write d4/d1b/d1f/d45/d8a/fed [510911,93744] 0 2024-04-20T10:59:33.661 INFO:tasks.workunit.client.0.smithi027.stdout:9/982: dread d4/d2a/fd2 [0,4194304] 0 2024-04-20T10:59:33.661 INFO:tasks.workunit.client.0.smithi027.stdout:9/983: truncate d4/d1b/d8f/de1/dfd/fea 332011 0 2024-04-20T10:59:33.661 INFO:tasks.workunit.client.0.smithi027.stdout:9/984: chown d4/d1b/d8f/de1 4 1 2024-04-20T10:59:33.662 INFO:tasks.workunit.client.0.smithi027.stdout:9/985: fsync d4/d1b/d1f/d45/f7d 0 2024-04-20T10:59:33.662 INFO:tasks.workunit.client.0.smithi027.stdout:9/986: rmdir d4/d1b/daf 39 2024-04-20T10:59:33.663 INFO:tasks.workunit.client.0.smithi027.stdout:9/987: rmdir d4/d1b/d1f/d45 39 2024-04-20T10:59:33.664 INFO:tasks.workunit.client.0.smithi027.stdout:9/988: symlink d4/d1b/d31/d23/d29/d7f/dec/l12e 0 2024-04-20T10:59:33.664 INFO:tasks.workunit.client.0.smithi027.stdout:9/989: truncate d4/d1b/d31/d23/d29/d7f/f110 418966 0 2024-04-20T10:59:33.664 INFO:tasks.workunit.client.0.smithi027.stdout:9/990: stat d4/d1b/d1f 0 2024-04-20T10:59:33.664 INFO:tasks.workunit.client.0.smithi027.stdout:9/991: stat d4/c12a 0 2024-04-20T10:59:33.664 INFO:tasks.workunit.client.0.smithi027.stdout:9/992: mkdir d4/d1b/d31/d23/d29/d7f/d12f 0 2024-04-20T10:59:33.665 INFO:tasks.workunit.client.0.smithi027.stdout:9/993: truncate d4/d2a/fd2 1365364 0 2024-04-20T10:59:33.666 INFO:tasks.workunit.client.0.smithi027.stdout:9/994: unlink d4/d1b/d1f/d45/d8a/fed 0 2024-04-20T10:59:33.667 INFO:tasks.workunit.client.0.smithi027.stdout:9/995: dread - d4/d1b/d50/fa2 zero size 2024-04-20T10:59:33.667 INFO:tasks.workunit.client.0.smithi027.stdout:9/996: fsync d4/d1b/daf/d10a/f27 0 2024-04-20T10:59:33.667 INFO:tasks.workunit.client.0.smithi027.stdout:9/997: readlink d4/d1b/d1f/l65 0 2024-04-20T10:59:33.675 INFO:tasks.workunit.client.0.smithi027.stdout:9/998: dread d4/d1b/d31/d23/d29/d7f/fad [0,4194304] 0 2024-04-20T10:59:33.691 INFO:tasks.workunit.client.0.smithi027.stdout:9/999: dwrite d4/d1b/daf/f122 [0,4194304] 0 2024-04-20T10:59:33.693 INFO:tasks.workunit.client.0.smithi027.stderr:+ rm -rf -- ./tmp.uXDY8wZYfi 2024-04-20T10:59:34.440 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:34.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.555+0000 7f93c0206640 1 Processor -- start 2024-04-20T10:59:34.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.555+0000 7f93c0206640 1 -- start start 2024-04-20T10:59:34.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93c0206640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 0x7f93b80cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:34.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93c0206640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93b80cc330 0x7f93b80b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:34.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93c0206640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f93b80b5a30 0x7f93b80d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:34.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93c0206640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f93b80b5ea0 con 0x7f93b80cc330 2024-04-20T10:59:34.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93c0206640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f93b815b9c0 con 0x7f93b80b5a30 2024-04-20T10:59:34.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93c0206640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f93b815bca0 con 0x7f93b8155950 2024-04-20T10:59:34.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93bd7a1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93b80cc330 0x7f93b80b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:34.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93bd7a1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93b80cc330 0x7f93b80b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39506/0 (socket says 172.21.15.27:39506) 2024-04-20T10:59:34.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93bd7a1640 1 -- 172.21.15.27:0/1950969860 learned_addr learned my addr 172.21.15.27:0/1950969860 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:34.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93bdfa2640 1 --2- 172.21.15.27:0/1950969860 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 0x7f93b80cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:34.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93be7a3640 1 --2- 172.21.15.27:0/1950969860 >> v2:172.21.15.137:3300/0 conn(0x7f93b80b5a30 0x7f93b80d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:34.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93bd7a1640 1 -- 172.21.15.27:0/1950969860 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 msgr2=0x7f93b80cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:34.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93bd7a1640 1 --2- 172.21.15.27:0/1950969860 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 0x7f93b80cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93bd7a1640 1 -- 172.21.15.27:0/1950969860 >> v2:172.21.15.137:3300/0 conn(0x7f93b80b5a30 msgr2=0x7f93b80d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:34.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93bd7a1640 1 --2- 172.21.15.27:0/1950969860 >> v2:172.21.15.137:3300/0 conn(0x7f93b80b5a30 0x7f93b80d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.556+0000 7f93bd7a1640 1 -- 172.21.15.27:0/1950969860 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b815bf80 con 0x7f93b80cc330 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.557+0000 7f93bd7a1640 1 --2- 172.21.15.27:0/1950969860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80cc330 0x7f93b80b54f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f93b808e4b0 tx=0x7f93ac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9290e72a842ff3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.557+0000 7f93bcfa0640 1 -- 172.21.15.27:0/1950969860 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac034070 con 0x7f93b80cc330 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.557+0000 7f93bcfa0640 1 -- 172.21.15.27:0/1950969860 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac030860 con 0x7f93b80cc330 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.557+0000 7f93bcfa0640 1 -- 172.21.15.27:0/1950969860 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac030c30 con 0x7f93b80cc330 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.558+0000 7f93c0206640 1 -- 172.21.15.27:0/1950969860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80cc330 msgr2=0x7f93b80b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.558+0000 7f93c0206640 1 --2- 172.21.15.27:0/1950969860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80cc330 0x7f93b80b54f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f93b808e4b0 tx=0x7f93ac02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.558+0000 7f93c0206640 1 -- 172.21.15.27:0/1950969860 shutdown_connections 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.558+0000 7f93c0206640 1 --2- 172.21.15.27:0/1950969860 >> v2:172.21.15.137:3300/0 conn(0x7f93b80b5a30 0x7f93b80d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.558+0000 7f93c0206640 1 --2- 172.21.15.27:0/1950969860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80cc330 0x7f93b80b54f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.558+0000 7f93c0206640 1 --2- 172.21.15.27:0/1950969860 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 0x7f93b80cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.558+0000 7f93c0206640 1 -- 172.21.15.27:0/1950969860 >> 172.21.15.27:0/1950969860 conn(0x7f93b80676c0 msgr2=0x7f93b8067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:34.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.558+0000 7f93c0206640 1 -- 172.21.15.27:0/1950969860 shutdown_connections 2024-04-20T10:59:34.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.558+0000 7f93c0206640 1 -- 172.21.15.27:0/1950969860 wait complete. 2024-04-20T10:59:34.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.559+0000 7f93c0206640 1 Processor -- start 2024-04-20T10:59:34.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.559+0000 7f93c0206640 1 -- start start 2024-04-20T10:59:34.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93c0206640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80b5a30 0x7f93b80c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:34.562 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93c0206640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93b80cc330 0x7f93b8150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93c0206640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 0x7f93b81f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93c0206640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f93b815fb90 con 0x7f93b80b5a30 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93c0206640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f93b815c540 con 0x7f93b80cc330 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93c0206640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f93b815fd10 con 0x7f93b8155950 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bdfa2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80b5a30 0x7f93b80c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bdfa2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80b5a30 0x7f93b80c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39508/0 (socket says 172.21.15.27:39508) 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bdfa2640 1 -- 172.21.15.27:0/695825580 learned_addr learned my addr 172.21.15.27:0/695825580 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93be7a3640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 0x7f93b81f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bd7a1640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93b80cc330 0x7f93b8150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bdfa2640 1 -- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 msgr2=0x7f93b81f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bdfa2640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 0x7f93b81f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bdfa2640 1 -- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93b80cc330 msgr2=0x7f93b8150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bdfa2640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93b80cc330 0x7f93b8150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bdfa2640 1 -- 172.21.15.27:0/695825580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b80c7940 con 0x7f93b80b5a30 2024-04-20T10:59:34.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93be7a3640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 0x7f93b81f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bd7a1640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93b80cc330 0x7f93b8150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93bdfa2640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80b5a30 0x7f93b80c6020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93b400e8a0 tx=0x7f93b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.560+0000 7f93aa7fc640 1 -- 172.21.15.27:0/695825580 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4017070 con 0x7f93b80b5a30 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.561+0000 7f93aa7fc640 1 -- 172.21.15.27:0/695825580 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b400c940 con 0x7f93b80b5a30 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.561+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b8159870 con 0x7f93b80b5a30 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.561+0000 7f93aa7fc640 1 -- 172.21.15.27:0/695825580 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b400cd30 con 0x7f93b80b5a30 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.561+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b8159e30 con 0x7f93b80b5a30 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.562+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b80b5850 con 0x7f93b80b5a30 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.562+0000 7f93aa7fc640 1 -- 172.21.15.27:0/695825580 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93b401a070 con 0x7f93b80b5a30 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.563+0000 7f93aa7fc640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939c06c8f0 0x7f939c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.563+0000 7f93aa7fc640 1 -- 172.21.15.27:0/695825580 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f93b408bf30 con 0x7f93b80b5a30 2024-04-20T10:59:34.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.563+0000 7f93bd7a1640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939c06c8f0 0x7f939c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:34.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.563+0000 7f93bd7a1640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939c06c8f0 0x7f939c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93ac036000 tx=0x7f93ac03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:34.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.567+0000 7f93aa7fc640 1 -- 172.21.15.27:0/695825580 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f93b80b5850 con 0x7f93b80b5a30 2024-04-20T10:59:34.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.792+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93b804fcf0 con 0x7f93b80b5a30 2024-04-20T10:59:34.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.792+0000 7f93aa7fc640 1 -- 172.21.15.27:0/695825580 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6933 (secure 0 0 0) 0x7f93b405a1b0 con 0x7f93b80b5a30 2024-04-20T10:59:34.792 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:34.792 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5029,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:59:34.792 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-04-20T10:59:34.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939c06c8f0 msgr2=0x7f939c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:34.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939c06c8f0 0x7f939c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93ac036000 tx=0x7f93ac03b040 comp rx=0 tx=0).stop 2024-04-20T10:59:34.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80b5a30 msgr2=0x7f93b80c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80b5a30 0x7f93b80c6020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93b400e8a0 tx=0x7f93b400ed60 comp rx=0 tx=0).stop 2024-04-20T10:59:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 shutdown_connections 2024-04-20T10:59:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f939c06c8f0 0x7f939c06edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f93b8155950 0x7f93b81f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93b80cc330 0x7f93b8150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 --2- 172.21.15.27:0/695825580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93b80b5a30 0x7f93b80c6020 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 >> 172.21.15.27:0/695825580 conn(0x7f93b80676c0 msgr2=0x7f93b8067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 shutdown_connections 2024-04-20T10:59:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:34.794+0000 7f93c0206640 1 -- 172.21.15.27:0/695825580 wait complete. 2024-04-20T10:59:34.806 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T10:59:36.807 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:36.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.922+0000 7f2c0184e640 1 Processor -- start 2024-04-20T10:59:36.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.922+0000 7f2c0184e640 1 -- start start 2024-04-20T10:59:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.923+0000 7f2c0184e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc15ed10 0x7f2bfc073970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.923+0000 7f2c0184e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2bfc072130 0x7f2bfc0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.923+0000 7f2c0184e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bfc073fc0 con 0x7f2bfc0c8d30 2024-04-20T10:59:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.923+0000 7f2c0184e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f2bfc0c3bc0 con 0x7f2bfc072130 2024-04-20T10:59:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.923+0000 7f2c0184e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2bfc0c4050 con 0x7f2bfc15ed10 2024-04-20T10:59:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.923+0000 7f2bf3fff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f2bfc072130 0x7f2bfc0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.923+0000 7f2bfb7fe640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f2bfc0c8d30 0x7f2bfc15f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:33842/0 (socket says 172.21.15.27:33842) 2024-04-20T10:59:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.923+0000 7f2bfb7fe640 1 -- 172.21.15.27:0/3108775614 learned_addr learned my addr 172.21.15.27:0/3108775614 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:36.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.923+0000 7f2bfaffd640 1 --2- 172.21.15.27:0/3108775614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc15ed10 0x7f2bfc073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.924+0000 7f2bfaffd640 1 -- 172.21.15.27:0/3108775614 >> v2:172.21.15.137:3300/0 conn(0x7f2bfc072130 msgr2=0x7f2bfc0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.924+0000 7f2bfaffd640 1 --2- 172.21.15.27:0/3108775614 >> v2:172.21.15.137:3300/0 conn(0x7f2bfc072130 0x7f2bfc0c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.924+0000 7f2bfaffd640 1 -- 172.21.15.27:0/3108775614 >> v1:172.21.15.137:6789/0 conn(0x7f2bfc0c8d30 legacy=0x7f2bfc15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.924+0000 7f2bfa7fc640 1 -- 172.21.15.27:0/3108775614 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078633693 0 0) 0x7f2bfc073fc0 con 0x7f2bfc0c8d30 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.924+0000 7f2bfaffd640 1 -- 172.21.15.27:0/3108775614 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bfc0c2fb0 con 0x7f2bfc15ed10 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.924+0000 7f2bf3fff640 1 --2- 172.21.15.27:0/3108775614 >> v2:172.21.15.137:3300/0 conn(0x7f2bfc072130 0x7f2bfc0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2bfaffd640 1 --2- 172.21.15.27:0/3108775614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc15ed10 0x7f2bfc073970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0097a0 tx=0x7f2bec02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29aeabfaee7fd59c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2bfa7fc640 1 -- 172.21.15.27:0/3108775614 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec035020 con 0x7f2bfc15ed10 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2bfa7fc640 1 -- 172.21.15.27:0/3108775614 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bec031830 con 0x7f2bfc15ed10 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2bfa7fc640 1 -- 172.21.15.27:0/3108775614 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec031c00 con 0x7f2bfc15ed10 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2c0184e640 1 -- 172.21.15.27:0/3108775614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc15ed10 msgr2=0x7f2bfc073970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2c0184e640 1 --2- 172.21.15.27:0/3108775614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc15ed10 0x7f2bfc073970 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0097a0 tx=0x7f2bec02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2c0184e640 1 -- 172.21.15.27:0/3108775614 shutdown_connections 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2c0184e640 1 --2- 172.21.15.27:0/3108775614 >> v2:172.21.15.137:3300/0 conn(0x7f2bfc072130 0x7f2bfc0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2c0184e640 1 --2- 172.21.15.27:0/3108775614 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc15ed10 0x7f2bfc073970 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2c0184e640 1 -- 172.21.15.27:0/3108775614 >> 172.21.15.27:0/3108775614 conn(0x7f2bfc0ca8f0 msgr2=0x7f2bfc0cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2c0184e640 1 -- 172.21.15.27:0/3108775614 shutdown_connections 2024-04-20T10:59:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.925+0000 7f2c0184e640 1 -- 172.21.15.27:0/3108775614 wait complete. 2024-04-20T10:59:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.927+0000 7f2c0184e640 1 Processor -- start 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.927+0000 7f2c0184e640 1 -- start start 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.927+0000 7f2c0184e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc072130 0x7f2bfc1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.927+0000 7f2c0184e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bfc0c8d30 0x7f2bfc14e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.927+0000 7f2c0184e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc15ed10 0x7f2bfc14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.927+0000 7f2c0184e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc0c3850 con 0x7f2bfc15ed10 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.927+0000 7f2c0184e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2bfc0c3570 con 0x7f2bfc0c8d30 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.927+0000 7f2c0184e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2bfc15f800 con 0x7f2bfc072130 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc15ed10 0x7f2bfc14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc15ed10 0x7f2bfc14ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36856/0 (socket says 172.21.15.27:36856) 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfb7fe640 1 -- 172.21.15.27:0/2918101927 learned_addr learned my addr 172.21.15.27:0/2918101927 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bf3fff640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bfc0c8d30 0x7f2bfc14e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfaffd640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc072130 0x7f2bfc1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfb7fe640 1 -- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc072130 msgr2=0x7f2bfc1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfb7fe640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc072130 0x7f2bfc1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfb7fe640 1 -- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bfc0c8d30 msgr2=0x7f2bfc14e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfb7fe640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bfc0c8d30 0x7f2bfc14e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfb7fe640 1 -- 172.21.15.27:0/2918101927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bfc155f70 con 0x7f2bfc15ed10 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfaffd640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc072130 0x7f2bfc1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bfb7fe640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc15ed10 0x7f2bfc14ec50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c7d0 tx=0x7f2bf400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bf8ff9640 1 -- 172.21.15.27:0/2918101927 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4018070 con 0x7f2bfc15ed10 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bf8ff9640 1 -- 172.21.15.27:0/2918101927 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf400a640 con 0x7f2bfc15ed10 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bfc159890 con 0x7f2bfc15ed10 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2bf8ff9640 1 -- 172.21.15.27:0/2918101927 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf400aa70 con 0x7f2bfc15ed10 2024-04-20T10:59:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.928+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bfc157ee0 con 0x7f2bfc15ed10 2024-04-20T10:59:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.930+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bfc0c44f0 con 0x7f2bfc15ed10 2024-04-20T10:59:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.930+0000 7f2bf8ff9640 1 -- 172.21.15.27:0/2918101927 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bf400ac10 con 0x7f2bfc15ed10 2024-04-20T10:59:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.930+0000 7f2bf8ff9640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2bdc06c8f0 0x7f2bdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.931+0000 7f2bf8ff9640 1 -- 172.21.15.27:0/2918101927 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5997+0+0 (secure 0 0 0) 0x7f2bf408b2d0 con 0x7f2bfc15ed10 2024-04-20T10:59:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.931+0000 7f2bfaffd640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2bdc06c8f0 0x7f2bdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.931+0000 7f2bfaffd640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2bdc06c8f0 0x7f2bdc06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2bec036000 tx=0x7f2bec008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:36.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:36.935+0000 7f2bf8ff9640 1 -- 172.21.15.27:0/2918101927 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2bfc0c44f0 con 0x7f2bfc15ed10 2024-04-20T10:59:37.158 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.158+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bfc159a60 con 0x7f2bfc15ed10 2024-04-20T10:59:37.159 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.159+0000 7f2bf8ff9640 1 -- 172.21.15.27:0/2918101927 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+6933 (secure 0 0 0) 0x7f2bfc159a60 con 0x7f2bfc15ed10 2024-04-20T10:59:37.159 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:37.159 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5029,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:58:43.225256+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":4272,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"c","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.27:6835/859496751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":859496751},{"type":"v1","addr":"172.21.15.27:6835","nonce":859496751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275,4272]},"id":1}]} 2024-04-20T10:59:37.159 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 29 2024-04-20T10:59:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2bdc06c8f0 msgr2=0x7f2bdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2bdc06c8f0 0x7f2bdc06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2bec036000 tx=0x7f2bec008030 comp rx=0 tx=0).stop 2024-04-20T10:59:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc15ed10 msgr2=0x7f2bfc14ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc15ed10 0x7f2bfc14ec50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c7d0 tx=0x7f2bf400cc90 comp rx=0 tx=0).stop 2024-04-20T10:59:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 shutdown_connections 2024-04-20T10:59:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f2bdc06c8f0 0x7f2bdc06edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:37.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bfc15ed10 0x7f2bfc14ec50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bfc0c8d30 0x7f2bfc14e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 --2- 172.21.15.27:0/2918101927 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2bfc072130 0x7f2bfc1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 >> 172.21.15.27:0/2918101927 conn(0x7f2bfc0ca8f0 msgr2=0x7f2bfc152a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 shutdown_connections 2024-04-20T10:59:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:37.161+0000 7f2c0184e640 1 -- 172.21.15.27:0/2918101927 wait complete. 2024-04-20T10:59:37.172 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-04-20T10:59:39.173 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:39.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.275+0000 7f07412a5640 1 Processor -- start 2024-04-20T10:59:39.275 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.275+0000 7f07412a5640 1 -- start start 2024-04-20T10:59:39.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f07412a5640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f073c073970 0x7f073c1552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f07412a5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f073c073fc0 con 0x7f073c15ed10 2024-04-20T10:59:39.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f07412a5640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f073c073d50 con 0x7f073c072130 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f07412a5640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f073c0c3bc0 con 0x7f073c073970 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f073affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f073c15ed10 0x7f073c0c87f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44284/0 (socket says 172.21.15.27:44284) 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f073affd640 1 -- 172.21.15.27:0/1338733714 learned_addr learned my addr 172.21.15.27:0/1338733714 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f073b7fe640 1 --2- 172.21.15.27:0/1338733714 >> v2:172.21.15.137:3300/0 conn(0x7f073c073970 0x7f073c1552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f0739ffb640 1 -- 172.21.15.27:0/1338733714 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486187298 0 0) 0x7f073c073fc0 con 0x7f073c15ed10 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f0739ffb640 1 -- 172.21.15.27:0/1338733714 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0728003520 con 0x7f073c15ed10 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f0739ffb640 1 -- 172.21.15.27:0/1338733714 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3673462051 0 0) 0x7f073c073d50 con 0x7f073c072130 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.276+0000 7f0739ffb640 1 -- 172.21.15.27:0/1338733714 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f073c073fc0 con 0x7f073c072130 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.277+0000 7f0739ffb640 1 -- 172.21.15.27:0/1338733714 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1457170159 0 0) 0x7f0728003520 con 0x7f073c15ed10 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.277+0000 7f073b7fe640 1 -- 172.21.15.27:0/1338733714 >> v1:172.21.15.137:6789/0 conn(0x7f073c072130 legacy=0x7f073c15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.277+0000 7f073b7fe640 1 -- 172.21.15.27:0/1338733714 >> v1:172.21.15.27:6789/0 conn(0x7f073c15ed10 legacy=0x7f073c0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.277+0000 7f073b7fe640 1 -- 172.21.15.27:0/1338733714 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c0c2fb0 con 0x7f073c073970 2024-04-20T10:59:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.277+0000 7f073b7fe640 1 --2- 172.21.15.27:0/1338733714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c073970 0x7f073c1552f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f072c009d90 tx=0x7f072c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75ed50d186f65244 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f0739ffb640 1 -- 172.21.15.27:0/1338733714 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c039070 con 0x7f073c073970 2024-04-20T10:59:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f0739ffb640 1 -- 172.21.15.27:0/1338733714 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072c02c940 con 0x7f073c073970 2024-04-20T10:59:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f0739ffb640 1 -- 172.21.15.27:0/1338733714 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734001670 con 0x7f073c073970 2024-04-20T10:59:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f07412a5640 1 -- 172.21.15.27:0/1338733714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c073970 msgr2=0x7f073c1552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f07412a5640 1 --2- 172.21.15.27:0/1338733714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c073970 0x7f073c1552f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f072c009d90 tx=0x7f072c02be10 comp rx=0 tx=0).stop 2024-04-20T10:59:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f07412a5640 1 -- 172.21.15.27:0/1338733714 shutdown_connections 2024-04-20T10:59:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f07412a5640 1 --2- 172.21.15.27:0/1338733714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c073970 0x7f073c1552f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f07412a5640 1 -- 172.21.15.27:0/1338733714 >> 172.21.15.27:0/1338733714 conn(0x7f073c0ca8f0 msgr2=0x7f073c0cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f07412a5640 1 -- 172.21.15.27:0/1338733714 shutdown_connections 2024-04-20T10:59:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.278+0000 7f07412a5640 1 -- 172.21.15.27:0/1338733714 wait complete. 2024-04-20T10:59:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.279+0000 7f07412a5640 1 Processor -- start 2024-04-20T10:59:39.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.279+0000 7f07412a5640 1 -- start start 2024-04-20T10:59:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f07412a5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f073c072130 0x7f073c147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f07412a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c073970 0x7f073c0cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f07412a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c15ed10 0x7f073c1e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f07412a5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0c3570 con 0x7f073c073970 2024-04-20T10:59:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f07412a5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0c4050 con 0x7f073c15ed10 2024-04-20T10:59:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f07412a5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f073c0c3850 con 0x7f073c072130 2024-04-20T10:59:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f073a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c073970 0x7f073c0cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.282 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f073a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c073970 0x7f073c0cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36864/0 (socket says 172.21.15.27:36864) 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f073a7fc640 1 -- 172.21.15.27:0/71956806 learned_addr learned my addr 172.21.15.27:0/71956806 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f073b7fe640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c15ed10 0x7f073c1e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.280+0000 7f073affd640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f073c072130 0x7f073c147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f073a7fc640 1 -- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f073c072130 msgr2=0x7f073c147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f073a7fc640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f073c072130 0x7f073c147a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f073a7fc640 1 -- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c15ed10 msgr2=0x7f073c1e4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f073a7fc640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c15ed10 0x7f073c1e4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f073a7fc640 1 -- 172.21.15.27:0/71956806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c0b3f90 con 0x7f073c073970 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f073affd640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f073c072130 0x7f073c147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f073a7fc640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c073970 0x7f073c0cd1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f072400c4c0 tx=0x7f072400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:39.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f07337fe640 1 -- 172.21.15.27:0/71956806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07240052d0 con 0x7f073c073970 2024-04-20T10:59:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f07337fe640 1 -- 172.21.15.27:0/71956806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0724005470 con 0x7f073c073970 2024-04-20T10:59:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c1f5a60 con 0x7f073c073970 2024-04-20T10:59:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f07337fe640 1 -- 172.21.15.27:0/71956806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724012870 con 0x7f073c073970 2024-04-20T10:59:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.281+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c0cc3e0 con 0x7f073c073970 2024-04-20T10:59:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.282+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f073c04fcf0 con 0x7f073c073970 2024-04-20T10:59:39.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.282+0000 7f07337fe640 1 -- 172.21.15.27:0/71956806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0724012a10 con 0x7f073c073970 2024-04-20T10:59:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.283+0000 7f07337fe640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f071c06c8f0 0x7f071c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.283+0000 7f07337fe640 1 -- 172.21.15.27:0/71956806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f072408ba00 con 0x7f073c073970 2024-04-20T10:59:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.283+0000 7f073affd640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f071c06c8f0 0x7f071c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.283+0000 7f073affd640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f071c06c8f0 0x7f071c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f07340017e0 tx=0x7f0734009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:39.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.287+0000 7f07337fe640 1 -- 172.21.15.27:0/71956806 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0724059c20 con 0x7f073c073970 2024-04-20T10:59:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.509+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f073c14be80 con 0x7f073c073970 2024-04-20T10:59:39.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.509+0000 7f07337fe640 1 -- 172.21.15.27:0/71956806 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6107 (secure 0 0 0) 0x7f0724059520 con 0x7f073c073970 2024-04-20T10:59:39.509 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:39.510 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:37.339572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4275,4278,4293,4894]},"id":1}]} 2024-04-20T10:59:39.510 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2024-04-20T10:59:39.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f071c06c8f0 msgr2=0x7f071c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f071c06c8f0 0x7f071c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f07340017e0 tx=0x7f0734009040 comp rx=0 tx=0).stop 2024-04-20T10:59:39.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c073970 msgr2=0x7f073c0cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c073970 0x7f073c0cd1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f072400c4c0 tx=0x7f072400c980 comp rx=0 tx=0).stop 2024-04-20T10:59:39.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 shutdown_connections 2024-04-20T10:59:39.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f071c06c8f0 0x7f071c06edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c15ed10 0x7f073c1e4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073c073970 0x7f073c0cd1a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 --2- 172.21.15.27:0/71956806 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f073c072130 0x7f073c147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 >> 172.21.15.27:0/71956806 conn(0x7f073c0ca8f0 msgr2=0x7f073c1e6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:39.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 shutdown_connections 2024-04-20T10:59:39.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.511+0000 7f07412a5640 1 -- 172.21.15.27:0/71956806 wait complete. 2024-04-20T10:59:39.522 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-04-20T10:59:39.522 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T10:59:39.522 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:39.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.623+0000 7f7cc9b46640 1 Processor -- start 2024-04-20T10:59:39.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.623+0000 7f7cc9b46640 1 -- start start 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.624+0000 7f7cc9b46640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f7cc4155950 0x7f7cc40cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.624+0000 7f7cc9b46640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7cc40cc330 0x7f7cc40b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.624+0000 7f7cc9b46640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc415b9c0 con 0x7f7cc40b5a30 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.624+0000 7f7cc9b46640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc415bca0 con 0x7f7cc40cc330 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.624+0000 7f7cc9b46640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc415bf80 con 0x7f7cc4155950 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.624+0000 7f7cc2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7cc40cc330 0x7f7cc40b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.624+0000 7f7cc3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7cc40b5a30 0x7f7cc40d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44292/0 (socket says 172.21.15.27:44292) 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.624+0000 7f7cc3fff640 1 -- 172.21.15.27:0/3123404303 learned_addr learned my addr 172.21.15.27:0/3123404303 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.625+0000 7f7cc37fe640 1 --2- 172.21.15.27:0/3123404303 >> v2:172.21.15.137:3300/0 conn(0x7f7cc4155950 0x7f7cc40cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.625+0000 7f7cc2ffd640 1 -- 172.21.15.27:0/3123404303 >> v2:172.21.15.137:3300/0 conn(0x7f7cc4155950 msgr2=0x7f7cc40cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.625+0000 7f7cc2ffd640 1 --2- 172.21.15.27:0/3123404303 >> v2:172.21.15.137:3300/0 conn(0x7f7cc4155950 0x7f7cc40cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.625+0000 7f7cc2ffd640 1 -- 172.21.15.27:0/3123404303 >> v1:172.21.15.27:6789/0 conn(0x7f7cc40b5a30 legacy=0x7f7cc40d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.625+0000 7f7cc2ffd640 1 -- 172.21.15.27:0/3123404303 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc415c260 con 0x7f7cc40cc330 2024-04-20T10:59:39.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.625+0000 7f7cc27fc640 1 -- 172.21.15.27:0/3123404303 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3138836100 0 0) 0x7f7cc415b9c0 con 0x7f7cc40b5a30 2024-04-20T10:59:39.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.625+0000 7f7cc2ffd640 1 --2- 172.21.15.27:0/3123404303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 0x7f7cc40b54f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40ce7c0 tx=0x7f7cb802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33aeb1d955826daf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:39.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc27fc640 1 -- 172.21.15.27:0/3123404303 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8034070 con 0x7f7cc40cc330 2024-04-20T10:59:39.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc27fc640 1 -- 172.21.15.27:0/3123404303 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb8030610 con 0x7f7cc40cc330 2024-04-20T10:59:39.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc27fc640 1 -- 172.21.15.27:0/3123404303 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80309e0 con 0x7f7cc40cc330 2024-04-20T10:59:39.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc9b46640 1 -- 172.21.15.27:0/3123404303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 msgr2=0x7f7cc40b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc9b46640 1 --2- 172.21.15.27:0/3123404303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 0x7f7cc40b54f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40ce7c0 tx=0x7f7cb802b960 comp rx=0 tx=0).stop 2024-04-20T10:59:39.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc9b46640 1 -- 172.21.15.27:0/3123404303 shutdown_connections 2024-04-20T10:59:39.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc9b46640 1 --2- 172.21.15.27:0/3123404303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 0x7f7cc40b54f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc9b46640 1 --2- 172.21.15.27:0/3123404303 >> v2:172.21.15.137:3300/0 conn(0x7f7cc4155950 0x7f7cc40cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc9b46640 1 -- 172.21.15.27:0/3123404303 >> 172.21.15.27:0/3123404303 conn(0x7f7cc40676c0 msgr2=0x7f7cc4067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:39.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc9b46640 1 -- 172.21.15.27:0/3123404303 shutdown_connections 2024-04-20T10:59:39.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.626+0000 7f7cc9b46640 1 -- 172.21.15.27:0/3123404303 wait complete. 2024-04-20T10:59:39.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.627+0000 7f7cc9b46640 1 Processor -- start 2024-04-20T10:59:39.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.627+0000 7f7cc9b46640 1 -- start start 2024-04-20T10:59:39.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc9b46640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cc40b5a30 0x7f7cc41f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc9b46640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 0x7f7cc41548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc9b46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc4155950 0x7f7cc415e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc9b46640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc415fb90 con 0x7f7cc40cc330 2024-04-20T10:59:39.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc9b46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc40b5e40 con 0x7f7cc4155950 2024-04-20T10:59:39.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc9b46640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc415fd10 con 0x7f7cc40b5a30 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 0x7f7cc41548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 0x7f7cc41548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36876/0 (socket says 172.21.15.27:36876) 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc2ffd640 1 -- 172.21.15.27:0/4230869893 learned_addr learned my addr 172.21.15.27:0/4230869893 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc37fe640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cc40b5a30 0x7f7cc41f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.628+0000 7f7cc3fff640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc4155950 0x7f7cc415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cc2ffd640 1 -- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cc40b5a30 msgr2=0x7f7cc41f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cc2ffd640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cc40b5a30 0x7f7cc41f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cc2ffd640 1 -- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc4155950 msgr2=0x7f7cc415e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cc2ffd640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc4155950 0x7f7cc415e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cc2ffd640 1 -- 172.21.15.27:0/4230869893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc4155120 con 0x7f7cc40cc330 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cc2ffd640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 0x7f7cc41548e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009890 tx=0x7f7cb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cabfff640 1 -- 172.21.15.27:0/4230869893 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8034040 con 0x7f7cc40cc330 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cabfff640 1 -- 172.21.15.27:0/4230869893 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb8030410 con 0x7f7cc40cc330 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc4152c80 con 0x7f7cc40cc330 2024-04-20T10:59:39.631 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cabfff640 1 -- 172.21.15.27:0/4230869893 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8004250 con 0x7f7cc40cc330 2024-04-20T10:59:39.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.629+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc4153240 con 0x7f7cc40cc330 2024-04-20T10:59:39.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.630+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc404fcf0 con 0x7f7cc40cc330 2024-04-20T10:59:39.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.630+0000 7f7cabfff640 1 -- 172.21.15.27:0/4230869893 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cb8037070 con 0x7f7cc40cc330 2024-04-20T10:59:39.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.631+0000 7f7cabfff640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb006c8f0 0x7f7cb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:39.632 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.631+0000 7f7cabfff640 1 -- 172.21.15.27:0/4230869893 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f7cb80ad910 con 0x7f7cc40cc330 2024-04-20T10:59:39.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.631+0000 7f7cc37fe640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb006c8f0 0x7f7cb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:39.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.632+0000 7f7cc37fe640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb006c8f0 0x7f7cb006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4003aa0 tx=0x7f7cb4009830 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:39.635 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.635+0000 7f7cabfff640 1 -- 172.21.15.27:0/4230869893 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7cb80305e0 con 0x7f7cc40cc330 2024-04-20T10:59:39.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.859+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cc40cd450 con 0x7f7cc40cc330 2024-04-20T10:59:39.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.859+0000 7f7cabfff640 1 -- 172.21.15.27:0/4230869893 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6107 (secure 0 0 0) 0x7f7cb807b600 con 0x7f7cc40cc330 2024-04-20T10:59:39.859 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:39.859 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:37.339572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4275,4278,4293,4894]},"id":1}]} 2024-04-20T10:59:39.859 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2024-04-20T10:59:39.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb006c8f0 msgr2=0x7f7cb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb006c8f0 0x7f7cb006edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4003aa0 tx=0x7f7cb4009830 comp rx=0 tx=0).stop 2024-04-20T10:59:39.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 msgr2=0x7f7cc41548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:39.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 0x7f7cc41548e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009890 tx=0x7f7cb80046c0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 shutdown_connections 2024-04-20T10:59:39.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7cb006c8f0 0x7f7cb006edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc4155950 0x7f7cc415e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc40cc330 0x7f7cc41548e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 --2- 172.21.15.27:0/4230869893 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7cc40b5a30 0x7f7cc41f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:39.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 >> 172.21.15.27:0/4230869893 conn(0x7f7cc40676c0 msgr2=0x7f7cc40c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:39.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 shutdown_connections 2024-04-20T10:59:39.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:39.861+0000 7f7cc9b46640 1 -- 172.21.15.27:0/4230869893 wait complete. 2024-04-20T10:59:39.873 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 30, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 30, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4275": { "addr": "172.21.15.27:6837/1680990583", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6834", "nonce": 1680990583, "type": "v2" }, { "addr": "172.21.15.27:6837", "nonce": 1680990583, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4275, "incarnation": 12, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4278": { "addr": "172.21.15.27:6836/4195543922", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6833", "nonce": 4195543922, "type": "v2" }, { "addr": "172.21.15.27:6836", "nonce": 4195543922, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4278, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4894": { "addr": "172.21.15.137:6838/2896327439", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6836", "nonce": 2896327439, "type": "v2" }, { "addr": "172.21.15.137:6838", "nonce": 2896327439, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4894, "incarnation": 24, "join_fscid": -1, "name": "b", "rank": 4, "state": "up:active", "state_seq": 17 }, "gid_5029": { "addr": "172.21.15.137:6835/4235742413", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6834", "nonce": 4235742413, "type": "v2" }, { "addr": "172.21.15.137:6835", "nonce": 4235742413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5029, "incarnation": 30, "join_fscid": -1, "name": "f", "rank": 3, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 25, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T10:59:37.339572+0000", "qdb_cluster": [ 4275, 4278, 4293, 4894 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 4278, "mds_2": 4275, "mds_3": 5029, "mds_4": 4894 } } } ], "standbys": [] } 2024-04-20T10:59:41.874 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.989+0000 7fbc1cc48640 1 Processor -- start 2024-04-20T10:59:41.989 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.989+0000 7fbc1cc48640 1 -- start start 2024-04-20T10:59:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.990+0000 7fbc1cc48640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fbc180b54f0 0x7fbc180b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.990+0000 7fbc1cc48640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 0x7fbc180d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.990+0000 7fbc1cc48640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc180b5e70 con 0x7fbc18155950 2024-04-20T10:59:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.990+0000 7fbc1cc48640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1815b9c0 con 0x7fbc180b54f0 2024-04-20T10:59:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.990+0000 7fbc1cc48640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1815bca0 con 0x7fbc180d0c90 2024-04-20T10:59:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc1659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbc18155950 0x7fbc180cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44302/0 (socket says 172.21.15.27:44302) 2024-04-20T10:59:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc1659c640 1 -- 172.21.15.27:0/2830332044 learned_addr learned my addr 172.21.15.27:0/2830332044 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc16d9d640 1 --2- 172.21.15.27:0/2830332044 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 0x7fbc180d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:41.991 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc15d9b640 1 --2- 172.21.15.27:0/2830332044 >> v2:172.21.15.137:3300/0 conn(0x7fbc180b54f0 0x7fbc180b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc1559a640 1 -- 172.21.15.27:0/2830332044 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1684483921 0 0) 0x7fbc180b5e70 con 0x7fbc18155950 2024-04-20T10:59:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc1559a640 1 -- 172.21.15.27:0/2830332044 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc04003520 con 0x7fbc18155950 2024-04-20T10:59:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc15d9b640 1 -- 172.21.15.27:0/2830332044 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 msgr2=0x7fbc180d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc15d9b640 1 --2- 172.21.15.27:0/2830332044 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 0x7fbc180d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc15d9b640 1 -- 172.21.15.27:0/2830332044 >> v1:172.21.15.27:6789/0 conn(0x7fbc18155950 legacy=0x7fbc180cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc15d9b640 1 -- 172.21.15.27:0/2830332044 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1815bf80 con 0x7fbc180b54f0 2024-04-20T10:59:41.992 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.991+0000 7fbc16d9d640 1 --2- 172.21.15.27:0/2830332044 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 0x7fbc180d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:59:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.992+0000 7fbc15d9b640 1 --2- 172.21.15.27:0/2830332044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 0x7fbc180b5930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbc000097a0 tx=0x7fbc0002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b19c462ecf1919f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.992+0000 7fbc1559a640 1 -- 172.21.15.27:0/2830332044 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00034070 con 0x7fbc180b54f0 2024-04-20T10:59:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.992+0000 7fbc1559a640 1 -- 172.21.15.27:0/2830332044 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc00030610 con 0x7fbc180b54f0 2024-04-20T10:59:41.993 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.992+0000 7fbc1559a640 1 -- 172.21.15.27:0/2830332044 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc000309e0 con 0x7fbc180b54f0 2024-04-20T10:59:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.993+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2830332044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 msgr2=0x7fbc180b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.993+0000 7fbc1cc48640 1 --2- 172.21.15.27:0/2830332044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 0x7fbc180b5930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbc000097a0 tx=0x7fbc0002b960 comp rx=0 tx=0).stop 2024-04-20T10:59:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.993+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2830332044 shutdown_connections 2024-04-20T10:59:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.993+0000 7fbc1cc48640 1 --2- 172.21.15.27:0/2830332044 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180d0c90 0x7fbc180d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.993+0000 7fbc1cc48640 1 --2- 172.21.15.27:0/2830332044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc180b54f0 0x7fbc180b5930 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.993+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2830332044 >> 172.21.15.27:0/2830332044 conn(0x7fbc180676c0 msgr2=0x7fbc18067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.993+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2830332044 shutdown_connections 2024-04-20T10:59:41.994 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.993+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2830332044 wait complete. 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.994+0000 7fbc1cc48640 1 Processor -- start 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.994+0000 7fbc1cc48640 1 -- start start 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc1cc48640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180b54f0 0x7fbc180c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc1cc48640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 0x7fbc18159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc1cc48640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc18155950 0x7fbc18159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc1cc48640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180d1620 con 0x7fbc180d0c90 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc1cc48640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1815c540 con 0x7fbc18155950 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc1cc48640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1815fb90 con 0x7fbc180b54f0 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc15d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 0x7fbc18159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc15d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 0x7fbc18159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36892/0 (socket says 172.21.15.27:36892) 2024-04-20T10:59:41.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc15d9b640 1 -- 172.21.15.27:0/2106046631 learned_addr learned my addr 172.21.15.27:0/2106046631 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc16d9d640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc18155950 0x7fbc18159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.995+0000 7fbc1659c640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180b54f0 0x7fbc180c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc15d9b640 1 -- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180b54f0 msgr2=0x7fbc180c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc15d9b640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180b54f0 0x7fbc180c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc15d9b640 1 -- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc18155950 msgr2=0x7fbc18159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc15d9b640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc18155950 0x7fbc18159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc15d9b640 1 -- 172.21.15.27:0/2106046631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc181546f0 con 0x7fbc180d0c90 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc15d9b640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 0x7fbc18159620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009890 tx=0x7fbc000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc0effd640 1 -- 172.21.15.27:0/2106046631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00034040 con 0x7fbc180d0c90 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc0effd640 1 -- 172.21.15.27:0/2106046631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0003aad0 con 0x7fbc180d0c90 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc0effd640 1 -- 172.21.15.27:0/2106046631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00006e10 con 0x7fbc180d0c90 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc181f14c0 con 0x7fbc180d0c90 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.996+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc181f1a60 con 0x7fbc180d0c90 2024-04-20T10:59:41.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.997+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1804fcf0 con 0x7fbc180d0c90 2024-04-20T10:59:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.998+0000 7fbc0effd640 1 -- 172.21.15.27:0/2106046631 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc0004b060 con 0x7fbc180d0c90 2024-04-20T10:59:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.998+0000 7fbc0effd640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbbf806c8f0 0x7fbbf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:41.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.998+0000 7fbc0effd640 1 -- 172.21.15.27:0/2106046631 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7fbc000ad8b0 con 0x7fbc180d0c90 2024-04-20T10:59:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.998+0000 7fbc1659c640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbbf806c8f0 0x7fbbf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:42.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:41.999+0000 7fbc1659c640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbbf806c8f0 0x7fbbf806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc08005880 tx=0x7fbc08009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:42.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.002+0000 7fbc0effd640 1 -- 172.21.15.27:0/2106046631 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbc0003f070 con 0x7fbc180d0c90 2024-04-20T10:59:42.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.229+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc180cd0a0 con 0x7fbc180d0c90 2024-04-20T10:59:42.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.230+0000 7fbc0effd640 1 -- 172.21.15.27:0/2106046631 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+6107 (secure 0 0 0) 0x7fbc0007b510 con 0x7fbc180d0c90 2024-04-20T10:59:42.230 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:42.230 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:37.339572+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:replay","state_seq":1,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4275,4278,4293,4894]},"id":1}]} 2024-04-20T10:59:42.230 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbbf806c8f0 msgr2=0x7fbbf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbbf806c8f0 0x7fbbf806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc08005880 tx=0x7fbc08009040 comp rx=0 tx=0).stop 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 msgr2=0x7fbc18159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 0x7fbc18159620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009890 tx=0x7fbc000046c0 comp rx=0 tx=0).stop 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 shutdown_connections 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbbf806c8f0 0x7fbbf806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc18155950 0x7fbc18159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc180d0c90 0x7fbc18159620 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 --2- 172.21.15.27:0/2106046631 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbc180b54f0 0x7fbc180c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 >> 172.21.15.27:0/2106046631 conn(0x7fbc180676c0 msgr2=0x7fbc180d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:42.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 shutdown_connections 2024-04-20T10:59:42.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:42.232+0000 7fbc1cc48640 1 -- 172.21.15.27:0/2106046631 wait complete. 2024-04-20T10:59:42.243 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:59:42.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:59:42.397 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:59:44.244 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:44.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.359+0000 7f8f03304640 1 Processor -- start 2024-04-20T10:59:44.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.359+0000 7f8f03304640 1 -- start start 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.360+0000 7f8f03304640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8efc152260 0x7f8efc0719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.360+0000 7f8f03304640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152640 0x7f8efc0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.360+0000 7f8f03304640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8efc15ba80 con 0x7f8efc0cd6f0 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.360+0000 7f8f03304640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f8efc15bd60 con 0x7f8efc152260 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.360+0000 7f8f03304640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8efc15c040 con 0x7f8efc152640 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.360+0000 7f8f018a1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8efc0cd6f0 0x7f8efc158760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44304/0 (socket says 172.21.15.27:44304) 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.360+0000 7f8f018a1640 1 -- 172.21.15.27:0/2635995719 learned_addr learned my addr 172.21.15.27:0/2635995719 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.360+0000 7f8f010a0640 1 --2- 172.21.15.27:0/2635995719 >> v2:172.21.15.137:3300/0 conn(0x7f8efc152260 0x7f8efc0719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.361+0000 7f8f0089f640 1 --2- 172.21.15.27:0/2635995719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152640 0x7f8efc0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.361+0000 7f8efbfff640 1 -- 172.21.15.27:0/2635995719 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2907892663 0 0) 0x7f8efc15ba80 con 0x7f8efc0cd6f0 2024-04-20T10:59:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.361+0000 7f8efbfff640 1 -- 172.21.15.27:0/2635995719 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ee4005730 con 0x7f8efc0cd6f0 2024-04-20T10:59:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.361+0000 7f8f010a0640 1 -- 172.21.15.27:0/2635995719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152640 msgr2=0x7f8efc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.361+0000 7f8f010a0640 1 --2- 172.21.15.27:0/2635995719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152640 0x7f8efc0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.361+0000 7f8f010a0640 1 -- 172.21.15.27:0/2635995719 >> v1:172.21.15.27:6789/0 conn(0x7f8efc0cd6f0 legacy=0x7f8efc158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.361+0000 7f8f010a0640 1 -- 172.21.15.27:0/2635995719 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc15c320 con 0x7f8efc152260 2024-04-20T10:59:44.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.361+0000 7f8f0089f640 1 --2- 172.21.15.27:0/2635995719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152640 0x7f8efc0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:59:44.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8f010a0640 1 --2- 172.21.15.27:0/2635995719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8efc152260 0x7f8efc0719c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40097a0 tx=0x7f8ef402b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43692b894daefc58 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8efbfff640 1 -- 172.21.15.27:0/2635995719 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4034070 con 0x7f8efc152260 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8efbfff640 1 -- 172.21.15.27:0/2635995719 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef4030800 con 0x7f8efc152260 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8efbfff640 1 -- 172.21.15.27:0/2635995719 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4030bb0 con 0x7f8efc152260 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8f03304640 1 -- 172.21.15.27:0/2635995719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8efc152260 msgr2=0x7f8efc0719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8f03304640 1 --2- 172.21.15.27:0/2635995719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8efc152260 0x7f8efc0719c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40097a0 tx=0x7f8ef402b940 comp rx=0 tx=0).stop 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8f03304640 1 -- 172.21.15.27:0/2635995719 shutdown_connections 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8f03304640 1 --2- 172.21.15.27:0/2635995719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152640 0x7f8efc0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8f03304640 1 --2- 172.21.15.27:0/2635995719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8efc152260 0x7f8efc0719c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.362+0000 7f8f03304640 1 -- 172.21.15.27:0/2635995719 >> 172.21.15.27:0/2635995719 conn(0x7f8efc0ccbd0 msgr2=0x7f8efc0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:44.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.363+0000 7f8f03304640 1 -- 172.21.15.27:0/2635995719 shutdown_connections 2024-04-20T10:59:44.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.363+0000 7f8f03304640 1 -- 172.21.15.27:0/2635995719 wait complete. 2024-04-20T10:59:44.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.364+0000 7f8f03304640 1 Processor -- start 2024-04-20T10:59:44.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.364+0000 7f8f03304640 1 -- start start 2024-04-20T10:59:44.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.364+0000 7f8f03304640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8efc0cd6f0 0x7f8efc144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.364+0000 7f8f03304640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152260 0x7f8efc150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.364+0000 7f8f03304640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8efc152640 0x7f8efc1f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.364+0000 7f8f03304640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc15c600 con 0x7f8efc152640 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.364+0000 7f8f03304640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc074260 con 0x7f8efc0cd6f0 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.364+0000 7f8f03304640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8efc15b720 con 0x7f8efc152260 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f018a1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8efc152640 0x7f8efc1f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f018a1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8efc152640 0x7f8efc1f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36896/0 (socket says 172.21.15.27:36896) 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f018a1640 1 -- 172.21.15.27:0/2203514380 learned_addr learned my addr 172.21.15.27:0/2203514380 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f0089f640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152260 0x7f8efc150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f010a0640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8efc0cd6f0 0x7f8efc144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f018a1640 1 -- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152260 msgr2=0x7f8efc150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f018a1640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152260 0x7f8efc150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:44.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f018a1640 1 -- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8efc0cd6f0 msgr2=0x7f8efc144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f018a1640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8efc0cd6f0 0x7f8efc144c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f018a1640 1 -- 172.21.15.27:0/2203514380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc1f20a0 con 0x7f8efc152640 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f018a1640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8efc152640 0x7f8efc1f4a70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00caa0 tx=0x7f8eec00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8ef9ffb640 1 -- 172.21.15.27:0/2203514380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec017070 con 0x7f8efc152640 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8ef9ffb640 1 -- 172.21.15.27:0/2203514380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec00ac00 con 0x7f8efc152640 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8efc1f2340 con 0x7f8efc152640 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.365+0000 7f8ef9ffb640 1 -- 172.21.15.27:0/2203514380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec012400 con 0x7f8efc152640 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.366+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8efc144420 con 0x7f8efc152640 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.367+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8efc0c8ef0 con 0x7f8efc152640 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.367+0000 7f8ef9ffb640 1 -- 172.21.15.27:0/2203514380 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8eec01a070 con 0x7f8efc152640 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.367+0000 7f8ef9ffb640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ee806c8f0 0x7f8ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.368+0000 7f8ef9ffb640 1 -- 172.21.15.27:0/2203514380 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f8eec08bac0 con 0x7f8efc152640 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.368+0000 7f8f010a0640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ee806c8f0 0x7f8ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:44.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.368+0000 7f8f010a0640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ee806c8f0 0x7f8ee806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4036000 tx=0x7f8ef403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:44.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.372+0000 7f8ef9ffb640 1 -- 172.21.15.27:0/2203514380 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8efc0c8ef0 con 0x7f8efc152640 2024-04-20T10:59:44.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.598+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8efc0c7030 con 0x7f8efc152640 2024-04-20T10:59:44.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.599+0000 7f8ef9ffb640 1 -- 172.21.15.27:0/2203514380 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+6109 (secure 0 0 0) 0x7f8eec059690 con 0x7f8efc152640 2024-04-20T10:59:44.599 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:44.599 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:42.348284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:resolve","state_seq":16,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275]},"id":1}]} 2024-04-20T10:59:44.599 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2024-04-20T10:59:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ee806c8f0 msgr2=0x7f8ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ee806c8f0 0x7f8ee806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4036000 tx=0x7f8ef403b040 comp rx=0 tx=0).stop 2024-04-20T10:59:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8efc152640 msgr2=0x7f8efc1f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8efc152640 0x7f8efc1f4a70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00caa0 tx=0x7f8eec00cf60 comp rx=0 tx=0).stop 2024-04-20T10:59:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 shutdown_connections 2024-04-20T10:59:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ee806c8f0 0x7f8ee806edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8efc152640 0x7f8efc1f4a70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8efc152260 0x7f8efc150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:44.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 --2- 172.21.15.27:0/2203514380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8efc0cd6f0 0x7f8efc144c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 >> 172.21.15.27:0/2203514380 conn(0x7f8efc0ccbd0 msgr2=0x7f8efc0c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 shutdown_connections 2024-04-20T10:59:44.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:44.601+0000 7f8f03304640 1 -- 172.21.15.27:0/2203514380 wait complete. 2024-04-20T10:59:44.612 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:59:46.614 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:46.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.727+0000 7f99fb0d2640 1 Processor -- start 2024-04-20T10:59:46.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.727+0000 7f99fb0d2640 1 -- start start 2024-04-20T10:59:46.728 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99fb0d2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 0x7f99f40cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99fb0d2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99f40b5c00 0x7f99f40d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99fb0d2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f415b9c0 con 0x7f99f40cc330 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99fb0d2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f99f415bca0 con 0x7f99f40b5c00 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99fb0d2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f99f415bf80 con 0x7f99f4155950 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99f966f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99f40b5c00 0x7f99f40d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99f3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f99f40cc330 0x7f99f40b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37050/0 (socket says 172.21.15.27:37050) 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99f3fff640 1 -- 172.21.15.27:0/192087586 learned_addr learned my addr 172.21.15.27:0/192087586 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99f8e6e640 1 --2- 172.21.15.27:0/192087586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 0x7f99f40cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99f966f640 1 -- 172.21.15.27:0/192087586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 msgr2=0x7f99f40cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99f966f640 1 --2- 172.21.15.27:0/192087586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 0x7f99f40cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99f966f640 1 -- 172.21.15.27:0/192087586 >> v1:172.21.15.27:6789/0 conn(0x7f99f40cc330 legacy=0x7f99f40b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99f37fe640 1 -- 172.21.15.27:0/192087586 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2424386601 0 0) 0x7f99f415b9c0 con 0x7f99f40cc330 2024-04-20T10:59:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.728+0000 7f99f966f640 1 -- 172.21.15.27:0/192087586 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f415c260 con 0x7f99f40b5c00 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.729+0000 7f99f966f640 1 --2- 172.21.15.27:0/192087586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40b5c00 0x7f99f40d0c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009d90 tx=0x7f99ec02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a194c209d455160 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.729+0000 7f99f37fe640 1 -- 172.21.15.27:0/192087586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec039070 con 0x7f99f40b5c00 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.729+0000 7f99f37fe640 1 -- 172.21.15.27:0/192087586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ec02cce0 con 0x7f99f40b5c00 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.729+0000 7f99f37fe640 1 -- 172.21.15.27:0/192087586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec034660 con 0x7f99f40b5c00 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.730+0000 7f99fb0d2640 1 -- 172.21.15.27:0/192087586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40b5c00 msgr2=0x7f99f40d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.730+0000 7f99fb0d2640 1 --2- 172.21.15.27:0/192087586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40b5c00 0x7f99f40d0c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009d90 tx=0x7f99ec02bf30 comp rx=0 tx=0).stop 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.730+0000 7f99fb0d2640 1 -- 172.21.15.27:0/192087586 shutdown_connections 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.730+0000 7f99fb0d2640 1 --2- 172.21.15.27:0/192087586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40b5c00 0x7f99f40d0c90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.730+0000 7f99fb0d2640 1 --2- 172.21.15.27:0/192087586 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 0x7f99f40cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.730+0000 7f99fb0d2640 1 -- 172.21.15.27:0/192087586 >> 172.21.15.27:0/192087586 conn(0x7f99f40676c0 msgr2=0x7f99f4067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:46.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.730+0000 7f99fb0d2640 1 -- 172.21.15.27:0/192087586 shutdown_connections 2024-04-20T10:59:46.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.730+0000 7f99fb0d2640 1 -- 172.21.15.27:0/192087586 wait complete. 2024-04-20T10:59:46.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.731+0000 7f99fb0d2640 1 Processor -- start 2024-04-20T10:59:46.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.731+0000 7f99fb0d2640 1 -- start start 2024-04-20T10:59:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99fb0d2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99f40b5c00 0x7f99f41e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99fb0d2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40cc330 0x7f99f41f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99fb0d2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 0x7f99f40c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99fb0d2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99f415fd10 con 0x7f99f40cc330 2024-04-20T10:59:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99fb0d2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f99f415fb90 con 0x7f99f40b5c00 2024-04-20T10:59:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99fb0d2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f99f4155d30 con 0x7f99f4155950 2024-04-20T10:59:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40cc330 0x7f99f41f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40cc330 0x7f99f41f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38118/0 (socket says 172.21.15.27:38118) 2024-04-20T10:59:46.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99f3fff640 1 -- 172.21.15.27:0/3454052196 learned_addr learned my addr 172.21.15.27:0/3454052196 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99f966f640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 0x7f99f40c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.732+0000 7f99f8e6e640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99f40b5c00 0x7f99f41e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99f3fff640 1 -- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 msgr2=0x7f99f40c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99f3fff640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 0x7f99f40c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99f3fff640 1 -- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99f40b5c00 msgr2=0x7f99f41e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99f3fff640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99f40b5c00 0x7f99f41e75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99f3fff640 1 -- 172.21.15.27:0/3454052196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f41e6850 con 0x7f99f40cc330 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99f3fff640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40cc330 0x7f99f41f4e50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f99e000c430 tx=0x7f99e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99f17fa640 1 -- 172.21.15.27:0/3454052196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e00074e0 con 0x7f99f40cc330 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99f17fa640 1 -- 172.21.15.27:0/3454052196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e00047e0 con 0x7f99f40cc330 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99f17fa640 1 -- 172.21.15.27:0/3454052196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0004bd0 con 0x7f99f40cc330 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f41e6b30 con 0x7f99f40cc330 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.733+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f41f4930 con 0x7f99f40cc330 2024-04-20T10:59:46.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.734+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f404fcf0 con 0x7f99f40cc330 2024-04-20T10:59:46.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.735+0000 7f99f17fa640 1 -- 172.21.15.27:0/3454052196 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99e0011070 con 0x7f99f40cc330 2024-04-20T10:59:46.738 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.735+0000 7f99f17fa640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99d806c8f0 0x7f99d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:46.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.735+0000 7f99f17fa640 1 -- 172.21.15.27:0/3454052196 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f99e008b880 con 0x7f99f40cc330 2024-04-20T10:59:46.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.735+0000 7f99f8e6e640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99d806c8f0 0x7f99d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:46.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.736+0000 7f99f8e6e640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99d806c8f0 0x7f99d806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009b40 tx=0x7f99e8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:46.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.740+0000 7f99f17fa640 1 -- 172.21.15.27:0/3454052196 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f99e0059450 con 0x7f99f40cc330 2024-04-20T10:59:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.968+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99f40cc930 con 0x7f99f40cc330 2024-04-20T10:59:46.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.968+0000 7f99f17fa640 1 -- 172.21.15.27:0/3454052196 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+6109 (secure 0 0 0) 0x7f99e005d750 con 0x7f99f40cc330 2024-04-20T10:59:46.968 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:46.968 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:42.348284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:resolve","state_seq":16,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275]},"id":1}]} 2024-04-20T10:59:46.969 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99d806c8f0 msgr2=0x7f99d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99d806c8f0 0x7f99d806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009b40 tx=0x7f99e8009190 comp rx=0 tx=0).stop 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40cc330 msgr2=0x7f99f41f4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40cc330 0x7f99f41f4e50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f99e000c430 tx=0x7f99e000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 shutdown_connections 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f99d806c8f0 0x7f99d806edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f99f4155950 0x7f99f40c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99f40cc330 0x7f99f41f4e50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 --2- 172.21.15.27:0/3454052196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99f40b5c00 0x7f99f41e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 >> 172.21.15.27:0/3454052196 conn(0x7f99f40676c0 msgr2=0x7f99f4067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 shutdown_connections 2024-04-20T10:59:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:46.971+0000 7f99fb0d2640 1 -- 172.21.15.27:0/3454052196 wait complete. 2024-04-20T10:59:46.984 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:59:48.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:49.092 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.092+0000 7fac08d0e640 1 Processor -- start 2024-04-20T10:59:49.093 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.092+0000 7fac08d0e640 1 -- start start 2024-04-20T10:59:49.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac08d0e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac0415f230 0x7fac0415c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:49.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac08d0e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac0415f9a0 con 0x7fac040ccbd0 2024-04-20T10:59:49.096 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac08d0e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac040cb870 con 0x7fac0415ee50 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac08d0e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fac040cbb30 con 0x7fac0415f230 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac02d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fac040ccbd0 0x7fac04155410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37064/0 (socket says 172.21.15.27:37064) 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac02d9d640 1 -- 172.21.15.27:0/3852242382 learned_addr learned my addr 172.21.15.27:0/3852242382 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac01d9b640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac0415f230 0x7fac0415c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1600051494 0 0) 0x7fabfc001490 con 0x7fac040ccbd0 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabf0003520 con 0x7fac040ccbd0 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3306926151 0 0) 0x7fabf0003520 con 0x7fac040ccbd0 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac0415f230 msgr2=0x7fac0415c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac0415f230 0x7fac0415c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 >> v1:172.21.15.137:6789/0 conn(0x7fac0415ee50 legacy=0x7fac0406cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac040cbe10 con 0x7fac040ccbd0 2024-04-20T10:59:49.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7fabfc001bc0 con 0x7fac040ccbd0 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 >> v1:172.21.15.27:6789/0 conn(0x7fac040ccbd0 legacy=0x7fac04155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fabf0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac0415f230 0x7fabf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabf0004d40 0x7fabf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fac040cbb30 con 0x7fabf00039e0 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fabf000b9c0 con 0x7fabf0004d40 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fabf000bbc0 con 0x7fac0415f230 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.093+0000 7fac0259c640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fabf0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.094+0000 7fac01d9b640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac0415f230 0x7fabf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.094+0000 7fac02d9d640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabf0004d40 0x7fabf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.094+0000 7fac0259c640 1 -- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac0415f230 msgr2=0x7fabf0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.094+0000 7fac0259c640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac0415f230 0x7fabf0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.094+0000 7fac0259c640 1 -- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabf0004d40 msgr2=0x7fabf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:49.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.094+0000 7fac0259c640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabf0004d40 0x7fabf00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.094+0000 7fac0259c640 1 -- 172.21.15.27:0/3852242382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf0003520 con 0x7fabf00039e0 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac0259c640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fabf0003dc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fabf4001860 tx=0x7fabf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93568429dca614d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4038070 con 0x7fabf00039e0 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf4033660 con 0x7fabf00039e0 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac0159a640 1 -- 172.21.15.27:0/3852242382 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4001430 con 0x7fabf00039e0 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac08d0e640 1 -- 172.21.15.27:0/3852242382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 msgr2=0x7fabf0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac08d0e640 1 --2- 172.21.15.27:0/3852242382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fabf0003dc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fabf4001860 tx=0x7fabf0005830 comp rx=0 tx=0).stop 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac02d9d640 1 -- 172.21.15.27:0/3852242382 reap_dead start 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac08d0e640 1 -- 172.21.15.27:0/3852242382 shutdown_connections 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac08d0e640 1 -- 172.21.15.27:0/3852242382 >> 172.21.15.27:0/3852242382 conn(0x7fac0406e010 msgr2=0x7fac040d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac08d0e640 1 -- 172.21.15.27:0/3852242382 shutdown_connections 2024-04-20T10:59:49.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.095+0000 7fac08d0e640 1 -- 172.21.15.27:0/3852242382 wait complete. 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.097+0000 7fac08d0e640 1 Processor -- start 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.097+0000 7fac08d0e640 1 -- start start 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.097+0000 7fac08d0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fac040c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.097+0000 7fac08d0e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac041f46c0 0x7fac0414c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.097+0000 7fac08d0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac0414c7a0 0x7fac0414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.097+0000 7fac08d0e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fac040cc6b0 con 0x7fabf00039e0 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.097+0000 7fac08d0e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac040cc3d0 con 0x7fac0414c7a0 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.097+0000 7fac08d0e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fac0415cae0 con 0x7fac041f46c0 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.097+0000 7fac0259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fac040c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:49.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac0259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fac040c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38142/0 (socket says 172.21.15.27:38142) 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac0259c640 1 -- 172.21.15.27:0/2366339213 learned_addr learned my addr 172.21.15.27:0/2366339213 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac02d9d640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac0414c7a0 0x7fac0414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac01d9b640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac041f46c0 0x7fac0414c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac0259c640 1 -- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac041f46c0 msgr2=0x7fac0414c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac0259c640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac041f46c0 0x7fac0414c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac0259c640 1 -- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac0414c7a0 msgr2=0x7fac0414f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac0259c640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac0414c7a0 0x7fac0414f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac0259c640 1 -- 172.21.15.27:0/2366339213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac040cae00 con 0x7fabf00039e0 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac0259c640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fac040c9e80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fabf402bd30 tx=0x7fabf4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fabfaffd640 1 -- 172.21.15.27:0/2366339213 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4043020 con 0x7fabf00039e0 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fabfaffd640 1 -- 172.21.15.27:0/2366339213 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf4039040 con 0x7fabf00039e0 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fabfaffd640 1 -- 172.21.15.27:0/2366339213 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4038070 con 0x7fabf00039e0 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0414d2a0 con 0x7fabf00039e0 2024-04-20T10:59:49.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.098+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac0414d840 con 0x7fabf00039e0 2024-04-20T10:59:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.099+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac040688e0 con 0x7fabf00039e0 2024-04-20T10:59:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.100+0000 7fabfaffd640 1 -- 172.21.15.27:0/2366339213 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabf4046020 con 0x7fabf00039e0 2024-04-20T10:59:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.100+0000 7fabfaffd640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fabe406c8f0 0x7fabe406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.100+0000 7fabfaffd640 1 -- 172.21.15.27:0/2366339213 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7fabf40ad7d0 con 0x7fabf00039e0 2024-04-20T10:59:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.100+0000 7fac01d9b640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fabe406c8f0 0x7fabe406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:49.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.101+0000 7fac01d9b640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fabe406c8f0 0x7fabe406edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fac040ca990 tx=0x7fabec009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:49.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.105+0000 7fabfaffd640 1 -- 172.21.15.27:0/2366339213 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fabf404d030 con 0x7fabf00039e0 2024-04-20T10:59:49.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.345+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac040693a0 con 0x7fabf00039e0 2024-04-20T10:59:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.346+0000 7fabfaffd640 1 -- 172.21.15.27:0/2366339213 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6111 (secure 0 0 0) 0x7fac040693a0 con 0x7fabf00039e0 2024-04-20T10:59:49.347 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:49.347 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:47.390251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4275,4278,4293,4894]},"id":1}]} 2024-04-20T10:59:49.347 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 32 2024-04-20T10:59:49.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fabe406c8f0 msgr2=0x7fabe406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:49.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fabe406c8f0 0x7fabe406edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fac040ca990 tx=0x7fabec009190 comp rx=0 tx=0).stop 2024-04-20T10:59:49.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 msgr2=0x7fac040c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:49.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fac040c9e80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fabf402bd30 tx=0x7fabf4001230 comp rx=0 tx=0).stop 2024-04-20T10:59:49.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 shutdown_connections 2024-04-20T10:59:49.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fabe406c8f0 0x7fabe406edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:49.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac0414c7a0 0x7fac0414f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:49.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fac041f46c0 0x7fac0414c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:49.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 --2- 172.21.15.27:0/2366339213 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabf00039e0 0x7fac040c9e80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:49.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 >> 172.21.15.27:0/2366339213 conn(0x7fac0406e010 msgr2=0x7fac0406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:49.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 shutdown_connections 2024-04-20T10:59:49.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:49.348+0000 7fac08d0e640 1 -- 172.21.15.27:0/2366339213 wait complete. 2024-04-20T10:59:49.361 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:59:51.362 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.465+0000 7fe1a5bbd640 1 Processor -- start 2024-04-20T10:59:51.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.465+0000 7fe1a5bbd640 1 -- start start 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe1a5bbd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe1a0158e70 0x7fe1a00719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe1a5bbd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a015ba80 con 0x7fe1a0152260 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe1a5bbd640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a015bd60 con 0x7fe1a0152640 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe1a5bbd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a015c040 con 0x7fe1a0158e70 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe19ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe1a0158e70 0x7fe1a00719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe19f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe1a0152260 0x7fe1a00c86e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37068/0 (socket says 172.21.15.27:37068) 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe19f7fe640 1 -- 172.21.15.27:0/1068545524 learned_addr learned my addr 172.21.15.27:0/1068545524 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe19ffff640 1 -- 172.21.15.27:0/1068545524 >> v1:172.21.15.137:6789/0 conn(0x7fe1a0152640 legacy=0x7fe1a0158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe19ffff640 1 -- 172.21.15.27:0/1068545524 >> v1:172.21.15.27:6789/0 conn(0x7fe1a0152260 legacy=0x7fe1a00c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:51.467 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.466+0000 7fe19ffff640 1 -- 172.21.15.27:0/1068545524 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a015c320 con 0x7fe1a0158e70 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.467+0000 7fe19ffff640 1 --2- 172.21.15.27:0/1068545524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0158e70 0x7fe1a00719c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe198009d90 tx=0x7fe19802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e0f3b8fdbe7171 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.467+0000 7fe19e7fc640 1 -- 172.21.15.27:0/1068545524 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198039070 con 0x7fe1a0158e70 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.467+0000 7fe19e7fc640 1 -- 172.21.15.27:0/1068545524 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19802cd80 con 0x7fe1a0158e70 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.467+0000 7fe19e7fc640 1 -- 172.21.15.27:0/1068545524 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198034660 con 0x7fe1a0158e70 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.468+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/1068545524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0158e70 msgr2=0x7fe1a00719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.468+0000 7fe1a5bbd640 1 --2- 172.21.15.27:0/1068545524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0158e70 0x7fe1a00719c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe198009d90 tx=0x7fe19802bfd0 comp rx=0 tx=0).stop 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.468+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/1068545524 shutdown_connections 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.468+0000 7fe1a5bbd640 1 --2- 172.21.15.27:0/1068545524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0158e70 0x7fe1a00719c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.468+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/1068545524 >> 172.21.15.27:0/1068545524 conn(0x7fe1a00ccbd0 msgr2=0x7fe1a00ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.468+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/1068545524 shutdown_connections 2024-04-20T10:59:51.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.468+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/1068545524 wait complete. 2024-04-20T10:59:51.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.469+0000 7fe1a5bbd640 1 Processor -- start 2024-04-20T10:59:51.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.469+0000 7fe1a5bbd640 1 -- start start 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe1a5bbd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1a0152260 0x7fe1a01f1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe1a5bbd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0152640 0x7fe1a0072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe1a5bbd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0158e70 0x7fe1a00c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe1a5bbd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a015c600 con 0x7fe1a0152640 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe1a5bbd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a0074260 con 0x7fe1a0158e70 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe1a5bbd640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a015b720 con 0x7fe1a0152260 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0152640 0x7fe1a0072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0152640 0x7fe1a0072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38158/0 (socket says 172.21.15.27:38158) 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19effd640 1 -- 172.21.15.27:0/828987517 learned_addr learned my addr 172.21.15.27:0/828987517 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:51.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19ffff640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0158e70 0x7fe1a00c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19f7fe640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1a0152260 0x7fe1a01f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19effd640 1 -- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1a0152260 msgr2=0x7fe1a01f1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19effd640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1a0152260 0x7fe1a01f1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19effd640 1 -- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0158e70 msgr2=0x7fe1a00c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19effd640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0158e70 0x7fe1a00c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19effd640 1 -- 172.21.15.27:0/828987517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a0147f10 con 0x7fe1a0152640 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19ffff640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0158e70 0x7fe1a00c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe19effd640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0152640 0x7fe1a0072d10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c400 tx=0x7fe19400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe187fff640 1 -- 172.21.15.27:0/828987517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1940052d0 con 0x7fe1a0152640 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe187fff640 1 -- 172.21.15.27:0/828987517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe194005490 con 0x7fe1a0152640 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a01481f0 con 0x7fe1a0152640 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe187fff640 1 -- 172.21.15.27:0/828987517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194007260 con 0x7fe1a0152640 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.470+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a0159b40 con 0x7fe1a0152640 2024-04-20T10:59:51.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.471+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a0156a20 con 0x7fe1a0152640 2024-04-20T10:59:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.472+0000 7fe187fff640 1 -- 172.21.15.27:0/828987517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe194007400 con 0x7fe1a0152640 2024-04-20T10:59:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.472+0000 7fe187fff640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe18c06c8f0 0x7fe18c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.472+0000 7fe187fff640 1 -- 172.21.15.27:0/828987517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7fe19408b200 con 0x7fe1a0152640 2024-04-20T10:59:51.474 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.472+0000 7fe19f7fe640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe18c06c8f0 0x7fe18c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:51.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.473+0000 7fe19f7fe640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe18c06c8f0 0x7fe18c06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe190005880 tx=0x7fe190009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:51.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.477+0000 7fe187fff640 1 -- 172.21.15.27:0/828987517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe1940594b0 con 0x7fe1a0152640 2024-04-20T10:59:51.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.699+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1a0159930 con 0x7fe1a0152640 2024-04-20T10:59:51.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.700+0000 7fe187fff640 1 -- 172.21.15.27:0/828987517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+6111 (secure 0 0 0) 0x7fe194058db0 con 0x7fe1a0152640 2024-04-20T10:59:51.701 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:51.701 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:47.390251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4275,4278,4293,4894]},"id":1}]} 2024-04-20T10:59:51.701 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 32 2024-04-20T10:59:51.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe18c06c8f0 msgr2=0x7fe18c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:51.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe18c06c8f0 0x7fe18c06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe190005880 tx=0x7fe190009040 comp rx=0 tx=0).stop 2024-04-20T10:59:51.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0152640 msgr2=0x7fe1a0072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:51.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0152640 0x7fe1a0072d10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c400 tx=0x7fe19400c8c0 comp rx=0 tx=0).stop 2024-04-20T10:59:51.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 shutdown_connections 2024-04-20T10:59:51.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe18c06c8f0 0x7fe18c06edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:51.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0158e70 0x7fe1a00c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:51.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1a0152640 0x7fe1a0072d10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:51.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 --2- 172.21.15.27:0/828987517 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe1a0152260 0x7fe1a01f1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:51.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 >> 172.21.15.27:0/828987517 conn(0x7fe1a00ccbd0 msgr2=0x7fe1a00c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:51.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 shutdown_connections 2024-04-20T10:59:51.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:51.702+0000 7fe1a5bbd640 1 -- 172.21.15.27:0/828987517 wait complete. 2024-04-20T10:59:51.715 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 32, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 32, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4275": { "addr": "172.21.15.27:6837/1680990583", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6834", "nonce": 1680990583, "type": "v2" }, { "addr": "172.21.15.27:6837", "nonce": 1680990583, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4275, "incarnation": 12, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 7 }, "gid_4278": { "addr": "172.21.15.27:6836/4195543922", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6833", "nonce": 4195543922, "type": "v2" }, { "addr": "172.21.15.27:6836", "nonce": 4195543922, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4278, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4894": { "addr": "172.21.15.137:6838/2896327439", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6836", "nonce": 2896327439, "type": "v2" }, { "addr": "172.21.15.137:6838", "nonce": 2896327439, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4894, "incarnation": 24, "join_fscid": -1, "name": "b", "rank": 4, "state": "up:active", "state_seq": 17 }, "gid_5029": { "addr": "172.21.15.137:6835/4235742413", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6834", "nonce": 4235742413, "type": "v2" }, { "addr": "172.21.15.137:6835", "nonce": 4235742413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5029, "incarnation": 30, "join_fscid": -1, "name": "f", "rank": 3, "state": "up:reconnect", "state_seq": 18 } }, "last_failure": 0, "last_failure_osd_epoch": 25, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T10:59:47.390251+0000", "qdb_cluster": [ 4275, 4278, 4293, 4894 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 4278, "mds_2": 4275, "mds_3": 5029, "mds_4": 4894 } } } ], "standbys": [] } 2024-04-20T10:59:53.715 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:53.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.819+0000 7f4759481640 1 Processor -- start 2024-04-20T10:59:53.820 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.819+0000 7f4759481640 1 -- start start 2024-04-20T10:59:53.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f4759481640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4754155950 0x7f47540cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:53.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f4759481640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f47540cc330 0x7f47540b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:53.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f4759481640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f475415b9c0 con 0x7f47540b5a30 2024-04-20T10:59:53.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f4759481640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f475415bca0 con 0x7f47540cc330 2024-04-20T10:59:53.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f4759481640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f475415bf80 con 0x7f4754155950 2024-04-20T10:59:53.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f47527fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f47540cc330 0x7f47540b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:53.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f47537fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f47540b5a30 0x7f47540d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37072/0 (socket says 172.21.15.27:37072) 2024-04-20T10:59:53.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f47537fe640 1 -- 172.21.15.27:0/3149208337 learned_addr learned my addr 172.21.15.27:0/3149208337 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:53.823 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f4752ffd640 1 --2- 172.21.15.27:0/3149208337 >> v2:172.21.15.137:3300/0 conn(0x7f4754155950 0x7f47540cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f47527fc640 1 -- 172.21.15.27:0/3149208337 >> v2:172.21.15.137:3300/0 conn(0x7f4754155950 msgr2=0x7f47540cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f47527fc640 1 --2- 172.21.15.27:0/3149208337 >> v2:172.21.15.137:3300/0 conn(0x7f4754155950 0x7f47540cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f47527fc640 1 -- 172.21.15.27:0/3149208337 >> v1:172.21.15.27:6789/0 conn(0x7f47540b5a30 legacy=0x7f47540d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f4751ffb640 1 -- 172.21.15.27:0/3149208337 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3333945539 0 0) 0x7f475415b9c0 con 0x7f47540b5a30 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.820+0000 7f47527fc640 1 -- 172.21.15.27:0/3149208337 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475415c260 con 0x7f47540cc330 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.821+0000 7f47527fc640 1 --2- 172.21.15.27:0/3149208337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47540cc330 0x7f47540b54f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f47540ce7c0 tx=0x7f473c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43371d3521e1dbdd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.821+0000 7f4751ffb640 1 -- 172.21.15.27:0/3149208337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c034070 con 0x7f47540cc330 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.821+0000 7f4751ffb640 1 -- 172.21.15.27:0/3149208337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f473c030610 con 0x7f47540cc330 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.821+0000 7f4751ffb640 1 -- 172.21.15.27:0/3149208337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c0309e0 con 0x7f47540cc330 2024-04-20T10:59:53.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.822+0000 7f4759481640 1 -- 172.21.15.27:0/3149208337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47540cc330 msgr2=0x7f47540b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:53.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.822+0000 7f4759481640 1 --2- 172.21.15.27:0/3149208337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47540cc330 0x7f47540b54f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f47540ce7c0 tx=0x7f473c02b960 comp rx=0 tx=0).stop 2024-04-20T10:59:53.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.822+0000 7f4759481640 1 -- 172.21.15.27:0/3149208337 shutdown_connections 2024-04-20T10:59:53.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.822+0000 7f4759481640 1 --2- 172.21.15.27:0/3149208337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47540cc330 0x7f47540b54f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:53.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.822+0000 7f4759481640 1 --2- 172.21.15.27:0/3149208337 >> v2:172.21.15.137:3300/0 conn(0x7f4754155950 0x7f47540cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:53.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.822+0000 7f4759481640 1 -- 172.21.15.27:0/3149208337 >> 172.21.15.27:0/3149208337 conn(0x7f47540676c0 msgr2=0x7f4754067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:53.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.822+0000 7f4759481640 1 -- 172.21.15.27:0/3149208337 shutdown_connections 2024-04-20T10:59:53.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.822+0000 7f4759481640 1 -- 172.21.15.27:0/3149208337 wait complete. 2024-04-20T10:59:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.823+0000 7f4759481640 1 Processor -- start 2024-04-20T10:59:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.823+0000 7f4759481640 1 -- start start 2024-04-20T10:59:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f4759481640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47540b5a30 0x7f47541f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f4759481640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47540cc330 0x7f47541548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f4759481640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4754155950 0x7f475415e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f4759481640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f475415fb90 con 0x7f4754155950 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f4759481640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47540b5e40 con 0x7f47540b5a30 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f4759481640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f475415fd10 con 0x7f47540cc330 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47537fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4754155950 0x7f475415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47537fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4754155950 0x7f475415e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38180/0 (socket says 172.21.15.27:38180) 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47537fe640 1 -- 172.21.15.27:0/1774919367 learned_addr learned my addr 172.21.15.27:0/1774919367 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f4752ffd640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47540b5a30 0x7f47541f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47527fc640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47540cc330 0x7f47541548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47537fe640 1 -- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47540cc330 msgr2=0x7f47541548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47537fe640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47540cc330 0x7f47541548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47537fe640 1 -- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47540b5a30 msgr2=0x7f47541f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47537fe640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47540b5a30 0x7f47541f1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47537fe640 1 -- 172.21.15.27:0/1774919367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4754155120 con 0x7f4754155950 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.824+0000 7f47537fe640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4754155950 0x7f475415e350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f474400c830 tx=0x7f474400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.825+0000 7f474b7fe640 1 -- 172.21.15.27:0/1774919367 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744018070 con 0x7f4754155950 2024-04-20T10:59:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.825+0000 7f474b7fe640 1 -- 172.21.15.27:0/1774919367 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474400ce20 con 0x7f4754155950 2024-04-20T10:59:53.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.825+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4754152c80 con 0x7f4754155950 2024-04-20T10:59:53.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.825+0000 7f474b7fe640 1 -- 172.21.15.27:0/1774919367 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474400a880 con 0x7f4754155950 2024-04-20T10:59:53.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.825+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4754153240 con 0x7f4754155950 2024-04-20T10:59:53.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.826+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f475404fcf0 con 0x7f4754155950 2024-04-20T10:59:53.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.826+0000 7f474b7fe640 1 -- 172.21.15.27:0/1774919367 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f474401b070 con 0x7f4754155950 2024-04-20T10:59:53.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.826+0000 7f474b7fe640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f473406c8f0 0x7f473406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:53.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.827+0000 7f474b7fe640 1 -- 172.21.15.27:0/1774919367 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f474408b210 con 0x7f4754155950 2024-04-20T10:59:53.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.827+0000 7f4752ffd640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f473406c8f0 0x7f473406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:53.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.827+0000 7f4752ffd640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f473406c8f0 0x7f473406edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f474c009aa0 tx=0x7f474c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:53.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:53.831+0000 7f474b7fe640 1 -- 172.21.15.27:0/1774919367 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4744058ea0 con 0x7f4754155950 2024-04-20T10:59:54.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.060+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47540cd450 con 0x7f4754155950 2024-04-20T10:59:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.060+0000 7f474b7fe640 1 -- 172.21.15.27:0/1774919367 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6108 (secure 0 0 0) 0x7f474405d1a0 con 0x7f4754155950 2024-04-20T10:59:54.061 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:54.061 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:52.403731+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275]},"id":1}]} 2024-04-20T10:59:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 33 2024-04-20T10:59:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f473406c8f0 msgr2=0x7f473406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f473406c8f0 0x7f473406edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f474c009aa0 tx=0x7f474c009190 comp rx=0 tx=0).stop 2024-04-20T10:59:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4754155950 msgr2=0x7f475415e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4754155950 0x7f475415e350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f474400c830 tx=0x7f474400ccf0 comp rx=0 tx=0).stop 2024-04-20T10:59:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 shutdown_connections 2024-04-20T10:59:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f473406c8f0 0x7f473406edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4754155950 0x7f475415e350 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f47540cc330 0x7f47541548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 --2- 172.21.15.27:0/1774919367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47540b5a30 0x7f47541f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 >> 172.21.15.27:0/1774919367 conn(0x7f47540676c0 msgr2=0x7f47540c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 shutdown_connections 2024-04-20T10:59:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:54.062+0000 7f4759481640 1 -- 172.21.15.27:0/1774919367 wait complete. 2024-04-20T10:59:54.075 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:59:56.076 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:56.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.189+0000 7f674f4a7640 1 Processor -- start 2024-04-20T10:59:56.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.189+0000 7f674f4a7640 1 -- start start 2024-04-20T10:59:56.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.190+0000 7f674f4a7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6748155690 0x7f674815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:56.191 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.190+0000 7f674f4a7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 0x7f6748070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.190+0000 7f674f4a7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67480cf580 con 0x7f674806d770 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.190+0000 7f674f4a7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f674806ca40 con 0x7f6748155690 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.190+0000 7f674f4a7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f67480ca2b0 con 0x7f67480cbe80 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.190+0000 7f674d243640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6748155690 0x7f674815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.190+0000 7f674ca42640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f674806d770 0x7f67480ceb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38736/0 (socket says 172.21.15.27:38736) 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.190+0000 7f674ca42640 1 -- 172.21.15.27:0/2568254649 learned_addr learned my addr 172.21.15.27:0/2568254649 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.190+0000 7f674da44640 1 --2- 172.21.15.27:0/2568254649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 0x7f6748070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.191+0000 7f674d243640 1 -- 172.21.15.27:0/2568254649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 msgr2=0x7f6748070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.191+0000 7f674d243640 1 --2- 172.21.15.27:0/2568254649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 0x7f6748070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.191+0000 7f674d243640 1 -- 172.21.15.27:0/2568254649 >> v1:172.21.15.27:6789/0 conn(0x7f674806d770 legacy=0x7f67480ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.191+0000 7f6747fff640 1 -- 172.21.15.27:0/2568254649 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3632659136 0 0) 0x7f67480cf580 con 0x7f674806d770 2024-04-20T10:59:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.191+0000 7f674d243640 1 -- 172.21.15.27:0/2568254649 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67480cf860 con 0x7f6748155690 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.191+0000 7f674d243640 1 --2- 172.21.15.27:0/2568254649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6748155690 0x7f674815b3f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6740009d90 tx=0x7f674002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19d1e687dfdab35 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f6747fff640 1 -- 172.21.15.27:0/2568254649 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740039070 con 0x7f6748155690 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f6747fff640 1 -- 172.21.15.27:0/2568254649 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674002cbc0 con 0x7f6748155690 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f6747fff640 1 -- 172.21.15.27:0/2568254649 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740034440 con 0x7f6748155690 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f674f4a7640 1 -- 172.21.15.27:0/2568254649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6748155690 msgr2=0x7f674815b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f674f4a7640 1 --2- 172.21.15.27:0/2568254649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6748155690 0x7f674815b3f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6740009d90 tx=0x7f674002be10 comp rx=0 tx=0).stop 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f674f4a7640 1 -- 172.21.15.27:0/2568254649 shutdown_connections 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f674f4a7640 1 --2- 172.21.15.27:0/2568254649 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 0x7f6748070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f674f4a7640 1 --2- 172.21.15.27:0/2568254649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6748155690 0x7f674815b3f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f674f4a7640 1 -- 172.21.15.27:0/2568254649 >> 172.21.15.27:0/2568254649 conn(0x7f674806c070 msgr2=0x7f67480676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f674f4a7640 1 -- 172.21.15.27:0/2568254649 shutdown_connections 2024-04-20T10:59:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.192+0000 7f674f4a7640 1 -- 172.21.15.27:0/2568254649 wait complete. 2024-04-20T10:59:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.194+0000 7f674f4a7640 1 Processor -- start 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.194+0000 7f674f4a7640 1 -- start start 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.194+0000 7f674f4a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f674806d770 0x7f674814de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.194+0000 7f674f4a7640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 0x7f6748148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.194+0000 7f674f4a7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6748155690 0x7f6748152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.194+0000 7f674f4a7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f674806d250 con 0x7f674806d770 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.194+0000 7f674f4a7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f674806cf70 con 0x7f6748155690 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.194+0000 7f674f4a7640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6748155a70 con 0x7f67480cbe80 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674d243640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f674806d770 0x7f674814de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674d243640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f674806d770 0x7f674814de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36268/0 (socket says 172.21.15.27:36268) 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674d243640 1 -- 172.21.15.27:0/482922748 learned_addr learned my addr 172.21.15.27:0/482922748 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674ca42640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 0x7f6748148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674da44640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6748155690 0x7f6748152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674d243640 1 -- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 msgr2=0x7f6748148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674d243640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 0x7f6748148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674d243640 1 -- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6748155690 msgr2=0x7f6748152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674d243640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6748155690 0x7f6748152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674d243640 1 -- 172.21.15.27:0/482922748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67481f4eb0 con 0x7f674806d770 2024-04-20T10:59:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674d243640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f674806d770 0x7f674814de40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6740037040 tx=0x7f6740004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f6745ffb640 1 -- 172.21.15.27:0/482922748 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740039050 con 0x7f674806d770 2024-04-20T10:59:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f6745ffb640 1 -- 172.21.15.27:0/482922748 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674003a040 con 0x7f674806d770 2024-04-20T10:59:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6748154110 con 0x7f674806d770 2024-04-20T10:59:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.195+0000 7f6745ffb640 1 -- 172.21.15.27:0/482922748 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674003e610 con 0x7f674806d770 2024-04-20T10:59:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.196+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67481f56e0 con 0x7f674806d770 2024-04-20T10:59:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.197+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6748073080 con 0x7f674806d770 2024-04-20T10:59:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.197+0000 7f6745ffb640 1 -- 172.21.15.27:0/482922748 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6740047020 con 0x7f674806d770 2024-04-20T10:59:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.197+0000 7f6745ffb640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f673406c8f0 0x7f673406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.197+0000 7f6745ffb640 1 -- 172.21.15.27:0/482922748 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f67400ade60 con 0x7f674806d770 2024-04-20T10:59:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.198+0000 7f674ca42640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f673406c8f0 0x7f673406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.198+0000 7f674ca42640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f673406c8f0 0x7f673406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6748152180 tx=0x7f673c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:56.202 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.202+0000 7f6745ffb640 1 -- 172.21.15.27:0/482922748 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6748073080 con 0x7f674806d770 2024-04-20T10:59:56.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.431+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f674804fcf0 con 0x7f674806d770 2024-04-20T10:59:56.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.431+0000 7f6745ffb640 1 -- 172.21.15.27:0/482922748 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+6108 (secure 0 0 0) 0x7f674004e020 con 0x7f674806d770 2024-04-20T10:59:56.432 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:56.432 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:52.403731+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,4275]},"id":1}]} 2024-04-20T10:59:56.432 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 33 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.433+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f673406c8f0 msgr2=0x7f673406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.433+0000 7f674f4a7640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f673406c8f0 0x7f673406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6748152180 tx=0x7f673c009040 comp rx=0 tx=0).stop 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.433+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f674806d770 msgr2=0x7f674814de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.433+0000 7f674f4a7640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f674806d770 0x7f674814de40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6740037040 tx=0x7f6740004a40 comp rx=0 tx=0).stop 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.433+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 shutdown_connections 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.433+0000 7f674f4a7640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f673406c8f0 0x7f673406edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.433+0000 7f674f4a7640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6748155690 0x7f6748152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.433+0000 7f674f4a7640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f67480cbe80 0x7f6748148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.433+0000 7f674f4a7640 1 --2- 172.21.15.27:0/482922748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f674806d770 0x7f674814de40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.434+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 >> 172.21.15.27:0/482922748 conn(0x7f674806c070 msgr2=0x7f67480676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.434+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 shutdown_connections 2024-04-20T10:59:56.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:56.434+0000 7f674f4a7640 1 -- 172.21.15.27:0/482922748 wait complete. 2024-04-20T10:59:56.445 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T10:59:58.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:59:58.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.552+0000 7f832fe32640 1 Processor -- start 2024-04-20T10:59:58.553 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.552+0000 7f832fe32640 1 -- start start 2024-04-20T10:59:58.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.553+0000 7f832fe32640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8328155950 0x7f83280cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.553+0000 7f832fe32640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83280b5c00 0x7f83280d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.553+0000 7f832fe32640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f832815b9c0 con 0x7f83280cc330 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.553+0000 7f832fe32640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f832815bca0 con 0x7f83280b5c00 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.553+0000 7f832fe32640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f832815bf80 con 0x7f8328155950 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.553+0000 7f832e3cf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83280b5c00 0x7f83280d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.553+0000 7f832e3cf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83280b5c00 0x7f83280d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36276/0 (socket says 172.21.15.27:36276) 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.553+0000 7f832e3cf640 1 -- 172.21.15.27:0/3756922518 learned_addr learned my addr 172.21.15.27:0/3756922518 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.553+0000 7f832dbce640 1 --2- 172.21.15.27:0/3756922518 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8328155950 0x7f83280cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.554+0000 7f832e3cf640 1 -- 172.21.15.27:0/3756922518 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8328155950 msgr2=0x7f83280cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.554+0000 7f832e3cf640 1 --2- 172.21.15.27:0/3756922518 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8328155950 0x7f83280cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.554+0000 7f832e3cf640 1 -- 172.21.15.27:0/3756922518 >> v1:172.21.15.137:6789/0 conn(0x7f83280cc330 legacy=0x7f83280b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.554+0000 7f832e3cf640 1 -- 172.21.15.27:0/3756922518 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832815c260 con 0x7f83280b5c00 2024-04-20T10:59:58.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.554+0000 7f832cbcc640 1 -- 172.21.15.27:0/3756922518 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 105495879 0 0) 0x7f832815b9c0 con 0x7f83280cc330 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.554+0000 7f832e3cf640 1 --2- 172.21.15.27:0/3756922518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 0x7f83280d0c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8318009870 tx=0x7f831802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2386164fe22dc937 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832cbcc640 1 -- 172.21.15.27:0/3756922518 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831802f840 con 0x7f83280b5c00 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832cbcc640 1 -- 172.21.15.27:0/3756922518 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f831802f9e0 con 0x7f83280b5c00 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832cbcc640 1 -- 172.21.15.27:0/3756922518 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831802fdb0 con 0x7f83280b5c00 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832fe32640 1 -- 172.21.15.27:0/3756922518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 msgr2=0x7f83280d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832fe32640 1 --2- 172.21.15.27:0/3756922518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 0x7f83280d0c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8318009870 tx=0x7f831802ba20 comp rx=0 tx=0).stop 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832fe32640 1 -- 172.21.15.27:0/3756922518 shutdown_connections 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832fe32640 1 --2- 172.21.15.27:0/3756922518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 0x7f83280d0c90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832fe32640 1 --2- 172.21.15.27:0/3756922518 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8328155950 0x7f83280cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832fe32640 1 -- 172.21.15.27:0/3756922518 >> 172.21.15.27:0/3756922518 conn(0x7f83280676c0 msgr2=0x7f8328067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832fe32640 1 -- 172.21.15.27:0/3756922518 shutdown_connections 2024-04-20T10:59:58.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.555+0000 7f832fe32640 1 -- 172.21.15.27:0/3756922518 wait complete. 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.556+0000 7f832fe32640 1 Processor -- start 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.556+0000 7f832fe32640 1 -- start start 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832fe32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 0x7f83281f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832fe32640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83280cc330 0x7f83280c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832fe32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8328155950 0x7f83280c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832fe32640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f832815fd10 con 0x7f83280b5c00 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832fe32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f832815fb90 con 0x7f8328155950 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832fe32640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8328155d30 con 0x7f83280cc330 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832dbce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 0x7f83281f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832dbce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 0x7f83281f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36292/0 (socket says 172.21.15.27:36292) 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832dbce640 1 -- 172.21.15.27:0/876923997 learned_addr learned my addr 172.21.15.27:0/876923997 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832d3cd640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83280cc330 0x7f83280c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.557+0000 7f832e3cf640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8328155950 0x7f83280c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:58.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f832dbce640 1 -- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83280cc330 msgr2=0x7f83280c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f832dbce640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83280cc330 0x7f83280c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f832dbce640 1 -- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8328155950 msgr2=0x7f83280c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f832dbce640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8328155950 0x7f83280c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f832dbce640 1 -- 172.21.15.27:0/876923997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83280c8ee0 con 0x7f83280b5c00 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f832dbce640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 0x7f83281f1db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f832000efc0 tx=0x7f832000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f83267fc640 1 -- 172.21.15.27:0/876923997 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320018020 con 0x7f83280b5c00 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f83267fc640 1 -- 172.21.15.27:0/876923997 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8320014480 con 0x7f83280b5c00 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83280c8260 con 0x7f83280b5c00 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f83267fc640 1 -- 172.21.15.27:0/876923997 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320014850 con 0x7f83280b5c00 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.558+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83280c8800 con 0x7f83280b5c00 2024-04-20T10:59:58.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.559+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f832804fcf0 con 0x7f83280b5c00 2024-04-20T10:59:58.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.559+0000 7f83267fc640 1 -- 172.21.15.27:0/876923997 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8320005d10 con 0x7f83280b5c00 2024-04-20T10:59:58.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.560+0000 7f83267fc640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f831406c8f0 0x7f831406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:59:58.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.560+0000 7f83267fc640 1 -- 172.21.15.27:0/876923997 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f832008c660 con 0x7f83280b5c00 2024-04-20T10:59:58.563 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.560+0000 7f832d3cd640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f831406c8f0 0x7f831406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:59:58.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.561+0000 7f832d3cd640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f831406c8f0 0x7f831406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8328152f00 tx=0x7f831c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:59:58.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.564+0000 7f83267fc640 1 -- 172.21.15.27:0/876923997 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8320022070 con 0x7f83280b5c00 2024-04-20T10:59:58.792 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.791+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83280cd0a0 con 0x7f83280b5c00 2024-04-20T10:59:58.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.792+0000 7f83267fc640 1 -- 172.21.15.27:0/876923997 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+6113 (secure 0 0 0) 0x7f8320010040 con 0x7f83280b5c00 2024-04-20T10:59:58.793 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T10:59:58.793 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:57.421458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4275,5029,4278,4293,4894]},"id":1}]} 2024-04-20T10:59:58.793 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-04-20T10:59:58.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f831406c8f0 msgr2=0x7f831406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:58.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f831406c8f0 0x7f831406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8328152f00 tx=0x7f831c009040 comp rx=0 tx=0).stop 2024-04-20T10:59:58.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 msgr2=0x7f83281f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:59:58.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 0x7f83281f1db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f832000efc0 tx=0x7f832000c490 comp rx=0 tx=0).stop 2024-04-20T10:59:58.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 shutdown_connections 2024-04-20T10:59:58.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f831406c8f0 0x7f831406edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:58.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8328155950 0x7f83280c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:58.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83280cc330 0x7f83280c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:58.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 --2- 172.21.15.27:0/876923997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83280b5c00 0x7f83281f1db0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:59:58.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 >> 172.21.15.27:0/876923997 conn(0x7f83280676c0 msgr2=0x7f832806d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:59:58.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 shutdown_connections 2024-04-20T10:59:58.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T10:59:58.794+0000 7f832fe32640 1 -- 172.21.15.27:0/876923997 wait complete. 2024-04-20T10:59:58.808 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=3, replacing gid=4272 2024-04-20T10:59:58.808 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.c 2024-04-20T11:00:01.809 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-04-20T11:00:01.809 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T11:00:01.809 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T11:00:01.812 INFO:tasks.ceph.mds.c:Started 2024-04-20T11:00:01.813 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:01.899 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-04-20T11:00:01.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.926+0000 7fc7ef5ce640 1 Processor -- start 2024-04-20T11:00:01.927 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.926+0000 7fc7ef5ce640 1 -- start start 2024-04-20T11:00:01.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7ef5ce640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e8155950 0x7fc7e80cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:01.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7ef5ce640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e815b9c0 con 0x7fc7e80b5c00 2024-04-20T11:00:01.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7ef5ce640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e815bca0 con 0x7fc7e80cc330 2024-04-20T11:00:01.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7ef5ce640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e815bf80 con 0x7fc7e8155950 2024-04-20T11:00:01.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7ed36a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e8155950 0x7fc7e80cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:01.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7edb6b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc7e80b5c00 0x7fc7e80d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:38758/0 (socket says 172.21.15.27:38758) 2024-04-20T11:00:01.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7edb6b640 1 -- 172.21.15.27:0/881205741 learned_addr learned my addr 172.21.15.27:0/881205741 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:01.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7ed36a640 1 -- 172.21.15.27:0/881205741 >> v1:172.21.15.137:6789/0 conn(0x7fc7e80cc330 legacy=0x7fc7e80b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:01.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7ed36a640 1 -- 172.21.15.27:0/881205741 >> v1:172.21.15.27:6789/0 conn(0x7fc7e80b5c00 legacy=0x7fc7e80d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:01.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7dbfff640 1 -- 172.21.15.27:0/881205741 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 99127748 0 0) 0x7fc7e815bca0 con 0x7fc7e80cc330 2024-04-20T11:00:01.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.927+0000 7fc7ed36a640 1 -- 172.21.15.27:0/881205741 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e815c260 con 0x7fc7e8155950 2024-04-20T11:00:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.928+0000 7fc7ed36a640 1 --2- 172.21.15.27:0/881205741 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e8155950 0x7fc7e80cbdf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40097d0 tx=0x7fc7e402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42386cae03edbed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.928+0000 7fc7dbfff640 1 -- 172.21.15.27:0/881205741 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4037070 con 0x7fc7e8155950 2024-04-20T11:00:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.928+0000 7fc7dbfff640 1 -- 172.21.15.27:0/881205741 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e4030710 con 0x7fc7e8155950 2024-04-20T11:00:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.928+0000 7fc7dbfff640 1 -- 172.21.15.27:0/881205741 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4030ae0 con 0x7fc7e8155950 2024-04-20T11:00:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.929+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/881205741 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e8155950 msgr2=0x7fc7e80cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.929+0000 7fc7ef5ce640 1 --2- 172.21.15.27:0/881205741 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e8155950 0x7fc7e80cbdf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40097d0 tx=0x7fc7e402b800 comp rx=0 tx=0).stop 2024-04-20T11:00:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.929+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/881205741 shutdown_connections 2024-04-20T11:00:01.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.929+0000 7fc7ef5ce640 1 --2- 172.21.15.27:0/881205741 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e8155950 0x7fc7e80cbdf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.929+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/881205741 >> 172.21.15.27:0/881205741 conn(0x7fc7e80676c0 msgr2=0x7fc7e8067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.929+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/881205741 shutdown_connections 2024-04-20T11:00:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.929+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/881205741 wait complete. 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.930+0000 7fc7ef5ce640 1 Processor -- start 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.930+0000 7fc7ef5ce640 1 -- start start 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ef5ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e80b5c00 0x7fc7e815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ef5ce640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e80cc330 0x7fc7e806d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ef5ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8155950 0x7fc7e81f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ef5ce640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e815fd10 con 0x7fc7e80b5c00 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ef5ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e815fb90 con 0x7fc7e8155950 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ef5ce640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e8155d30 con 0x7fc7e80cc330 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ed36a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e80b5c00 0x7fc7e815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ed36a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e80b5c00 0x7fc7e815b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36320/0 (socket says 172.21.15.27:36320) 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ed36a640 1 -- 172.21.15.27:0/103015014 learned_addr learned my addr 172.21.15.27:0/103015014 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:01.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7edb6b640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8155950 0x7fc7e81f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ecb69640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e80cc330 0x7fc7e806d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ecb69640 1 -- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8155950 msgr2=0x7fc7e81f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ecb69640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8155950 0x7fc7e81f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ecb69640 1 -- 172.21.15.27:0/103015014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e80b5c00 msgr2=0x7fc7e815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ecb69640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e80b5c00 0x7fc7e815b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ecb69640 1 -- 172.21.15.27:0/103015014 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e815caa0 con 0x7fc7e80cc330 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.931+0000 7fc7ed36a640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e80b5c00 0x7fc7e815b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.932+0000 7fc7ecb69640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e80cc330 0x7fc7e806d920 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00c450 tx=0x7fc7dc00c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.932+0000 7fc7d9ffb640 1 -- 172.21.15.27:0/103015014 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc017070 con 0x7fc7e80cc330 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.932+0000 7fc7d9ffb640 1 -- 172.21.15.27:0/103015014 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7dc0035f0 con 0x7fc7e80cc330 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.932+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e815cd80 con 0x7fc7e80cc330 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.932+0000 7fc7d9ffb640 1 -- 172.21.15.27:0/103015014 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc0039e0 con 0x7fc7e80cc330 2024-04-20T11:00:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.932+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e815ab10 con 0x7fc7e80cc330 2024-04-20T11:00:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.933+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e815af90 con 0x7fc7e80cc330 2024-04-20T11:00:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.933+0000 7fc7d9ffb640 1 -- 172.21.15.27:0/103015014 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7dc01a070 con 0x7fc7e80cc330 2024-04-20T11:00:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.934+0000 7fc7d9ffb640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc7cc06c920 0x7fc7cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.934+0000 7fc7d9ffb640 1 -- 172.21.15.27:0/103015014 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7fc7dc08afe0 con 0x7fc7e80cc330 2024-04-20T11:00:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.934+0000 7fc7ed36a640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc7cc06c920 0x7fc7cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:01.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.935+0000 7fc7ed36a640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc7cc06c920 0x7fc7cc06ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4000c00 tx=0x7fc7e4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:01.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:01.939+0000 7fc7d9ffb640 1 -- 172.21.15.27:0/103015014 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc7dc058c50 con 0x7fc7e80cc330 2024-04-20T11:00:02.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.163+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e80686d0 con 0x7fc7e80cc330 2024-04-20T11:00:02.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.164+0000 7fc7d9ffb640 1 -- 172.21.15.27:0/103015014 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+6113 (secure 0 0 0) 0x7fc7e80686d0 con 0x7fc7e80cc330 2024-04-20T11:00:02.165 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:02.165 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:57.421458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4275,5029,4278,4293,4894]},"id":1}]} 2024-04-20T11:00:02.165 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 34 2024-04-20T11:00:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc7cc06c920 msgr2=0x7fc7cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc7cc06c920 0x7fc7cc06ede0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4000c00 tx=0x7fc7e4008030 comp rx=0 tx=0).stop 2024-04-20T11:00:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e80cc330 msgr2=0x7fc7e806d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e80cc330 0x7fc7e806d920 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00c450 tx=0x7fc7dc00c910 comp rx=0 tx=0).stop 2024-04-20T11:00:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 shutdown_connections 2024-04-20T11:00:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc7cc06c920 0x7fc7cc06ede0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8155950 0x7fc7e81f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc7e80cc330 0x7fc7e806d920 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:02.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 --2- 172.21.15.27:0/103015014 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e80b5c00 0x7fc7e815b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:02.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 >> 172.21.15.27:0/103015014 conn(0x7fc7e80676c0 msgr2=0x7fc7e80b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:02.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 shutdown_connections 2024-04-20T11:00:02.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:02.166+0000 7fc7ef5ce640 1 -- 172.21.15.27:0/103015014 wait complete. 2024-04-20T11:00:02.178 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-04-20T11:00:04.179 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:04.280 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-20T11:00:04.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-20T11:00:04.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.280+0000 7f5c1a5e2640 1 Processor -- start 2024-04-20T11:00:04.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.281+0000 7f5c1a5e2640 1 -- start start 2024-04-20T11:00:04.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.281+0000 7f5c1a5e2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140b54f0 0x7f5c140b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:04.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.281+0000 7f5c1a5e2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c140d0c90 0x7f5c140d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:04.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.281+0000 7f5c1a5e2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c140b5e70 con 0x7f5c14155950 2024-04-20T11:00:04.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.281+0000 7f5c1a5e2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5c1415b9c0 con 0x7f5c140d0c90 2024-04-20T11:00:04.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.281+0000 7f5c1a5e2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1415bca0 con 0x7f5c140b54f0 2024-04-20T11:00:04.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.281+0000 7f5c18b7f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c140d0c90 0x7f5c140d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:04.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.282+0000 7f5c18b7f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c140d0c90 0x7f5c140d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36324/0 (socket says 172.21.15.27:36324) 2024-04-20T11:00:04.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.282+0000 7f5c137fe640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140b54f0 0x7f5c140b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:04.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.282+0000 7f5c18b7f640 1 -- 172.21.15.27:0/1575808683 learned_addr learned my addr 172.21.15.27:0/1575808683 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:04.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.282+0000 7f5c13fff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f5c14155950 0x7f5c140cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:47270/0 (socket says 172.21.15.27:47270) 2024-04-20T11:00:04.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.282+0000 7f5c18b7f640 1 -- 172.21.15.27:0/1575808683 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140b54f0 msgr2=0x7f5c140b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:04.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.282+0000 7f5c18b7f640 1 --2- 172.21.15.27:0/1575808683 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140b54f0 0x7f5c140b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.282+0000 7f5c18b7f640 1 -- 172.21.15.27:0/1575808683 >> v1:172.21.15.137:6789/0 conn(0x7f5c14155950 legacy=0x7f5c140cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:04.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.282+0000 7f5c18b7f640 1 -- 172.21.15.27:0/1575808683 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1415bf80 con 0x7f5c140d0c90 2024-04-20T11:00:04.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c18b7f640 1 --2- 172.21.15.27:0/1575808683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c140d0c90 0x7f5c140d1100 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009e50 tx=0x7f5c0c02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476418695b6e192d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:04.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c12ffd640 1 -- 172.21.15.27:0/1575808683 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c02cb90 con 0x7f5c140d0c90 2024-04-20T11:00:04.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c12ffd640 1 -- 172.21.15.27:0/1575808683 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c02cd30 con 0x7f5c140d0c90 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c12ffd640 1 -- 172.21.15.27:0/1575808683 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c034610 con 0x7f5c140d0c90 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/1575808683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c140d0c90 msgr2=0x7f5c140d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c1a5e2640 1 --2- 172.21.15.27:0/1575808683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c140d0c90 0x7f5c140d1100 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009e50 tx=0x7f5c0c02bff0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/1575808683 shutdown_connections 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c1a5e2640 1 --2- 172.21.15.27:0/1575808683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c140d0c90 0x7f5c140d1100 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c1a5e2640 1 --2- 172.21.15.27:0/1575808683 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140b54f0 0x7f5c140b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/1575808683 >> 172.21.15.27:0/1575808683 conn(0x7f5c140676c0 msgr2=0x7f5c14067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/1575808683 shutdown_connections 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.283+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/1575808683 wait complete. 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.285+0000 7f5c1a5e2640 1 Processor -- start 2024-04-20T11:00:04.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.285+0000 7f5c1a5e2640 1 -- start start 2024-04-20T11:00:04.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.285+0000 7f5c1a5e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c140b54f0 0x7f5c14153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:04.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.285+0000 7f5c1a5e2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140d0c90 0x7f5c1414c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:04.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.285+0000 7f5c1a5e2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c14155950 0x7f5c1414cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:04.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.285+0000 7f5c1a5e2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c140d1640 con 0x7f5c14155950 2024-04-20T11:00:04.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.285+0000 7f5c1a5e2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1415c540 con 0x7f5c140b54f0 2024-04-20T11:00:04.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.285+0000 7f5c1a5e2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5c1415fb90 con 0x7f5c140d0c90 2024-04-20T11:00:04.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c18b7f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c14155950 0x7f5c1414cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:04.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c18b7f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c14155950 0x7f5c1414cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36328/0 (socket says 172.21.15.27:36328) 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c18b7f640 1 -- 172.21.15.27:0/2319052047 learned_addr learned my addr 172.21.15.27:0/2319052047 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c137fe640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140d0c90 0x7f5c1414c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c13fff640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c140b54f0 0x7f5c14153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c18b7f640 1 -- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140d0c90 msgr2=0x7f5c1414c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c18b7f640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140d0c90 0x7f5c1414c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c18b7f640 1 -- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c140b54f0 msgr2=0x7f5c14153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c18b7f640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c140b54f0 0x7f5c14153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c18b7f640 1 -- 172.21.15.27:0/2319052047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c14159e20 con 0x7f5c14155950 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c18b7f640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c14155950 0x7f5c1414cb20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009e50 tx=0x7f5c0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c10ff9640 1 -- 172.21.15.27:0/2319052047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c043020 con 0x7f5c14155950 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c10ff9640 1 -- 172.21.15.27:0/2319052047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c03e450 con 0x7f5c14155950 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c10ff9640 1 -- 172.21.15.27:0/2319052047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c009510 con 0x7f5c14155950 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c141f15e0 con 0x7f5c14155950 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.286+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c141f1ba0 con 0x7f5c14155950 2024-04-20T11:00:04.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.287+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1404fcf0 con 0x7f5c14155950 2024-04-20T11:00:04.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.288+0000 7f5c10ff9640 1 -- 172.21.15.27:0/2319052047 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c0c046020 con 0x7f5c14155950 2024-04-20T11:00:04.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.288+0000 7f5c10ff9640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5c0006c8f0 0x7f5c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:04.290 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.289+0000 7f5c10ff9640 1 -- 172.21.15.27:0/2319052047 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f5c0c0ae1e0 con 0x7f5c14155950 2024-04-20T11:00:04.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.289+0000 7f5c13fff640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5c0006c8f0 0x7f5c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:04.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.289+0000 7f5c13fff640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5c0006c8f0 0x7f5c0006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c04007fd0 tx=0x7f5c04007e80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:04.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.293+0000 7f5c10ff9640 1 -- 172.21.15.27:0/2319052047 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5c0c04d030 con 0x7f5c14155950 2024-04-20T11:00:04.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.519+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c141591d0 con 0x7f5c14155950 2024-04-20T11:00:04.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.520+0000 7f5c10ff9640 1 -- 172.21.15.27:0/2319052047 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6937 (secure 0 0 0) 0x7f5c0c07bdb0 con 0x7f5c14155950 2024-04-20T11:00:04.521 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:04.521 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:57.421458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,5029,4275]},"id":1}]} 2024-04-20T11:00:04.521 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 35 2024-04-20T11:00:04.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5c0006c8f0 msgr2=0x7f5c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:04.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5c0006c8f0 0x7f5c0006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c04007fd0 tx=0x7f5c04007e80 comp rx=0 tx=0).stop 2024-04-20T11:00:04.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c14155950 msgr2=0x7f5c1414cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:04.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c14155950 0x7f5c1414cb20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009e50 tx=0x7f5c0c004a40 comp rx=0 tx=0).stop 2024-04-20T11:00:04.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 shutdown_connections 2024-04-20T11:00:04.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5c0006c8f0 0x7f5c0006edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c14155950 0x7f5c1414cb20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5c140d0c90 0x7f5c1414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 --2- 172.21.15.27:0/2319052047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c140b54f0 0x7f5c14153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:04.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 >> 172.21.15.27:0/2319052047 conn(0x7f5c140676c0 msgr2=0x7f5c14067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:04.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 shutdown_connections 2024-04-20T11:00:04.524 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:04.522+0000 7f5c1a5e2640 1 -- 172.21.15.27:0/2319052047 wait complete. 2024-04-20T11:00:04.535 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-04-20T11:00:04.536 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 26 secs before thrashing 2024-04-20T11:00:12.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:00:12.513 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:00:30.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:30.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.650+0000 7f38b5648640 1 Processor -- start 2024-04-20T11:00:30.651 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.650+0000 7f38b5648640 1 -- start start 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.651+0000 7f38b5648640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f38b0155950 0x7f38b00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.651+0000 7f38b5648640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00cc330 0x7f38b00b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.651+0000 7f38b5648640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38b00b5ea0 con 0x7f38b00b5a30 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.651+0000 7f38b5648640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f38b015b9c0 con 0x7f38b0155950 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.651+0000 7f38b5648640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f38b015bca0 con 0x7f38b00cc330 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.651+0000 7f38af7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f38b00b5a30 0x7f38b00d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52144/0 (socket says 172.21.15.27:52144) 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.651+0000 7f38af7fe640 1 -- 172.21.15.27:0/2370558782 learned_addr learned my addr 172.21.15.27:0/2370558782 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38a67fc640 1 --2- 172.21.15.27:0/2370558782 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00cc330 0x7f38b00b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38aeffd640 1 --2- 172.21.15.27:0/2370558782 >> v2:172.21.15.137:3300/0 conn(0x7f38b0155950 0x7f38b00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38ae7fc640 1 -- 172.21.15.27:0/2370558782 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3695610562 0 0) 0x7f38b00b5ea0 con 0x7f38b00b5a30 2024-04-20T11:00:30.653 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38ae7fc640 1 -- 172.21.15.27:0/2370558782 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f389c003520 con 0x7f38b00b5a30 2024-04-20T11:00:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38aeffd640 1 -- 172.21.15.27:0/2370558782 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00cc330 msgr2=0x7f38b00b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38aeffd640 1 --2- 172.21.15.27:0/2370558782 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00cc330 0x7f38b00b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38ae7fc640 1 -- 172.21.15.27:0/2370558782 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1996092141 0 0) 0x7f389c003520 con 0x7f38b00b5a30 2024-04-20T11:00:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38aeffd640 1 -- 172.21.15.27:0/2370558782 >> v1:172.21.15.27:6789/0 conn(0x7f38b00b5a30 legacy=0x7f38b00d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38aeffd640 1 -- 172.21.15.27:0/2370558782 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b015bf80 con 0x7f38b0155950 2024-04-20T11:00:30.654 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.652+0000 7f38a67fc640 1 --2- 172.21.15.27:0/2370558782 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00cc330 0x7f38b00b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:00:30.655 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.653+0000 7f38aeffd640 1 --2- 172.21.15.27:0/2370558782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38b0155950 0x7f38b00cbdf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38a00097a0 tx=0x7f38a002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146dae7e87ff82ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.653+0000 7f38ae7fc640 1 -- 172.21.15.27:0/2370558782 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0038070 con 0x7f38b0155950 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.653+0000 7f38ae7fc640 1 -- 172.21.15.27:0/2370558782 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a0031500 con 0x7f38b0155950 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.653+0000 7f38ae7fc640 1 -- 172.21.15.27:0/2370558782 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a80019f0 con 0x7f38b0155950 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.654+0000 7f38b5648640 1 -- 172.21.15.27:0/2370558782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38b0155950 msgr2=0x7f38b00cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.654+0000 7f38b5648640 1 --2- 172.21.15.27:0/2370558782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38b0155950 0x7f38b00cbdf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f38a00097a0 tx=0x7f38a002b820 comp rx=0 tx=0).stop 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.654+0000 7f38b5648640 1 -- 172.21.15.27:0/2370558782 shutdown_connections 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.654+0000 7f38b5648640 1 --2- 172.21.15.27:0/2370558782 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00cc330 0x7f38b00b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.654+0000 7f38b5648640 1 --2- 172.21.15.27:0/2370558782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38b0155950 0x7f38b00cbdf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.654+0000 7f38b5648640 1 -- 172.21.15.27:0/2370558782 >> 172.21.15.27:0/2370558782 conn(0x7f38b00676c0 msgr2=0x7f38b0067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.654+0000 7f38b5648640 1 -- 172.21.15.27:0/2370558782 shutdown_connections 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.654+0000 7f38b5648640 1 -- 172.21.15.27:0/2370558782 wait complete. 2024-04-20T11:00:30.656 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.655+0000 7f38b5648640 1 Processor -- start 2024-04-20T11:00:30.657 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.655+0000 7f38b5648640 1 -- start start 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38b5648640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00b5a30 0x7f38b0159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38b5648640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38b00cc330 0x7f38b01e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38b5648640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38b0155950 0x7f38b01e6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38b5648640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f38b015fb90 con 0x7f38b0155950 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38b5648640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f38b015c540 con 0x7f38b00cc330 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38b5648640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f38b015fd10 con 0x7f38b00b5a30 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38b0155950 0x7f38b01e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38b0155950 0x7f38b01e6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60386/0 (socket says 172.21.15.27:60386) 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38af7fe640 1 -- 172.21.15.27:0/4174100460 learned_addr learned my addr 172.21.15.27:0/4174100460 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38a67fc640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38b00cc330 0x7f38b01e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38aeffd640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00b5a30 0x7f38b0159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38af7fe640 1 -- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00b5a30 msgr2=0x7f38b0159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38af7fe640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00b5a30 0x7f38b0159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:30.659 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38af7fe640 1 -- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38b00cc330 msgr2=0x7f38b01e5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38af7fe640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38b00cc330 0x7f38b01e5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38af7fe640 1 -- 172.21.15.27:0/4174100460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b01f1800 con 0x7f38b0155950 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.656+0000 7f38af7fe640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38b0155950 0x7f38b01e6240 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f38a800cc10 tx=0x7f38a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.657+0000 7f38a7fff640 1 -- 172.21.15.27:0/4174100460 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a800ad40 con 0x7f38b0155950 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.657+0000 7f38a7fff640 1 -- 172.21.15.27:0/4174100460 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a8012400 con 0x7f38b0155950 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.657+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b01f1ae0 con 0x7f38b0155950 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.657+0000 7f38a7fff640 1 -- 172.21.15.27:0/4174100460 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a80127f0 con 0x7f38b0155950 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.657+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b01f20a0 con 0x7f38b0155950 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.658+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b00686d0 con 0x7f38b0155950 2024-04-20T11:00:30.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.658+0000 7f38a7fff640 1 -- 172.21.15.27:0/4174100460 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38a8012990 con 0x7f38b0155950 2024-04-20T11:00:30.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.659+0000 7f38a7fff640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f389006c8f0 0x7f389006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:30.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.659+0000 7f38a7fff640 1 -- 172.21.15.27:0/4174100460 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f38a808c290 con 0x7f38b0155950 2024-04-20T11:00:30.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.659+0000 7f38aeffd640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f389006c8f0 0x7f389006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:30.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.659+0000 7f38aeffd640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f389006c8f0 0x7f389006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f38a0035000 tx=0x7f38a0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:30.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.663+0000 7f38a7fff640 1 -- 172.21.15.27:0/4174100460 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f38b00686d0 con 0x7f38b0155950 2024-04-20T11:00:30.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.890+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38b004fcf0 con 0x7f38b0155950 2024-04-20T11:00:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.891+0000 7f38a7fff640 1 -- 172.21.15.27:0/4174100460 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6937 (secure 0 0 0) 0x7f38a8059f80 con 0x7f38b0155950 2024-04-20T11:00:30.892 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:30.892 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:57.421458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,5029,4275]},"id":1}]} 2024-04-20T11:00:30.892 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 35 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f389006c8f0 msgr2=0x7f389006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f389006c8f0 0x7f389006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f38a0035000 tx=0x7f38a0008030 comp rx=0 tx=0).stop 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38b0155950 msgr2=0x7f38b01e6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38b0155950 0x7f38b01e6240 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f38a800cc10 tx=0x7f38a800a430 comp rx=0 tx=0).stop 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 shutdown_connections 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f389006c8f0 0x7f389006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38b0155950 0x7f38b01e6240 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38b00cc330 0x7f38b01e5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 --2- 172.21.15.27:0/4174100460 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f38b00b5a30 0x7f38b0159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 >> 172.21.15.27:0/4174100460 conn(0x7f38b00676c0 msgr2=0x7f38b00cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 shutdown_connections 2024-04-20T11:00:30.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:30.893+0000 7f38b5648640 1 -- 172.21.15.27:0/4174100460 wait complete. 2024-04-20T11:00:30.905 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=2) 2024-04-20T11:00:30.905 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-20T11:00:30.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:00:30.915 INFO:tasks.ceph.mds.e:Stopped 2024-04-20T11:00:30.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:31.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.056+0000 7febf9ceb640 1 Processor -- start 2024-04-20T11:00:31.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.056+0000 7febf9ceb640 1 -- start start 2024-04-20T11:00:31.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.056+0000 7febf9ceb640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7febf4072130 0x7febf4073970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:31.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.056+0000 7febf9ceb640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf415f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:31.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.056+0000 7febf9ceb640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf4073fc0 con 0x7febf415ed10 2024-04-20T11:00:31.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.056+0000 7febf9ceb640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7febf40c3bc0 con 0x7febf4072130 2024-04-20T11:00:31.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.056+0000 7febf9ceb640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7febf40c4050 con 0x7febf415f0f0 2024-04-20T11:00:31.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.057+0000 7febf3fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf415f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:31.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.057+0000 7febf2ffd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7febf4072130 0x7febf4073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:31.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.057+0000 7febf3fff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf415f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:59816/0 (socket says 172.21.15.27:59816) 2024-04-20T11:00:31.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.057+0000 7febf3fff640 1 -- 172.21.15.27:0/203707394 learned_addr learned my addr 172.21.15.27:0/203707394 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.057+0000 7febf3fff640 1 -- 172.21.15.27:0/203707394 >> v2:172.21.15.137:3300/0 conn(0x7febf4072130 msgr2=0x7febf4073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.057+0000 7febf3fff640 1 --2- 172.21.15.27:0/203707394 >> v2:172.21.15.137:3300/0 conn(0x7febf4072130 0x7febf4073970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.057+0000 7febf3fff640 1 -- 172.21.15.27:0/203707394 >> v1:172.21.15.137:6789/0 conn(0x7febf415ed10 legacy=0x7febf40c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:31.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.057+0000 7febf3fff640 1 -- 172.21.15.27:0/203707394 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf40c2fb0 con 0x7febf415f0f0 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.057+0000 7febf2ffd640 1 --2- 172.21.15.27:0/203707394 >> v2:172.21.15.137:3300/0 conn(0x7febf4072130 0x7febf4073970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.058+0000 7febf3fff640 1 --2- 172.21.15.27:0/203707394 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf415f520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7febec009e50 tx=0x7febec02bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc77d0bcca76e2bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.058+0000 7febf27fc640 1 -- 172.21.15.27:0/203707394 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec02caf0 con 0x7febf415f0f0 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.058+0000 7febf27fc640 1 -- 172.21.15.27:0/203707394 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec02cc90 con 0x7febf415f0f0 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.058+0000 7febf27fc640 1 -- 172.21.15.27:0/203707394 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec034610 con 0x7febf415f0f0 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.059+0000 7febf9ceb640 1 -- 172.21.15.27:0/203707394 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 msgr2=0x7febf415f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.059+0000 7febf9ceb640 1 --2- 172.21.15.27:0/203707394 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf415f520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7febec009e50 tx=0x7febec02bf50 comp rx=0 tx=0).stop 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.059+0000 7febf9ceb640 1 -- 172.21.15.27:0/203707394 shutdown_connections 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.059+0000 7febf9ceb640 1 --2- 172.21.15.27:0/203707394 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf415f520 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.059+0000 7febf9ceb640 1 --2- 172.21.15.27:0/203707394 >> v2:172.21.15.137:3300/0 conn(0x7febf4072130 0x7febf4073970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.059+0000 7febf9ceb640 1 -- 172.21.15.27:0/203707394 >> 172.21.15.27:0/203707394 conn(0x7febf40ca8f0 msgr2=0x7febf40cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.059+0000 7febf9ceb640 1 -- 172.21.15.27:0/203707394 shutdown_connections 2024-04-20T11:00:31.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.059+0000 7febf9ceb640 1 -- 172.21.15.27:0/203707394 wait complete. 2024-04-20T11:00:31.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.060+0000 7febf9ceb640 1 Processor -- start 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.060+0000 7febf9ceb640 1 -- start start 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf9ceb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf4072130 0x7febf4157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf9ceb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febf415ed10 0x7febf41f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf9ceb640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf4157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf9ceb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7febf40c3850 con 0x7febf415ed10 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf9ceb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febf40c3570 con 0x7febf4072130 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf9ceb640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7febf40c8f00 con 0x7febf415f0f0 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febf415ed10 0x7febf41f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febf415ed10 0x7febf41f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60402/0 (socket says 172.21.15.27:60402) 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf2ffd640 1 -- 172.21.15.27:0/593634666 learned_addr learned my addr 172.21.15.27:0/593634666 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf37fe640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf4072130 0x7febf4157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf3fff640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf4157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf2ffd640 1 -- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 msgr2=0x7febf4157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf2ffd640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf4157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf2ffd640 1 -- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf4072130 msgr2=0x7febf4157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf2ffd640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf4072130 0x7febf4157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf2ffd640 1 -- 172.21.15.27:0/593634666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf4155e00 con 0x7febf415ed10 2024-04-20T11:00:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf3fff640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf4157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:00:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febf2ffd640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febf415ed10 0x7febf41f2750 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7febe800ec10 tx=0x7febe800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.061+0000 7febdbfff640 1 -- 172.21.15.27:0/593634666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8017070 con 0x7febf415ed10 2024-04-20T11:00:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.062+0000 7febdbfff640 1 -- 172.21.15.27:0/593634666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe800cc30 con 0x7febf415ed10 2024-04-20T11:00:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.062+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf41560a0 con 0x7febf415ed10 2024-04-20T11:00:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.062+0000 7febdbfff640 1 -- 172.21.15.27:0/593634666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8013470 con 0x7febf415ed10 2024-04-20T11:00:31.064 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.062+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf4156660 con 0x7febf415ed10 2024-04-20T11:00:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.063+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf404fcf0 con 0x7febf415ed10 2024-04-20T11:00:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.063+0000 7febdbfff640 1 -- 172.21.15.27:0/593634666 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febe801a070 con 0x7febf415ed10 2024-04-20T11:00:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.064+0000 7febdbfff640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febe006c920 0x7febe006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.064+0000 7febdbfff640 1 -- 172.21.15.27:0/593634666 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7febe808bb30 con 0x7febf415ed10 2024-04-20T11:00:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.064+0000 7febf37fe640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febe006c920 0x7febe006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:31.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.064+0000 7febf37fe640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febe006c920 0x7febe006ede0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7febe4001780 tx=0x7febe4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:31.069 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.068+0000 7febdbfff640 1 -- 172.21.15.27:0/593634666 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7febe8059790 con 0x7febf415ed10 2024-04-20T11:00:31.347 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.347+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febf4072b60 con 0x7febf415ed10 2024-04-20T11:00:31.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.347+0000 7febdbfff640 1 -- 172.21.15.27:0/593634666 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6937 (secure 0 0 0) 0x7febe805da90 con 0x7febf415ed10 2024-04-20T11:00:31.348 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:31.348 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:57.421458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,5029,4275]},"id":1}]} 2024-04-20T11:00:31.349 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 35 2024-04-20T11:00:31.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febe006c920 msgr2=0x7febe006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:31.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febe006c920 0x7febe006ede0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7febe4001780 tx=0x7febe4009040 comp rx=0 tx=0).stop 2024-04-20T11:00:31.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febf415ed10 msgr2=0x7febf41f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:31.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febf415ed10 0x7febf41f2750 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7febe800ec10 tx=0x7febe800c520 comp rx=0 tx=0).stop 2024-04-20T11:00:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 shutdown_connections 2024-04-20T11:00:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febe006c920 0x7febe006ede0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7febf415f0f0 0x7febf4157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febf415ed10 0x7febf41f2750 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 --2- 172.21.15.27:0/593634666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf4072130 0x7febf4157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 >> 172.21.15.27:0/593634666 conn(0x7febf40ca8f0 msgr2=0x7febf40cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 shutdown_connections 2024-04-20T11:00:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:31.349+0000 7febf9ceb640 1 -- 172.21.15.27:0/593634666 wait complete. 2024-04-20T11:00:31.361 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-04-20T11:00:33.362 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.465+0000 7f33dd94a640 1 Processor -- start 2024-04-20T11:00:33.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.465+0000 7f33dd94a640 1 -- start start 2024-04-20T11:00:33.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33dd94a640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f33d815ed10 0x7f33d8073970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33dd94a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d815f0f0 0x7f33d815f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33dd94a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33d8073fc0 con 0x7f33d8072130 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33dd94a640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f33d80c3bc0 con 0x7f33d815ed10 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33dd94a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f33d80c4050 con 0x7f33d815f0f0 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33d67fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f33d8072130 0x7f33d80c87f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:52158/0 (socket says 172.21.15.27:52158) 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33d67fc640 1 -- 172.21.15.27:0/1951322570 learned_addr learned my addr 172.21.15.27:0/1951322570 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33d6ffd640 1 --2- 172.21.15.27:0/1951322570 >> v2:172.21.15.137:3300/0 conn(0x7f33d815ed10 0x7f33d8073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33d77fe640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d815f0f0 0x7f33d815f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2515500758 0 0) 0x7f33d8073fc0 con 0x7f33d8072130 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.466+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33c4003520 con 0x7f33d8072130 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3995051148 0 0) 0x7f33c4003520 con 0x7f33d8072130 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d815f0f0 msgr2=0x7f33d815f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:33.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d815f0f0 0x7f33d815f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 >> v2:172.21.15.137:3300/0 conn(0x7f33d815ed10 msgr2=0x7f33d8073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 --2- 172.21.15.27:0/1951322570 >> v2:172.21.15.137:3300/0 conn(0x7f33d815ed10 0x7f33d8073970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d80c2fb0 con 0x7f33d8072130 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f33c0001670 con 0x7f33d8072130 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 >> v1:172.21.15.27:6789/0 conn(0x7f33d8072130 legacy=0x7f33d80c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d815f0f0 0x7f33c40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c4003f30 0x7f33c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c4004d40 0x7f33c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f33d80c3bc0 con 0x7f33c4004d40 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33d80c4050 con 0x7f33c4003f30 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f33c400b9c0 con 0x7f33d815f0f0 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d77fe640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c4004d40 0x7f33c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d6ffd640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d815f0f0 0x7f33c40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d67fc640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c4003f30 0x7f33c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d77fe640 1 -- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d815f0f0 msgr2=0x7f33c40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:33.471 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d77fe640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d815f0f0 0x7f33c40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d77fe640 1 -- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c4003f30 msgr2=0x7f33c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d77fe640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c4003f30 0x7f33c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.467+0000 7f33d77fe640 1 -- 172.21.15.27:0/1951322570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c4003520 con 0x7f33c4004d40 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.468+0000 7f33d77fe640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c4004d40 0x7f33c40051c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009fc0 tx=0x7f33c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1241c96871150616 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.468+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8039070 con 0x7f33c4004d40 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.468+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33c802cc00 con 0x7f33c4004d40 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.468+0000 7f33d5ffb640 1 -- 172.21.15.27:0/1951322570 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8009c10 con 0x7f33c4004d40 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.469+0000 7f33dd94a640 1 -- 172.21.15.27:0/1951322570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c4004d40 msgr2=0x7f33c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.469+0000 7f33dd94a640 1 --2- 172.21.15.27:0/1951322570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c4004d40 0x7f33c40051c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009fc0 tx=0x7f33c4005830 comp rx=0 tx=0).stop 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.469+0000 7f33d77fe640 1 -- 172.21.15.27:0/1951322570 reap_dead start 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.469+0000 7f33dd94a640 1 -- 172.21.15.27:0/1951322570 shutdown_connections 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.469+0000 7f33dd94a640 1 -- 172.21.15.27:0/1951322570 >> 172.21.15.27:0/1951322570 conn(0x7f33d80ca8f0 msgr2=0x7f33d80cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.469+0000 7f33dd94a640 1 -- 172.21.15.27:0/1951322570 shutdown_connections 2024-04-20T11:00:33.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.469+0000 7f33dd94a640 1 -- 172.21.15.27:0/1951322570 wait complete. 2024-04-20T11:00:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.470+0000 7f33dd94a640 1 Processor -- start 2024-04-20T11:00:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.470+0000 7f33dd94a640 1 -- start start 2024-04-20T11:00:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33dd94a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d81463d0 0x7f33d815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33dd94a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33d81467b0 0x7f33d806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:33.475 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33dd94a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d806d300 0x7f33d81f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33dd94a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f33d80c3850 con 0x7f33d81463d0 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33dd94a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33d80c3570 con 0x7f33d81467b0 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33dd94a640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f33d80c8f00 con 0x7f33d806d300 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d81463d0 0x7f33d815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d81463d0 0x7f33d815a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60422/0 (socket says 172.21.15.27:60422) 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d6ffd640 1 -- 172.21.15.27:0/4079428092 learned_addr learned my addr 172.21.15.27:0/4079428092 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d67fc640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33d81467b0 0x7f33d806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d77fe640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d806d300 0x7f33d81f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d6ffd640 1 -- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d806d300 msgr2=0x7f33d81f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:33.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d6ffd640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d806d300 0x7f33d81f1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d6ffd640 1 -- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33d81467b0 msgr2=0x7f33d806cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d6ffd640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33d81467b0 0x7f33d806cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d6ffd640 1 -- 172.21.15.27:0/4079428092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d81f5010 con 0x7f33d81463d0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33d6ffd640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d81463d0 0x7f33d815a300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f33d0011a70 tx=0x7f33d0011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33cffff640 1 -- 172.21.15.27:0/4079428092 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d000fa90 con 0x7f33d81463d0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33cffff640 1 -- 172.21.15.27:0/4079428092 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d000fc30 con 0x7f33d81463d0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d81f52f0 con 0x7f33d81463d0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33cffff640 1 -- 172.21.15.27:0/4079428092 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0014440 con 0x7f33d81463d0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.471+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d81f58b0 con 0x7f33d81463d0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.473+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d8154a70 con 0x7f33d81463d0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.473+0000 7f33cffff640 1 -- 172.21.15.27:0/4079428092 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33d0012040 con 0x7f33d81463d0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.473+0000 7f33cffff640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f33b806c8f0 0x7f33b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.473+0000 7f33cffff640 1 -- 172.21.15.27:0/4079428092 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f33d008c2f0 con 0x7f33d81463d0 2024-04-20T11:00:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.473+0000 7f33d67fc640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f33b806c8f0 0x7f33b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.474+0000 7f33d67fc640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f33b806c8f0 0x7f33b806edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f33d8146bb0 tx=0x7f33c00078f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.478+0000 7f33cffff640 1 -- 172.21.15.27:0/4079428092 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f33d8154a70 con 0x7f33d81463d0 2024-04-20T11:00:33.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.704+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33d804fcf0 con 0x7f33d81463d0 2024-04-20T11:00:33.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.705+0000 7f33cffff640 1 -- 172.21.15.27:0/4079428092 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6937 (secure 0 0 0) 0x7f33d0019030 con 0x7f33d81463d0 2024-04-20T11:00:33.706 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:33.706 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:57.421458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,5029,4275]},"id":1}]} 2024-04-20T11:00:33.707 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 35 2024-04-20T11:00:33.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f33b806c8f0 msgr2=0x7f33b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f33b806c8f0 0x7f33b806edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f33d8146bb0 tx=0x7f33c00078f0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d81463d0 msgr2=0x7f33d815a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d81463d0 0x7f33d815a300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f33d0011a70 tx=0x7f33d0011e70 comp rx=0 tx=0).stop 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 shutdown_connections 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f33b806c8f0 0x7f33b806edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f33d806d300 0x7f33d81f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33d81467b0 0x7f33d806cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 --2- 172.21.15.27:0/4079428092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33d81463d0 0x7f33d815a300 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.707+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 >> 172.21.15.27:0/4079428092 conn(0x7f33d80ca8f0 msgr2=0x7f33d81550c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.708+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 shutdown_connections 2024-04-20T11:00:33.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:33.708+0000 7f33dd94a640 1 -- 172.21.15.27:0/4079428092 wait complete. 2024-04-20T11:00:33.719 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-04-20T11:00:35.720 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:35.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.843+0000 7f8cab538640 1 Processor -- start 2024-04-20T11:00:35.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.843+0000 7f8cab538640 1 -- start start 2024-04-20T11:00:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8cab538640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 0x7f8ca40b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8cab538640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca40b5e70 con 0x7f8ca40d0c90 2024-04-20T11:00:35.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8cab538640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca415b9c0 con 0x7f8ca4155950 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8cab538640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca415bca0 con 0x7f8ca40b54f0 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8ca9ad5640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8ca40d0c90 0x7f8ca40d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56686/0 (socket says 172.21.15.27:56686) 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8ca9ad5640 1 -- 172.21.15.27:0/926967487 learned_addr learned my addr 172.21.15.27:0/926967487 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8ca8ad3640 1 --2- 172.21.15.27:0/926967487 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 0x7f8ca40b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8c9bfff640 1 -- 172.21.15.27:0/926967487 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559128592 0 0) 0x7f8ca40b5e70 con 0x7f8ca40d0c90 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8c9bfff640 1 -- 172.21.15.27:0/926967487 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c90003520 con 0x7f8ca40d0c90 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8c9bfff640 1 -- 172.21.15.27:0/926967487 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1748775944 0 0) 0x7f8ca415b9c0 con 0x7f8ca4155950 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8c9bfff640 1 -- 172.21.15.27:0/926967487 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ca40b5e70 con 0x7f8ca4155950 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8ca8ad3640 1 -- 172.21.15.27:0/926967487 >> v1:172.21.15.137:6789/0 conn(0x7f8ca4155950 legacy=0x7f8ca40cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8ca8ad3640 1 -- 172.21.15.27:0/926967487 >> v1:172.21.15.27:6789/0 conn(0x7f8ca40d0c90 legacy=0x7f8ca40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8c9bfff640 1 -- 172.21.15.27:0/926967487 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 552307054 0 0) 0x7f8c90003520 con 0x7f8ca40d0c90 2024-04-20T11:00:35.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.844+0000 7f8ca8ad3640 1 -- 172.21.15.27:0/926967487 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca415bf80 con 0x7f8ca40b54f0 2024-04-20T11:00:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.845+0000 7f8ca8ad3640 1 --2- 172.21.15.27:0/926967487 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 0x7f8ca40b5930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c940097d0 tx=0x7f8c9402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8ea23ee2413e740 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.845+0000 7f8c9bfff640 1 -- 172.21.15.27:0/926967487 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0019f0 con 0x7f8ca40b54f0 2024-04-20T11:00:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.845+0000 7f8c9bfff640 1 -- 172.21.15.27:0/926967487 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c94037070 con 0x7f8ca40b54f0 2024-04-20T11:00:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.845+0000 7f8c9bfff640 1 -- 172.21.15.27:0/926967487 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94031780 con 0x7f8ca40b54f0 2024-04-20T11:00:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.846+0000 7f8cab538640 1 -- 172.21.15.27:0/926967487 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 msgr2=0x7f8ca40b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.846+0000 7f8cab538640 1 --2- 172.21.15.27:0/926967487 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 0x7f8ca40b5930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c940097d0 tx=0x7f8c9402b8a0 comp rx=0 tx=0).stop 2024-04-20T11:00:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.846+0000 7f8cab538640 1 -- 172.21.15.27:0/926967487 shutdown_connections 2024-04-20T11:00:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.846+0000 7f8cab538640 1 --2- 172.21.15.27:0/926967487 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 0x7f8ca40b5930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:35.848 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.846+0000 7f8cab538640 1 -- 172.21.15.27:0/926967487 >> 172.21.15.27:0/926967487 conn(0x7f8ca40676c0 msgr2=0x7f8ca4067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:35.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.846+0000 7f8cab538640 1 -- 172.21.15.27:0/926967487 shutdown_connections 2024-04-20T11:00:35.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.846+0000 7f8cab538640 1 -- 172.21.15.27:0/926967487 wait complete. 2024-04-20T11:00:35.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.847+0000 7f8cab538640 1 Processor -- start 2024-04-20T11:00:35.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.847+0000 7f8cab538640 1 -- start start 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8cab538640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 0x7f8ca41f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8cab538640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca40d0c90 0x7f8ca41f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8cab538640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ca4155950 0x7f8ca40c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8cab538640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca415fb90 con 0x7f8ca4155950 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8cab538640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca415c540 con 0x7f8ca40d0c90 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8cab538640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca415fd10 con 0x7f8ca40b54f0 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca9ad5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ca4155950 0x7f8ca40c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca9ad5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ca4155950 0x7f8ca40c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55026/0 (socket says 172.21.15.27:55026) 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca9ad5640 1 -- 172.21.15.27:0/4034063880 learned_addr learned my addr 172.21.15.27:0/4034063880 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca92d4640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 0x7f8ca41f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca8ad3640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca40d0c90 0x7f8ca41f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca9ad5640 1 -- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 msgr2=0x7f8ca41f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca9ad5640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 0x7f8ca41f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca9ad5640 1 -- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca40d0c90 msgr2=0x7f8ca41f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca9ad5640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca40d0c90 0x7f8ca41f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.848+0000 7f8ca9ad5640 1 -- 172.21.15.27:0/4034063880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca41f1fb0 con 0x7f8ca4155950 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.849+0000 7f8ca9ad5640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ca4155950 0x7f8ca40c6f90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00cc10 tx=0x7f8c9c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.849+0000 7f8c99ffb640 1 -- 172.21.15.27:0/4034063880 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00ce20 con 0x7f8ca4155950 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.849+0000 7f8c99ffb640 1 -- 172.21.15.27:0/4034063880 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c012400 con 0x7f8ca4155950 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.849+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca415dbd0 con 0x7f8ca4155950 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.849+0000 7f8c99ffb640 1 -- 172.21.15.27:0/4034063880 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0127f0 con 0x7f8ca4155950 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.849+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca415e190 con 0x7f8ca4155950 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.850+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca404fcf0 con 0x7f8ca4155950 2024-04-20T11:00:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.850+0000 7f8c99ffb640 1 -- 172.21.15.27:0/4034063880 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c9c012990 con 0x7f8ca4155950 2024-04-20T11:00:35.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.851+0000 7f8c99ffb640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8c8806c8f0 0x7f8c8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:35.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.851+0000 7f8c99ffb640 1 -- 172.21.15.27:0/4034063880 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f8c9c08c5a0 con 0x7f8ca4155950 2024-04-20T11:00:35.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.851+0000 7f8ca92d4640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8c8806c8f0 0x7f8c8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:35.853 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.852+0000 7f8ca92d4640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8c8806c8f0 0x7f8c8806edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00011e0 tx=0x7f8ca0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:35.856 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:35.855+0000 7f8c99ffb640 1 -- 172.21.15.27:0/4034063880 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c9c05a7c0 con 0x7f8ca4155950 2024-04-20T11:00:36.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.082+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ca40cd0a0 con 0x7f8ca4155950 2024-04-20T11:00:36.083 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.082+0000 7f8c99ffb640 1 -- 172.21.15.27:0/4034063880 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6937 (secure 0 0 0) 0x7f8c9c05a0c0 con 0x7f8ca4155950 2024-04-20T11:00:36.083 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:36.083 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:57.421458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,5029,4275]},"id":1}]} 2024-04-20T11:00:36.084 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 35 2024-04-20T11:00:36.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8c8806c8f0 msgr2=0x7f8c8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:36.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8c8806c8f0 0x7f8c8806edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00011e0 tx=0x7f8ca0009040 comp rx=0 tx=0).stop 2024-04-20T11:00:36.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ca4155950 msgr2=0x7f8ca40c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:36.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ca4155950 0x7f8ca40c6f90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00cc10 tx=0x7f8c9c00a430 comp rx=0 tx=0).stop 2024-04-20T11:00:36.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 shutdown_connections 2024-04-20T11:00:36.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8c8806c8f0 0x7f8c8806edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ca4155950 0x7f8ca40c6f90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca40d0c90 0x7f8ca41f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 --2- 172.21.15.27:0/4034063880 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8ca40b54f0 0x7f8ca41f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 >> 172.21.15.27:0/4034063880 conn(0x7f8ca40676c0 msgr2=0x7f8ca40b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 shutdown_connections 2024-04-20T11:00:36.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:36.084+0000 7f8cab538640 1 -- 172.21.15.27:0/4034063880 wait complete. 2024-04-20T11:00:36.096 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-04-20T11:00:38.097 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:38.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.227+0000 7f5b87e32640 1 Processor -- start 2024-04-20T11:00:38.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.227+0000 7f5b87e32640 1 -- start start 2024-04-20T11:00:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.228+0000 7f5b87e32640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80155320 0x7f5b800c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.228+0000 7f5b853cd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5b80073470 0x7f5b8006f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56700/0 (socket says 172.21.15.27:56700) 2024-04-20T11:00:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.228+0000 7f5b87e32640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b800c8760 con 0x7f5b80073470 2024-04-20T11:00:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.228+0000 7f5b853cd640 1 -- 172.21.15.27:0/3161017932 learned_addr learned my addr 172.21.15.27:0/3161017932 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.228+0000 7f5b87e32640 1 -- 172.21.15.27:0/3161017932 --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8015b4d0 con 0x7f5b800722b0 2024-04-20T11:00:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.228+0000 7f5b87e32640 1 -- 172.21.15.27:0/3161017932 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8015b7b0 con 0x7f5b80155320 2024-04-20T11:00:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.228+0000 7f5b85bce640 1 --2- 172.21.15.27:0/3161017932 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80155320 0x7f5b800c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.228+0000 7f5b84bcc640 1 -- 172.21.15.27:0/3161017932 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 799269458 0 0) 0x7f5b800c8760 con 0x7f5b80073470 2024-04-20T11:00:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.228+0000 7f5b84bcc640 1 -- 172.21.15.27:0/3161017932 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b6c003520 con 0x7f5b80073470 2024-04-20T11:00:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.229+0000 7f5b85bce640 1 -- 172.21.15.27:0/3161017932 >> v1:172.21.15.137:6789/0 conn(0x7f5b800722b0 legacy=0x7f5b80072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.229+0000 7f5b85bce640 1 -- 172.21.15.27:0/3161017932 >> v1:172.21.15.27:6789/0 conn(0x7f5b80073470 legacy=0x7f5b8006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.229+0000 7f5b85bce640 1 -- 172.21.15.27:0/3161017932 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8015ba90 con 0x7f5b80155320 2024-04-20T11:00:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.230+0000 7f5b85bce640 1 --2- 172.21.15.27:0/3161017932 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80155320 0x7f5b800c9e90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0097d0 tx=0x7f5b7c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ecc6e50239f871 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.230+0000 7f5b84bcc640 1 -- 172.21.15.27:0/3161017932 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c035020 con 0x7f5b80155320 2024-04-20T11:00:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.230+0000 7f5b84bcc640 1 -- 172.21.15.27:0/3161017932 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c031890 con 0x7f5b80155320 2024-04-20T11:00:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.230+0000 7f5b84bcc640 1 -- 172.21.15.27:0/3161017932 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c031c60 con 0x7f5b80155320 2024-04-20T11:00:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.231+0000 7f5b87e32640 1 -- 172.21.15.27:0/3161017932 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80155320 msgr2=0x7f5b800c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.231+0000 7f5b87e32640 1 --2- 172.21.15.27:0/3161017932 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80155320 0x7f5b800c9e90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0097d0 tx=0x7f5b7c02b930 comp rx=0 tx=0).stop 2024-04-20T11:00:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.231+0000 7f5b87e32640 1 -- 172.21.15.27:0/3161017932 shutdown_connections 2024-04-20T11:00:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.231+0000 7f5b87e32640 1 --2- 172.21.15.27:0/3161017932 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80155320 0x7f5b800c9e90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.231+0000 7f5b87e32640 1 -- 172.21.15.27:0/3161017932 >> 172.21.15.27:0/3161017932 conn(0x7f5b800cd9c0 msgr2=0x7f5b800cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.231+0000 7f5b87e32640 1 -- 172.21.15.27:0/3161017932 shutdown_connections 2024-04-20T11:00:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.231+0000 7f5b87e32640 1 -- 172.21.15.27:0/3161017932 wait complete. 2024-04-20T11:00:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.232+0000 7f5b87e32640 1 Processor -- start 2024-04-20T11:00:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b87e32640 1 -- start start 2024-04-20T11:00:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b87e32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b800722b0 0x7f5b801f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b87e32640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80073470 0x7f5b80156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b87e32640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b80155320 0x7f5b80156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b87e32640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b800c8340 con 0x7f5b80155320 2024-04-20T11:00:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b87e32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8015c050 con 0x7f5b800722b0 2024-04-20T11:00:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b87e32640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5b80155700 con 0x7f5b80073470 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b863cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b80155320 0x7f5b80156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b85bce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b800722b0 0x7f5b801f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b853cd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80073470 0x7f5b80156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b863cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b80155320 0x7f5b80156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55028/0 (socket says 172.21.15.27:55028) 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b85bce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b800722b0 0x7f5b801f1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:36936/0 (socket says 172.21.15.27:36936) 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.233+0000 7f5b863cf640 1 -- 172.21.15.27:0/3567999179 learned_addr learned my addr 172.21.15.27:0/3567999179 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b863cf640 1 -- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80073470 msgr2=0x7f5b80156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b863cf640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80073470 0x7f5b80156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b863cf640 1 -- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b800722b0 msgr2=0x7f5b801f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b863cf640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b800722b0 0x7f5b801f1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b863cf640 1 -- 172.21.15.27:0/3567999179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b801f19c0 con 0x7f5b80155320 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b85bce640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b800722b0 0x7f5b801f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b863cf640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b80155320 0x7f5b80156760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c760 tx=0x7f5b7800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b767fc640 1 -- 172.21.15.27:0/3567999179 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78003b90 con 0x7f5b80155320 2024-04-20T11:00:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b767fc640 1 -- 172.21.15.27:0/3567999179 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b78003d30 con 0x7f5b80155320 2024-04-20T11:00:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b801f4110 con 0x7f5b80155320 2024-04-20T11:00:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b767fc640 1 -- 172.21.15.27:0/3567999179 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800aa90 con 0x7f5b80155320 2024-04-20T11:00:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.234+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b801f0f60 con 0x7f5b80155320 2024-04-20T11:00:38.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.235+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b801f0a70 con 0x7f5b80155320 2024-04-20T11:00:38.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.236+0000 7f5b767fc640 1 -- 172.21.15.27:0/3567999179 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b7800ac30 con 0x7f5b80155320 2024-04-20T11:00:38.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.236+0000 7f5b767fc640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b6406c8f0 0x7f5b6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:38.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.236+0000 7f5b767fc640 1 -- 172.21.15.27:0/3567999179 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7f5b7808ba20 con 0x7f5b80155320 2024-04-20T11:00:38.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.236+0000 7f5b85bce640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b6406c8f0 0x7f5b6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:38.238 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.237+0000 7f5b85bce640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b6406c8f0 0x7f5b6406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c036000 tx=0x7f5b7c03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.241+0000 7f5b767fc640 1 -- 172.21.15.27:0/3567999179 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5b78059690 con 0x7f5b80155320 2024-04-20T11:00:38.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.469+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b8004fcf0 con 0x7f5b80155320 2024-04-20T11:00:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.469+0000 7f5b767fc640 1 -- 172.21.15.27:0/3567999179 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6937 (secure 0 0 0) 0x7f5b7805d990 con 0x7f5b80155320 2024-04-20T11:00:38.470 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:38.470 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:57.421458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,5029,4275]},"id":1}]} 2024-04-20T11:00:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 35 2024-04-20T11:00:38.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b6406c8f0 msgr2=0x7f5b6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:38.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b6406c8f0 0x7f5b6406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c036000 tx=0x7f5b7c03c040 comp rx=0 tx=0).stop 2024-04-20T11:00:38.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b80155320 msgr2=0x7f5b80156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:38.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b80155320 0x7f5b80156760 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c760 tx=0x7f5b7800cc20 comp rx=0 tx=0).stop 2024-04-20T11:00:38.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 shutdown_connections 2024-04-20T11:00:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b6406c8f0 0x7f5b6406edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b80155320 0x7f5b80156760 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b80073470 0x7f5b80156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 --2- 172.21.15.27:0/3567999179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b800722b0 0x7f5b801f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 >> 172.21.15.27:0/3567999179 conn(0x7f5b800cd9c0 msgr2=0x7f5b80158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 shutdown_connections 2024-04-20T11:00:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:38.471+0000 7f5b87e32640 1 -- 172.21.15.27:0/3567999179 wait complete. 2024-04-20T11:00:38.483 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-04-20T11:00:40.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:40.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.598+0000 7fa829c14640 1 Processor -- start 2024-04-20T11:00:40.599 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.598+0000 7fa829c14640 1 -- start start 2024-04-20T11:00:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.598+0000 7fa829c14640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa824155b20 0x7fa8240c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.598+0000 7fa829c14640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415ec60 0x7fa82415be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.598+0000 7fa829c14640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa82415c390 0x7fa8240cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.598+0000 7fa829c14640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa8240c67c0 con 0x7fa824155b20 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.598+0000 7fa829c14640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fa82415b670 con 0x7fa82415c390 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.598+0000 7fa829c14640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fa82415f150 con 0x7fa82415ec60 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa8237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa824155b20 0x7fa8240c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa8237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa824155b20 0x7fa8240c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55030/0 (socket says 172.21.15.27:55030) 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa8237fe640 1 -- 172.21.15.27:0/635558034 learned_addr learned my addr 172.21.15.27:0/635558034 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa823fff640 1 --2- 172.21.15.27:0/635558034 >> v2:172.21.15.137:3300/0 conn(0x7fa82415c390 0x7fa8240cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa822ffd640 1 --2- 172.21.15.27:0/635558034 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415ec60 0x7fa82415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa8237fe640 1 -- 172.21.15.27:0/635558034 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415ec60 msgr2=0x7fa82415be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa8237fe640 1 --2- 172.21.15.27:0/635558034 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415ec60 0x7fa82415be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa8237fe640 1 -- 172.21.15.27:0/635558034 >> v2:172.21.15.137:3300/0 conn(0x7fa82415c390 msgr2=0x7fa8240cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa8237fe640 1 --2- 172.21.15.27:0/635558034 >> v2:172.21.15.137:3300/0 conn(0x7fa82415c390 0x7fa8240cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa8237fe640 1 -- 172.21.15.27:0/635558034 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82415f430 con 0x7fa824155b20 2024-04-20T11:00:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.599+0000 7fa823fff640 1 --2- 172.21.15.27:0/635558034 >> v2:172.21.15.137:3300/0 conn(0x7fa82415c390 0x7fa8240cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:00:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.600+0000 7fa8237fe640 1 --2- 172.21.15.27:0/635558034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824155b20 0x7fa8240c5bb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa82406d580 tx=0x7fa81802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d1ea0eb175c6de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.600+0000 7fa8227fc640 1 -- 172.21.15.27:0/635558034 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818035020 con 0x7fa824155b20 2024-04-20T11:00:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.600+0000 7fa8227fc640 1 -- 172.21.15.27:0/635558034 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa818031870 con 0x7fa824155b20 2024-04-20T11:00:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.600+0000 7fa8227fc640 1 -- 172.21.15.27:0/635558034 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818031c40 con 0x7fa824155b20 2024-04-20T11:00:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.600+0000 7fa829c14640 1 -- 172.21.15.27:0/635558034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824155b20 msgr2=0x7fa8240c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.600+0000 7fa829c14640 1 --2- 172.21.15.27:0/635558034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824155b20 0x7fa8240c5bb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa82406d580 tx=0x7fa81802b910 comp rx=0 tx=0).stop 2024-04-20T11:00:40.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.601+0000 7fa829c14640 1 -- 172.21.15.27:0/635558034 shutdown_connections 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.601+0000 7fa829c14640 1 --2- 172.21.15.27:0/635558034 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415ec60 0x7fa82415be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.601+0000 7fa829c14640 1 --2- 172.21.15.27:0/635558034 >> v2:172.21.15.137:3300/0 conn(0x7fa82415c390 0x7fa8240cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.601+0000 7fa829c14640 1 --2- 172.21.15.27:0/635558034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa824155b20 0x7fa8240c5bb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.601+0000 7fa829c14640 1 -- 172.21.15.27:0/635558034 >> 172.21.15.27:0/635558034 conn(0x7fa8240678a0 msgr2=0x7fa824067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.601+0000 7fa829c14640 1 -- 172.21.15.27:0/635558034 shutdown_connections 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.601+0000 7fa829c14640 1 -- 172.21.15.27:0/635558034 wait complete. 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.602+0000 7fa829c14640 1 Processor -- start 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.602+0000 7fa829c14640 1 -- start start 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa829c14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa824155b20 0x7fa8241e7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa829c14640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415c390 0x7fa82414fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa829c14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82415ec60 0x7fa82415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa829c14640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa82415b930 con 0x7fa82415ec60 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa829c14640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa82415f9f0 con 0x7fa824155b20 2024-04-20T11:00:40.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa829c14640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fa8240c6270 con 0x7fa82415c390 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa823fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82415ec60 0x7fa82415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa823fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82415ec60 0x7fa82415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55042/0 (socket says 172.21.15.27:55042) 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa823fff640 1 -- 172.21.15.27:0/544954013 learned_addr learned my addr 172.21.15.27:0/544954013 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa8237fe640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa824155b20 0x7fa8241e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa822ffd640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415c390 0x7fa82414fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa823fff640 1 -- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415c390 msgr2=0x7fa82414fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa823fff640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415c390 0x7fa82414fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa823fff640 1 -- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa824155b20 msgr2=0x7fa8241e7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa823fff640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa824155b20 0x7fa8241e7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa823fff640 1 -- 172.21.15.27:0/544954013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8240ce490 con 0x7fa82415ec60 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa823fff640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82415ec60 0x7fa82415ad80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007ef0 tx=0x7fa81c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.603+0000 7fa80bfff640 1 -- 172.21.15.27:0/544954013 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c01a070 con 0x7fa82415ec60 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.604+0000 7fa80bfff640 1 -- 172.21.15.27:0/544954013 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c015600 con 0x7fa82415ec60 2024-04-20T11:00:40.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.604+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa824159ce0 con 0x7fa82415ec60 2024-04-20T11:00:40.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.604+0000 7fa80bfff640 1 -- 172.21.15.27:0/544954013 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c0159f0 con 0x7fa82415ec60 2024-04-20T11:00:40.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.604+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82415a2a0 con 0x7fa82415ec60 2024-04-20T11:00:40.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.605+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8240635c0 con 0x7fa82415ec60 2024-04-20T11:00:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.605+0000 7fa80bfff640 1 -- 172.21.15.27:0/544954013 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa81c01d070 con 0x7fa82415ec60 2024-04-20T11:00:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.605+0000 7fa80bfff640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa81006c8f0 0x7fa81006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.606+0000 7fa80bfff640 1 -- 172.21.15.27:0/544954013 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6083+0+0 (secure 0 0 0) 0x7fa81c08bf10 con 0x7fa82415ec60 2024-04-20T11:00:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.606+0000 7fa8237fe640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa81006c8f0 0x7fa81006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:40.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.606+0000 7fa8237fe640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa81006c8f0 0x7fa81006edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa818036000 tx=0x7fa81803c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:40.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.610+0000 7fa80bfff640 1 -- 172.21.15.27:0/544954013 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa8240635c0 con 0x7fa82415ec60 2024-04-20T11:00:40.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.845+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa82404fcf0 con 0x7fa82415ec60 2024-04-20T11:00:40.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.846+0000 7fa80bfff640 1 -- 172.21.15.27:0/544954013 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+6937 (secure 0 0 0) 0x7fa81c010070 con 0x7fa82415ec60 2024-04-20T11:00:40.847 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:40.847 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5143,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T10:59:57.421458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":25,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":4275,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"e","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6837/1680990583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1680990583},{"type":"v1","addr":"172.21.15.27:6837","nonce":1680990583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4894,4293,4278,5029,4275]},"id":1}]} 2024-04-20T11:00:40.847 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 35 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa81006c8f0 msgr2=0x7fa81006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa81006c8f0 0x7fa81006edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa818036000 tx=0x7fa81803c040 comp rx=0 tx=0).stop 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82415ec60 msgr2=0x7fa82415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82415ec60 0x7fa82415ad80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007ef0 tx=0x7fa81c00d4a0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 shutdown_connections 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa81006c8f0 0x7fa81006edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa82415ec60 0x7fa82415ad80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa82415c390 0x7fa82414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 --2- 172.21.15.27:0/544954013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa824155b20 0x7fa8241e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:40.849 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 >> 172.21.15.27:0/544954013 conn(0x7fa8240678a0 msgr2=0x7fa824152620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:40.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 shutdown_connections 2024-04-20T11:00:40.850 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:40.848+0000 7fa829c14640 1 -- 172.21.15.27:0/544954013 wait complete. 2024-04-20T11:00:40.860 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-04-20T11:00:42.598 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:00:42.601 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:00:42.861 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.964+0000 7f27796ba640 1 Processor -- start 2024-04-20T11:00:42.965 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.964+0000 7f27796ba640 1 -- start start 2024-04-20T11:00:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.964+0000 7f27796ba640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 0x7f27740c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.964+0000 7f27796ba640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f27740c6270 0x7f27740cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.964+0000 7f27796ba640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27740c67c0 con 0x7f277415ec60 2024-04-20T11:00:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.964+0000 7f27796ba640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f277415b670 con 0x7f27740c6270 2024-04-20T11:00:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.964+0000 7f27796ba640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f277415f150 con 0x7f2774155b20 2024-04-20T11:00:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f276bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f277415ec60 0x7f277415be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56704/0 (socket says 172.21.15.27:56704) 2024-04-20T11:00:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f276bfff640 1 -- 172.21.15.27:0/749154910 learned_addr learned my addr 172.21.15.27:0/749154910 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f27737fe640 1 --2- 172.21.15.27:0/749154910 >> v2:172.21.15.137:3300/0 conn(0x7f27740c6270 0x7f27740cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f2772ffd640 1 --2- 172.21.15.27:0/749154910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 0x7f27740c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f27727fc640 1 -- 172.21.15.27:0/749154910 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1872405325 0 0) 0x7f27740c67c0 con 0x7f277415ec60 2024-04-20T11:00:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f27727fc640 1 -- 172.21.15.27:0/749154910 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2760003520 con 0x7f277415ec60 2024-04-20T11:00:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f27737fe640 1 -- 172.21.15.27:0/749154910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 msgr2=0x7f27740c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f27737fe640 1 --2- 172.21.15.27:0/749154910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 0x7f27740c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f27737fe640 1 -- 172.21.15.27:0/749154910 >> v1:172.21.15.27:6789/0 conn(0x7f277415ec60 legacy=0x7f277415be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f27737fe640 1 -- 172.21.15.27:0/749154910 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277415f430 con 0x7f27740c6270 2024-04-20T11:00:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.965+0000 7f2772ffd640 1 --2- 172.21.15.27:0/749154910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 0x7f27740c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:00:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.966+0000 7f27737fe640 1 --2- 172.21.15.27:0/749154910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27740c6270 0x7f27740cbd90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f276c009d90 tx=0x7f276c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f3917e27dd479a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.966+0000 7f27727fc640 1 -- 172.21.15.27:0/749154910 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c039070 con 0x7f27740c6270 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.966+0000 7f27727fc640 1 -- 172.21.15.27:0/749154910 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276c02cc50 con 0x7f27740c6270 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.966+0000 7f27727fc640 1 -- 172.21.15.27:0/749154910 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c034440 con 0x7f27740c6270 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.967+0000 7f27796ba640 1 -- 172.21.15.27:0/749154910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27740c6270 msgr2=0x7f27740cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.967+0000 7f27796ba640 1 --2- 172.21.15.27:0/749154910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27740c6270 0x7f27740cbd90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f276c009d90 tx=0x7f276c02bea0 comp rx=0 tx=0).stop 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.967+0000 7f27796ba640 1 -- 172.21.15.27:0/749154910 shutdown_connections 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.967+0000 7f27796ba640 1 --2- 172.21.15.27:0/749154910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27740c6270 0x7f27740cbd90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.967+0000 7f27796ba640 1 --2- 172.21.15.27:0/749154910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 0x7f27740c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.967+0000 7f27796ba640 1 -- 172.21.15.27:0/749154910 >> 172.21.15.27:0/749154910 conn(0x7f27740678a0 msgr2=0x7f2774067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.967+0000 7f27796ba640 1 -- 172.21.15.27:0/749154910 shutdown_connections 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.967+0000 7f27796ba640 1 -- 172.21.15.27:0/749154910 wait complete. 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.968+0000 7f27796ba640 1 Processor -- start 2024-04-20T11:00:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.968+0000 7f27796ba640 1 -- start start 2024-04-20T11:00:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f27796ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740c6270 0x7f27741f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f27796ba640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 0x7f27740c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f27796ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277415ec60 0x7f2774146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f27796ba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f277415b930 con 0x7f27740c6270 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f27796ba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f277415f9f0 con 0x7f277415ec60 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f27796ba640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f277415c560 con 0x7f2774155b20 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f2772ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740c6270 0x7f27741f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f2772ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740c6270 0x7f27741f2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55058/0 (socket says 172.21.15.27:55058) 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f2772ffd640 1 -- 172.21.15.27:0/3936085364 learned_addr learned my addr 172.21.15.27:0/3936085364 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f27737fe640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277415ec60 0x7f2774146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f276bfff640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 0x7f27740c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f2772ffd640 1 -- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 msgr2=0x7f27740c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f2772ffd640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 0x7f27740c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f2772ffd640 1 -- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277415ec60 msgr2=0x7f2774146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f2772ffd640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277415ec60 0x7f2774146590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f2772ffd640 1 -- 172.21.15.27:0/3936085364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277415a890 con 0x7f27740c6270 2024-04-20T11:00:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f2772ffd640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740c6270 0x7f27741f2240 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f276400e990 tx=0x7f276400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f276b7fe640 1 -- 172.21.15.27:0/3936085364 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764017070 con 0x7f27740c6270 2024-04-20T11:00:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f276b7fe640 1 -- 172.21.15.27:0/3936085364 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276400c9e0 con 0x7f27740c6270 2024-04-20T11:00:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277415ab50 con 0x7f27740c6270 2024-04-20T11:00:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.969+0000 7f276b7fe640 1 -- 172.21.15.27:0/3936085364 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276400cdb0 con 0x7f27740c6270 2024-04-20T11:00:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.970+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27740ceaf0 con 0x7f27740c6270 2024-04-20T11:00:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.971+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f277404fcf0 con 0x7f27740c6270 2024-04-20T11:00:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.971+0000 7f276b7fe640 1 -- 172.21.15.27:0/3936085364 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f276401a070 con 0x7f27740c6270 2024-04-20T11:00:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.972+0000 7f276b7fe640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f275406c8f0 0x7f275406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.972+0000 7f276b7fe640 1 -- 172.21.15.27:0/3936085364 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f276408bb90 con 0x7f27740c6270 2024-04-20T11:00:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.972+0000 7f276bfff640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f275406c8f0 0x7f275406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.972+0000 7f276bfff640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f275406c8f0 0x7f275406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2774147550 tx=0x7f275c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:42.976+0000 7f276b7fe640 1 -- 172.21.15.27:0/3936085364 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2764059710 con 0x7f27740c6270 2024-04-20T11:00:43.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.224+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27740b3880 con 0x7f27740c6270 2024-04-20T11:00:43.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.225+0000 7f276b7fe640 1 -- 172.21.15.27:0/3936085364 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6108 (secure 0 0 0) 0x7f276405da10 con 0x7f27740c6270 2024-04-20T11:00:43.226 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:43.226 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:00:42.349233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4278,4293,4894,5029]},"id":1}]} 2024-04-20T11:00:43.226 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 36 2024-04-20T11:00:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f275406c8f0 msgr2=0x7f275406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f275406c8f0 0x7f275406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2774147550 tx=0x7f275c009040 comp rx=0 tx=0).stop 2024-04-20T11:00:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740c6270 msgr2=0x7f27741f2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740c6270 0x7f27741f2240 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f276400e990 tx=0x7f276400ee50 comp rx=0 tx=0).stop 2024-04-20T11:00:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 shutdown_connections 2024-04-20T11:00:43.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f275406c8f0 0x7f275406edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277415ec60 0x7f2774146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2774155b20 0x7f27740c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 --2- 172.21.15.27:0/3936085364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27740c6270 0x7f27741f2240 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 >> 172.21.15.27:0/3936085364 conn(0x7f27740678a0 msgr2=0x7f27740b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:43.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 shutdown_connections 2024-04-20T11:00:43.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.227+0000 7f27796ba640 1 -- 172.21.15.27:0/3936085364 wait complete. 2024-04-20T11:00:43.239 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-04-20T11:00:43.240 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T11:00:43.240 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.341+0000 7f826e1ce640 1 Processor -- start 2024-04-20T11:00:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.341+0000 7f826e1ce640 1 -- start start 2024-04-20T11:00:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f826e1ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8268155950 0x7f82680cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f826e1ce640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826815b9c0 con 0x7f82680b5c00 2024-04-20T11:00:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f826e1ce640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826815bca0 con 0x7f82680cc330 2024-04-20T11:00:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f826e1ce640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f826815bf80 con 0x7f8268155950 2024-04-20T11:00:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f82677fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8268155950 0x7f82680cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f8267fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f82680b5c00 0x7f82680d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56718/0 (socket says 172.21.15.27:56718) 2024-04-20T11:00:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f8267fff640 1 -- 172.21.15.27:0/3571825543 learned_addr learned my addr 172.21.15.27:0/3571825543 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f82677fe640 1 -- 172.21.15.27:0/3571825543 >> v1:172.21.15.137:6789/0 conn(0x7f82680cc330 legacy=0x7f82680b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f82677fe640 1 -- 172.21.15.27:0/3571825543 >> v1:172.21.15.27:6789/0 conn(0x7f82680b5c00 legacy=0x7f82680d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f82677fe640 1 -- 172.21.15.27:0/3571825543 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826815c260 con 0x7f8268155950 2024-04-20T11:00:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.342+0000 7f82667fc640 1 -- 172.21.15.27:0/3571825543 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2023162447 0 0) 0x7f826815b9c0 con 0x7f82680b5c00 2024-04-20T11:00:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.343+0000 7f82677fe640 1 --2- 172.21.15.27:0/3571825543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8268155950 0x7f82680cbdf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82580097a0 tx=0x7f825802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edba40d907159d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.343+0000 7f82667fc640 1 -- 172.21.15.27:0/3571825543 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825802f810 con 0x7f8268155950 2024-04-20T11:00:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.343+0000 7f82667fc640 1 -- 172.21.15.27:0/3571825543 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825802f9b0 con 0x7f8268155950 2024-04-20T11:00:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.343+0000 7f82667fc640 1 -- 172.21.15.27:0/3571825543 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825802fd80 con 0x7f8268155950 2024-04-20T11:00:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.344+0000 7f826e1ce640 1 -- 172.21.15.27:0/3571825543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8268155950 msgr2=0x7f82680cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.344+0000 7f826e1ce640 1 --2- 172.21.15.27:0/3571825543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8268155950 0x7f82680cbdf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82580097a0 tx=0x7f825802b9f0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.344+0000 7f826e1ce640 1 -- 172.21.15.27:0/3571825543 shutdown_connections 2024-04-20T11:00:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.344+0000 7f826e1ce640 1 --2- 172.21.15.27:0/3571825543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8268155950 0x7f82680cbdf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.344+0000 7f826e1ce640 1 -- 172.21.15.27:0/3571825543 >> 172.21.15.27:0/3571825543 conn(0x7f82680676c0 msgr2=0x7f8268067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:43.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.344+0000 7f826e1ce640 1 -- 172.21.15.27:0/3571825543 shutdown_connections 2024-04-20T11:00:43.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.344+0000 7f826e1ce640 1 -- 172.21.15.27:0/3571825543 wait complete. 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.345+0000 7f826e1ce640 1 Processor -- start 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.345+0000 7f826e1ce640 1 -- start start 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f826e1ce640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82680b5c00 0x7f82680cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f826e1ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82680cc330 0x7f82680d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f826e1ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268155950 0x7f82681e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f826e1ce640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f826815fd10 con 0x7f82680cc330 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f826e1ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f826815fb90 con 0x7f8268155950 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f826e1ce640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8268155d30 con 0x7f82680b5c00 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8266ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82680cc330 0x7f82680d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8266ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82680cc330 0x7f82680d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55082/0 (socket says 172.21.15.27:55082) 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8266ffd640 1 -- 172.21.15.27:0/826968787 learned_addr learned my addr 172.21.15.27:0/826968787 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f82677fe640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82680b5c00 0x7f82680cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8267fff640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268155950 0x7f82681e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8266ffd640 1 -- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82680b5c00 msgr2=0x7f82680cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8266ffd640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82680b5c00 0x7f82680cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8266ffd640 1 -- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268155950 msgr2=0x7f82681e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8266ffd640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268155950 0x7f82681e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8266ffd640 1 -- 172.21.15.27:0/826968787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82681f5510 con 0x7f82680cc330 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.346+0000 7f8266ffd640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82680cc330 0x7f82680d0c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f825c00c430 tx=0x7f825c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.347+0000 7f824ffff640 1 -- 172.21.15.27:0/826968787 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c017070 con 0x7f82680cc330 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.347+0000 7f824ffff640 1 -- 172.21.15.27:0/826968787 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825c0052d0 con 0x7f82680cc330 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.347+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82681f1520 con 0x7f82680cc330 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.347+0000 7f824ffff640 1 -- 172.21.15.27:0/826968787 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c003830 con 0x7f82680cc330 2024-04-20T11:00:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.347+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82681f1ae0 con 0x7f82680cc330 2024-04-20T11:00:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.348+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826804fcf0 con 0x7f82680cc330 2024-04-20T11:00:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.348+0000 7f824ffff640 1 -- 172.21.15.27:0/826968787 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f825c01a070 con 0x7f82680cc330 2024-04-20T11:00:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.349+0000 7f824ffff640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f825406c8f0 0x7f825406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.349+0000 7f824ffff640 1 -- 172.21.15.27:0/826968787 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f825c08af50 con 0x7f82680cc330 2024-04-20T11:00:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.349+0000 7f82677fe640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f825406c8f0 0x7f825406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:43.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.349+0000 7f82677fe640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f825406c8f0 0x7f825406edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8258036000 tx=0x7f8258008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:43.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.353+0000 7f824ffff640 1 -- 172.21.15.27:0/826968787 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f825c058bf0 con 0x7f82680cc330 2024-04-20T11:00:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.588+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82680cc930 con 0x7f82680cc330 2024-04-20T11:00:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.589+0000 7f824ffff640 1 -- 172.21.15.27:0/826968787 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6108 (secure 0 0 0) 0x7f825c05cef0 con 0x7f82680cc330 2024-04-20T11:00:43.590 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:43.590 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:00:42.349233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4278,4293,4894,5029]},"id":1}]} 2024-04-20T11:00:43.590 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 36 2024-04-20T11:00:43.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.590+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f825406c8f0 msgr2=0x7f825406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:43.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.590+0000 7f826e1ce640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f825406c8f0 0x7f825406edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8258036000 tx=0x7f8258008030 comp rx=0 tx=0).stop 2024-04-20T11:00:43.591 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.590+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82680cc330 msgr2=0x7f82680d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.590+0000 7f826e1ce640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82680cc330 0x7f82680d0c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f825c00c430 tx=0x7f825c00c8f0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.591+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 shutdown_connections 2024-04-20T11:00:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.591+0000 7f826e1ce640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f825406c8f0 0x7f825406edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.591+0000 7f826e1ce640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268155950 0x7f82681e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.591+0000 7f826e1ce640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82680cc330 0x7f82680d0c10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.591+0000 7f826e1ce640 1 --2- 172.21.15.27:0/826968787 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f82680b5c00 0x7f82680cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.591+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 >> 172.21.15.27:0/826968787 conn(0x7f82680676c0 msgr2=0x7f82681517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.591+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 shutdown_connections 2024-04-20T11:00:43.592 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:43.591+0000 7f826e1ce640 1 -- 172.21.15.27:0/826968787 wait complete. 2024-04-20T11:00:43.604 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 36, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 36, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4278": { "addr": "172.21.15.27:6836/4195543922", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6833", "nonce": 4195543922, "type": "v2" }, { "addr": "172.21.15.27:6836", "nonce": 4195543922, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4278, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4894": { "addr": "172.21.15.137:6838/2896327439", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6836", "nonce": 2896327439, "type": "v2" }, { "addr": "172.21.15.137:6838", "nonce": 2896327439, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4894, "incarnation": 24, "join_fscid": -1, "name": "b", "rank": 4, "state": "up:active", "state_seq": 17 }, "gid_5029": { "addr": "172.21.15.137:6835/4235742413", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6834", "nonce": 4235742413, "type": "v2" }, { "addr": "172.21.15.137:6835", "nonce": 4235742413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5029, "incarnation": 30, "join_fscid": -1, "name": "f", "rank": 3, "state": "up:active", "state_seq": 22 }, "gid_5143": { "addr": "172.21.15.27:6835/2020174232", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6832", "nonce": 2020174232, "type": "v2" }, { "addr": "172.21.15.27:6835", "nonce": 2020174232, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5143, "incarnation": 36, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 26, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T11:00:42.349233+0000", "qdb_cluster": [ 4278, 4293, 4894, 5029 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 4278, "mds_2": 5143, "mds_3": 5029, "mds_4": 4894 } } } ], "standbys": [] } 2024-04-20T11:00:45.605 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:45.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.721+0000 7fee4e2ba640 1 Processor -- start 2024-04-20T11:00:45.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.721+0000 7fee4e2ba640 1 -- start start 2024-04-20T11:00:45.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee4e2ba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee4806ca40 0x7fee4815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:45.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee4e2ba640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee4815ec70 con 0x7fee4815f710 2024-04-20T11:00:45.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee4e2ba640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee4806e510 con 0x7fee481556f0 2024-04-20T11:00:45.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee4e2ba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fee4806ead0 con 0x7fee4806ca40 2024-04-20T11:00:45.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee4c857640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee4806ca40 0x7fee4815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:45.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee477fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fee4815f710 0x7fee48163e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47178/0 (socket says 172.21.15.27:47178) 2024-04-20T11:00:45.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee477fe640 1 -- 172.21.15.27:0/1193091017 learned_addr learned my addr 172.21.15.27:0/1193091017 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:45.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee4c857640 1 -- 172.21.15.27:0/1193091017 >> v1:172.21.15.137:6789/0 conn(0x7fee481556f0 legacy=0x7fee480cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:45.723 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee4c857640 1 -- 172.21.15.27:0/1193091017 >> v1:172.21.15.27:6789/0 conn(0x7fee4815f710 legacy=0x7fee48163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:45.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee46ffd640 1 -- 172.21.15.27:0/1193091017 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 211319798 0 0) 0x7fee4806e510 con 0x7fee481556f0 2024-04-20T11:00:45.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.722+0000 7fee4c857640 1 -- 172.21.15.27:0/1193091017 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4806e7f0 con 0x7fee4806ca40 2024-04-20T11:00:45.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.723+0000 7fee4c857640 1 --2- 172.21.15.27:0/1193091017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4806ca40 0x7fee4815c9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fee40009d90 tx=0x7fee4002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9188c18a8c94325c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:45.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.723+0000 7fee46ffd640 1 -- 172.21.15.27:0/1193091017 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40039070 con 0x7fee4806ca40 2024-04-20T11:00:45.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.723+0000 7fee46ffd640 1 -- 172.21.15.27:0/1193091017 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4002cb40 con 0x7fee4806ca40 2024-04-20T11:00:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.723+0000 7fee46ffd640 1 -- 172.21.15.27:0/1193091017 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40034440 con 0x7fee4806ca40 2024-04-20T11:00:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.724+0000 7fee4e2ba640 1 -- 172.21.15.27:0/1193091017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4806ca40 msgr2=0x7fee4815c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.724+0000 7fee4e2ba640 1 --2- 172.21.15.27:0/1193091017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4806ca40 0x7fee4815c9d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fee40009d90 tx=0x7fee4002be10 comp rx=0 tx=0).stop 2024-04-20T11:00:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.724+0000 7fee4e2ba640 1 -- 172.21.15.27:0/1193091017 shutdown_connections 2024-04-20T11:00:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.724+0000 7fee4e2ba640 1 --2- 172.21.15.27:0/1193091017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4806ca40 0x7fee4815c9d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.724+0000 7fee4e2ba640 1 -- 172.21.15.27:0/1193091017 >> 172.21.15.27:0/1193091017 conn(0x7fee48067ae0 msgr2=0x7fee48067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.724+0000 7fee4e2ba640 1 -- 172.21.15.27:0/1193091017 shutdown_connections 2024-04-20T11:00:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.724+0000 7fee4e2ba640 1 -- 172.21.15.27:0/1193091017 wait complete. 2024-04-20T11:00:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.725+0000 7fee4e2ba640 1 Processor -- start 2024-04-20T11:00:45.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.725+0000 7fee4e2ba640 1 -- start start 2024-04-20T11:00:45.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4e2ba640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fee4806ca40 0x7fee481f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:45.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4e2ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee481556f0 0x7fee48159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:45.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4e2ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4815f710 0x7fee48159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:45.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4e2ba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fee480c9800 con 0x7fee4815f710 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4e2ba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fee4815f1f0 con 0x7fee481556f0 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4e2ba640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fee480c9980 con 0x7fee4806ca40 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4c857640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4815f710 0x7fee48159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4c857640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4815f710 0x7fee48159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35572/0 (socket says 172.21.15.27:35572) 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4c857640 1 -- 172.21.15.27:0/3811038163 learned_addr learned my addr 172.21.15.27:0/3811038163 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee477fe640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee481556f0 0x7fee48159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee47fff640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fee4806ca40 0x7fee481f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4c857640 1 -- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fee4806ca40 msgr2=0x7fee481f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4c857640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fee4806ca40 0x7fee481f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4c857640 1 -- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee481556f0 msgr2=0x7fee48159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4c857640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee481556f0 0x7fee48159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.726+0000 7fee4c857640 1 -- 172.21.15.27:0/3811038163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4814d590 con 0x7fee4815f710 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.727+0000 7fee4c857640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4815f710 0x7fee48159ec0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fee40009d90 tx=0x7fee40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.727+0000 7fee44ff9640 1 -- 172.21.15.27:0/3811038163 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40044030 con 0x7fee4815f710 2024-04-20T11:00:45.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.727+0000 7fee44ff9640 1 -- 172.21.15.27:0/3811038163 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4002c940 con 0x7fee4815f710 2024-04-20T11:00:45.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.727+0000 7fee44ff9640 1 -- 172.21.15.27:0/3811038163 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40039040 con 0x7fee4815f710 2024-04-20T11:00:45.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.727+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee481f5160 con 0x7fee4815f710 2024-04-20T11:00:45.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.727+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee481534c0 con 0x7fee4815f710 2024-04-20T11:00:45.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.728+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee480688e0 con 0x7fee4815f710 2024-04-20T11:00:45.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.729+0000 7fee44ff9640 1 -- 172.21.15.27:0/3811038163 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee40047070 con 0x7fee4815f710 2024-04-20T11:00:45.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.729+0000 7fee44ff9640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee3406c920 0x7fee3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:45.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.729+0000 7fee44ff9640 1 -- 172.21.15.27:0/3811038163 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7fee400ade50 con 0x7fee4815f710 2024-04-20T11:00:45.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.729+0000 7fee47fff640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee3406c920 0x7fee3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:45.732 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.730+0000 7fee47fff640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee3406c920 0x7fee3406ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fee38000f60 tx=0x7fee38009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:45.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.733+0000 7fee44ff9640 1 -- 172.21.15.27:0/3811038163 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fee4007bb20 con 0x7fee4815f710 2024-04-20T11:00:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.966+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee480cea20 con 0x7fee4815f710 2024-04-20T11:00:45.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.967+0000 7fee44ff9640 1 -- 172.21.15.27:0/3811038163 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6108 (secure 0 0 0) 0x7fee4007fe20 con 0x7fee4815f710 2024-04-20T11:00:45.968 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:45.968 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:00:42.349233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4278,4293,4894,5029]},"id":1}]} 2024-04-20T11:00:45.968 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 36 2024-04-20T11:00:45.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee3406c920 msgr2=0x7fee3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:45.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee3406c920 0x7fee3406ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fee38000f60 tx=0x7fee38009040 comp rx=0 tx=0).stop 2024-04-20T11:00:45.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4815f710 msgr2=0x7fee48159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:45.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4815f710 0x7fee48159ec0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fee40009d90 tx=0x7fee40004a40 comp rx=0 tx=0).stop 2024-04-20T11:00:45.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 shutdown_connections 2024-04-20T11:00:45.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fee3406c920 0x7fee3406ede0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:45.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee4815f710 0x7fee48159ec0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:45.970 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee481556f0 0x7fee48159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:45.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 --2- 172.21.15.27:0/3811038163 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fee4806ca40 0x7fee481f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:45.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 >> 172.21.15.27:0/3811038163 conn(0x7fee48067ae0 msgr2=0x7fee480cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:45.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 shutdown_connections 2024-04-20T11:00:45.971 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:45.969+0000 7fee4e2ba640 1 -- 172.21.15.27:0/3811038163 wait complete. 2024-04-20T11:00:45.981 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:00:47.981 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:48.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.095+0000 7f5567166640 1 Processor -- start 2024-04-20T11:00:48.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.095+0000 7f5567166640 1 -- start start 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.096+0000 7f5567166640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f556015ed10 0x7f5560073970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.096+0000 7f5567166640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f556015f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.096+0000 7f5567166640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5560073fc0 con 0x7f5560072130 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.096+0000 7f5567166640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f55600c3bc0 con 0x7f556015ed10 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.096+0000 7f5567166640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f55600c4050 con 0x7f556015f0f0 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.096+0000 7f5564f02640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f556015ed10 0x7f5560073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.096+0000 7f5564f02640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f556015ed10 0x7f5560073970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35588/0 (socket says 172.21.15.27:35588) 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.096+0000 7f5564f02640 1 -- 172.21.15.27:0/3950553529 learned_addr learned my addr 172.21.15.27:0/3950553529 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.097+0000 7f5565703640 1 --2- 172.21.15.27:0/3950553529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f556015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.097+0000 7f5564f02640 1 -- 172.21.15.27:0/3950553529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 msgr2=0x7f556015f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.097+0000 7f5564f02640 1 --2- 172.21.15.27:0/3950553529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f556015f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.097+0000 7f5564f02640 1 -- 172.21.15.27:0/3950553529 >> v1:172.21.15.137:6789/0 conn(0x7f5560072130 legacy=0x7f55600c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:48.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.097+0000 7f5564f02640 1 -- 172.21.15.27:0/3950553529 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55600c2fb0 con 0x7f556015ed10 2024-04-20T11:00:48.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.097+0000 7f5565703640 1 --2- 172.21.15.27:0/3950553529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f556015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:00:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.097+0000 7f5564f02640 1 --2- 172.21.15.27:0/3950553529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f556015ed10 0x7f5560073970 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5550009870 tx=0x7f555002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c98718b46344be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f555f7fe640 1 -- 172.21.15.27:0/3950553529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550035020 con 0x7f556015ed10 2024-04-20T11:00:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f555f7fe640 1 -- 172.21.15.27:0/3950553529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5550031640 con 0x7f556015ed10 2024-04-20T11:00:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f555f7fe640 1 -- 172.21.15.27:0/3950553529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555002d040 con 0x7f556015ed10 2024-04-20T11:00:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f5567166640 1 -- 172.21.15.27:0/3950553529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f556015ed10 msgr2=0x7f5560073970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f5567166640 1 --2- 172.21.15.27:0/3950553529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f556015ed10 0x7f5560073970 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5550009870 tx=0x7f555002b8f0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f5567166640 1 -- 172.21.15.27:0/3950553529 shutdown_connections 2024-04-20T11:00:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f5567166640 1 --2- 172.21.15.27:0/3950553529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f556015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f5567166640 1 --2- 172.21.15.27:0/3950553529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f556015ed10 0x7f5560073970 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f5567166640 1 -- 172.21.15.27:0/3950553529 >> 172.21.15.27:0/3950553529 conn(0x7f55600ca8f0 msgr2=0x7f55600cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f5567166640 1 -- 172.21.15.27:0/3950553529 shutdown_connections 2024-04-20T11:00:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.098+0000 7f5567166640 1 -- 172.21.15.27:0/3950553529 wait complete. 2024-04-20T11:00:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.099+0000 7f5567166640 1 Processor -- start 2024-04-20T11:00:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.100+0000 7f5567166640 1 -- start start 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.100+0000 7f5567166640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5560072130 0x7f556014ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.100+0000 7f5567166640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556015ed10 0x7f556015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.100+0000 7f5567166640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f5560144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.100+0000 7f5567166640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55600c3850 con 0x7f5560072130 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.100+0000 7f5567166640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55600c3570 con 0x7f556015ed10 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.100+0000 7f5567166640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f55600c8f00 con 0x7f556015f0f0 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5564f02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5560072130 0x7f556014ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5564f02640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5560072130 0x7f556014ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35604/0 (socket says 172.21.15.27:35604) 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5564f02640 1 -- 172.21.15.27:0/1802884006 learned_addr learned my addr 172.21.15.27:0/1802884006 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5565703640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f5560144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f555ffff640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556015ed10 0x7f556015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5564f02640 1 -- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 msgr2=0x7f5560144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5564f02640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f5560144a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5564f02640 1 -- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556015ed10 msgr2=0x7f556015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5564f02640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556015ed10 0x7f556015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5564f02640 1 -- 172.21.15.27:0/1802884006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556014eaf0 con 0x7f5560072130 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5565703640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f5560144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f555ffff640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556015ed10 0x7f556015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5564f02640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5560072130 0x7f556014ab50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f555002b930 tx=0x7f55500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f555d7fa640 1 -- 172.21.15.27:0/1802884006 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550040070 con 0x7f5560072130 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f555d7fa640 1 -- 172.21.15.27:0/1802884006 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5550039bf0 con 0x7f5560072130 2024-04-20T11:00:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f555d7fa640 1 -- 172.21.15.27:0/1802884006 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550035040 con 0x7f5560072130 2024-04-20T11:00:48.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556014edd0 con 0x7f5560072130 2024-04-20T11:00:48.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.101+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556014d720 con 0x7f5560072130 2024-04-20T11:00:48.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.102+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556004fcf0 con 0x7f5560072130 2024-04-20T11:00:48.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.103+0000 7f555d7fa640 1 -- 172.21.15.27:0/1802884006 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5550038020 con 0x7f5560072130 2024-04-20T11:00:48.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.103+0000 7f555d7fa640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f554c06c920 0x7f554c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:48.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.103+0000 7f555d7fa640 1 -- 172.21.15.27:0/1802884006 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f55500ad860 con 0x7f5560072130 2024-04-20T11:00:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.103+0000 7f555ffff640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f554c06c920 0x7f554c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.104+0000 7f555ffff640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f554c06c920 0x7f554c06ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f556006ccb0 tx=0x7f5554009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.108+0000 7f555d7fa640 1 -- 172.21.15.27:0/1802884006 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f555007bac0 con 0x7f5560072130 2024-04-20T11:00:48.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.353+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5560072b60 con 0x7f5560072130 2024-04-20T11:00:48.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.354+0000 7f555d7fa640 1 -- 172.21.15.27:0/1802884006 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6110 (secure 0 0 0) 0x7f555007b3c0 con 0x7f5560072130 2024-04-20T11:00:48.354 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:48.354 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:00:47.358424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:resolve","state_seq":13,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:00:48.354 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 37 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f554c06c920 msgr2=0x7f554c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f554c06c920 0x7f554c06ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f556006ccb0 tx=0x7f5554009040 comp rx=0 tx=0).stop 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5560072130 msgr2=0x7f556014ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5560072130 0x7f556014ab50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f555002b930 tx=0x7f55500046c0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 shutdown_connections 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f554c06c920 0x7f554c06ede0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f556015f0f0 0x7f5560144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f556015ed10 0x7f556015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 --2- 172.21.15.27:0/1802884006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5560072130 0x7f556014ab50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 >> 172.21.15.27:0/1802884006 conn(0x7f55600ca8f0 msgr2=0x7f55601550c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 shutdown_connections 2024-04-20T11:00:48.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:48.356+0000 7f5567166640 1 -- 172.21.15.27:0/1802884006 wait complete. 2024-04-20T11:00:48.367 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:00:50.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:50.476 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.477+0000 7f40e6a06640 1 Processor -- start 2024-04-20T11:00:50.477 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.477+0000 7f40e6a06640 1 -- start start 2024-04-20T11:00:50.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.477+0000 7f40e6a06640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015f710 0x7f40e006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:50.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.477+0000 7f40e6a06640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e015ce20 con 0x7f40e015c9d0 2024-04-20T11:00:50.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.477+0000 7f40e6a06640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e015ec70 con 0x7f40e01556f0 2024-04-20T11:00:50.478 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.477+0000 7f40e6a06640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f40e006ead0 con 0x7f40e015f710 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40e4fa3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f40e015c9d0 0x7f40e0165e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47188/0 (socket says 172.21.15.27:47188) 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40e4fa3640 1 -- 172.21.15.27:0/556043719 learned_addr learned my addr 172.21.15.27:0/556043719 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40df7fe640 1 --2- 172.21.15.27:0/556043719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015f710 0x7f40e006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40deffd640 1 -- 172.21.15.27:0/556043719 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325047343 0 0) 0x7f40e015ce20 con 0x7f40e015c9d0 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40deffd640 1 -- 172.21.15.27:0/556043719 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40c8003520 con 0x7f40e015c9d0 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40deffd640 1 -- 172.21.15.27:0/556043719 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2769516109 0 0) 0x7f40e015ec70 con 0x7f40e01556f0 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40deffd640 1 -- 172.21.15.27:0/556043719 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40e015ce20 con 0x7f40e01556f0 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40df7fe640 1 -- 172.21.15.27:0/556043719 >> v1:172.21.15.137:6789/0 conn(0x7f40e01556f0 legacy=0x7f40e00cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40df7fe640 1 -- 172.21.15.27:0/556043719 >> v1:172.21.15.27:6789/0 conn(0x7f40e015c9d0 legacy=0x7f40e0165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.478+0000 7f40df7fe640 1 -- 172.21.15.27:0/556043719 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e006e510 con 0x7f40e015f710 2024-04-20T11:00:50.479 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.479+0000 7f40df7fe640 1 --2- 172.21.15.27:0/556043719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015f710 0x7f40e006ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40d40097d0 tx=0x7f40d402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5604f423665e1b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:50.480 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.479+0000 7f40deffd640 1 -- 172.21.15.27:0/556043719 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4034070 con 0x7f40e015f710 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.479+0000 7f40deffd640 1 -- 172.21.15.27:0/556043719 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d4030880 con 0x7f40e015f710 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.479+0000 7f40deffd640 1 -- 172.21.15.27:0/556043719 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4030c50 con 0x7f40e015f710 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.479+0000 7f40e6a06640 1 -- 172.21.15.27:0/556043719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015f710 msgr2=0x7f40e006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.479+0000 7f40e6a06640 1 --2- 172.21.15.27:0/556043719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015f710 0x7f40e006ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40d40097d0 tx=0x7f40d402b9c0 comp rx=0 tx=0).stop 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.480+0000 7f40e6a06640 1 -- 172.21.15.27:0/556043719 shutdown_connections 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.480+0000 7f40e6a06640 1 --2- 172.21.15.27:0/556043719 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015f710 0x7f40e006ca40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.480+0000 7f40e6a06640 1 -- 172.21.15.27:0/556043719 >> 172.21.15.27:0/556043719 conn(0x7f40e0067ae0 msgr2=0x7f40e0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.480+0000 7f40e6a06640 1 -- 172.21.15.27:0/556043719 shutdown_connections 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.480+0000 7f40e6a06640 1 -- 172.21.15.27:0/556043719 wait complete. 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.481+0000 7f40e6a06640 1 Processor -- start 2024-04-20T11:00:50.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.481+0000 7f40e6a06640 1 -- start start 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40e6a06640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40e01556f0 0x7f40e0150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40e6a06640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015c9d0 0x7f40e01f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40e6a06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40e015f710 0x7f40e01f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40e6a06640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f40e015f1f0 con 0x7f40e01556f0 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40e6a06640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f40e015ef10 con 0x7f40e015f710 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40e6a06640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f40e00c9800 con 0x7f40e015c9d0 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40e01556f0 0x7f40e0150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40e01556f0 0x7f40e0150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35616/0 (socket says 172.21.15.27:35616) 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dffff640 1 -- 172.21.15.27:0/1966853620 learned_addr learned my addr 172.21.15.27:0/1966853620 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40e4fa3640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40e015f710 0x7f40e01f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40df7fe640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015c9d0 0x7f40e01f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:50.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dffff640 1 -- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015c9d0 msgr2=0x7f40e01f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dffff640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015c9d0 0x7f40e01f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dffff640 1 -- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40e015f710 msgr2=0x7f40e01f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dffff640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40e015f710 0x7f40e01f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dffff640 1 -- 172.21.15.27:0/1966853620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e00b3f00 con 0x7f40e01556f0 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dffff640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40e01556f0 0x7f40e0150200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f40d000c530 tx=0x7f40d000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dcff9640 1 -- 172.21.15.27:0/1966853620 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d00052d0 con 0x7f40e01556f0 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.482+0000 7f40dcff9640 1 -- 172.21.15.27:0/1966853620 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d0005490 con 0x7f40e01556f0 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.483+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e00b3c20 con 0x7f40e01556f0 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.483+0000 7f40dcff9640 1 -- 172.21.15.27:0/1966853620 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0012870 con 0x7f40e01556f0 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.483+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e015dbf0 con 0x7f40e01556f0 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.484+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e00688e0 con 0x7f40e01556f0 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.484+0000 7f40dcff9640 1 -- 172.21.15.27:0/1966853620 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40d0012a10 con 0x7f40e01556f0 2024-04-20T11:00:50.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.484+0000 7f40dcff9640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40cc06c920 0x7f40cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:50.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.484+0000 7f40dcff9640 1 -- 172.21.15.27:0/1966853620 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f40d008b820 con 0x7f40e01556f0 2024-04-20T11:00:50.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.485+0000 7f40df7fe640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40cc06c920 0x7f40cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:50.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.485+0000 7f40df7fe640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40cc06c920 0x7f40cc06ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f40d4036000 tx=0x7f40d403b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:50.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.489+0000 7f40dcff9640 1 -- 172.21.15.27:0/1966853620 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f40d0059440 con 0x7f40e01556f0 2024-04-20T11:00:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.717+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40e004fcf0 con 0x7f40e01556f0 2024-04-20T11:00:50.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.717+0000 7f40dcff9640 1 -- 172.21.15.27:0/1966853620 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+6110 (secure 0 0 0) 0x7f40d005d740 con 0x7f40e01556f0 2024-04-20T11:00:50.717 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:50.717 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:00:47.358424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:resolve","state_seq":13,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:00:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 37 2024-04-20T11:00:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40cc06c920 msgr2=0x7f40cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40cc06c920 0x7f40cc06ede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f40d4036000 tx=0x7f40d403b040 comp rx=0 tx=0).stop 2024-04-20T11:00:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40e01556f0 msgr2=0x7f40e0150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40e01556f0 0x7f40e0150200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f40d000c530 tx=0x7f40d000c9f0 comp rx=0 tx=0).stop 2024-04-20T11:00:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 shutdown_connections 2024-04-20T11:00:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f40cc06c920 0x7f40cc06ede0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40e015f710 0x7f40e01f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f40e015c9d0 0x7f40e01f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 --2- 172.21.15.27:0/1966853620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40e01556f0 0x7f40e0150200 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 >> 172.21.15.27:0/1966853620 conn(0x7f40e0067ae0 msgr2=0x7f40e0070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 shutdown_connections 2024-04-20T11:00:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:50.719+0000 7f40e6a06640 1 -- 172.21.15.27:0/1966853620 wait complete. 2024-04-20T11:00:50.731 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:00:52.732 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:52.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.835+0000 7f5e5e02c640 1 Processor -- start 2024-04-20T11:00:52.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.835+0000 7f5e5e02c640 1 -- start start 2024-04-20T11:00:52.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e5e02c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5e58159260 0x7f5e58164b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:52.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e5e02c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e580c7cd0 con 0x7f5e58153410 2024-04-20T11:00:52.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e5e02c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e580c7fb0 con 0x7f5e581537f0 2024-04-20T11:00:52.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e5e02c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f5e5806cad0 con 0x7f5e58159260 2024-04-20T11:00:52.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e577fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5e58153410 0x7f5e5815d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47204/0 (socket says 172.21.15.27:47204) 2024-04-20T11:00:52.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e577fe640 1 -- 172.21.15.27:0/2589014358 learned_addr learned my addr 172.21.15.27:0/2589014358 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:52.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e57fff640 1 --2- 172.21.15.27:0/2589014358 >> v2:172.21.15.137:3300/0 conn(0x7f5e58159260 0x7f5e58164b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e567fc640 1 -- 172.21.15.27:0/2589014358 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61774604 0 0) 0x7f5e580c7cd0 con 0x7f5e58153410 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e567fc640 1 -- 172.21.15.27:0/2589014358 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e40003520 con 0x7f5e58153410 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.836+0000 7f5e567fc640 1 -- 172.21.15.27:0/2589014358 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1589390741 0 0) 0x7f5e580c7fb0 con 0x7f5e581537f0 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.837+0000 7f5e567fc640 1 -- 172.21.15.27:0/2589014358 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e580c7cd0 con 0x7f5e581537f0 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.837+0000 7f5e567fc640 1 -- 172.21.15.27:0/2589014358 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 543340690 0 0) 0x7f5e40003520 con 0x7f5e58153410 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.837+0000 7f5e57fff640 1 -- 172.21.15.27:0/2589014358 >> v1:172.21.15.137:6789/0 conn(0x7f5e581537f0 legacy=0x7f5e580c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.837+0000 7f5e57fff640 1 -- 172.21.15.27:0/2589014358 >> v1:172.21.15.27:6789/0 conn(0x7f5e58153410 legacy=0x7f5e5815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.837+0000 7f5e57fff640 1 -- 172.21.15.27:0/2589014358 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e580c8290 con 0x7f5e58159260 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.837+0000 7f5e57fff640 1 --2- 172.21.15.27:0/2589014358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e58159260 0x7f5e58164b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009d90 tx=0x7f5e5002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e981f9e9061c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e567fc640 1 -- 172.21.15.27:0/2589014358 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50039070 con 0x7f5e58159260 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e567fc640 1 -- 172.21.15.27:0/2589014358 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5002c9d0 con 0x7f5e58159260 2024-04-20T11:00:52.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e567fc640 1 -- 172.21.15.27:0/2589014358 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48001670 con 0x7f5e58159260 2024-04-20T11:00:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e5e02c640 1 -- 172.21.15.27:0/2589014358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e58159260 msgr2=0x7f5e58164b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e5e02c640 1 --2- 172.21.15.27:0/2589014358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e58159260 0x7f5e58164b90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009d90 tx=0x7f5e5002bea0 comp rx=0 tx=0).stop 2024-04-20T11:00:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e5e02c640 1 -- 172.21.15.27:0/2589014358 shutdown_connections 2024-04-20T11:00:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e5e02c640 1 --2- 172.21.15.27:0/2589014358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e58159260 0x7f5e58164b90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e5e02c640 1 -- 172.21.15.27:0/2589014358 >> 172.21.15.27:0/2589014358 conn(0x7f5e580ccbd0 msgr2=0x7f5e580ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e5e02c640 1 -- 172.21.15.27:0/2589014358 shutdown_connections 2024-04-20T11:00:52.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.838+0000 7f5e5e02c640 1 -- 172.21.15.27:0/2589014358 wait complete. 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.839+0000 7f5e5e02c640 1 Processor -- start 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.839+0000 7f5e5e02c640 1 -- start start 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e5e02c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e58153410 0x7f5e581f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e5e02c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e581537f0 0x7f5e580c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e5e02c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58159260 0x7f5e580cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e5e02c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5e580c75d0 con 0x7f5e58159260 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e5e02c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5815cc50 con 0x7f5e581537f0 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e5e02c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5e580c6e40 con 0x7f5e58153410 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58159260 0x7f5e580cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:52.843 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58159260 0x7f5e580cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35622/0 (socket says 172.21.15.27:35622) 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e57fff640 1 -- 172.21.15.27:0/3506725825 learned_addr learned my addr 172.21.15.27:0/3506725825 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e56ffd640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e581537f0 0x7f5e580c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.840+0000 7f5e577fe640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e58153410 0x7f5e581f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e57fff640 1 -- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e58153410 msgr2=0x7f5e581f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e57fff640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e58153410 0x7f5e581f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e57fff640 1 -- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e581537f0 msgr2=0x7f5e580c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e57fff640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e581537f0 0x7f5e580c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e57fff640 1 -- 172.21.15.27:0/3506725825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e581f2d70 con 0x7f5e58159260 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e57fff640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58159260 0x7f5e580cb230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009eb0 tx=0x7f5e50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e3ffff640 1 -- 172.21.15.27:0/3506725825 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50039050 con 0x7f5e58159260 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e3ffff640 1 -- 172.21.15.27:0/3506725825 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e50045070 con 0x7f5e58159260 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e580caa70 con 0x7f5e58159260 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e3ffff640 1 -- 172.21.15.27:0/3506725825 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50009450 con 0x7f5e58159260 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.841+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e58151920 con 0x7f5e58159260 2024-04-20T11:00:52.844 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.842+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e5804fcf0 con 0x7f5e58159260 2024-04-20T11:00:52.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.842+0000 7f5e3ffff640 1 -- 172.21.15.27:0/3506725825 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e50040690 con 0x7f5e58159260 2024-04-20T11:00:52.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.843+0000 7f5e3ffff640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5e4406c8f0 0x7f5e4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:52.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.843+0000 7f5e3ffff640 1 -- 172.21.15.27:0/3506725825 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f5e500ae5b0 con 0x7f5e58159260 2024-04-20T11:00:52.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.843+0000 7f5e577fe640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5e4406c8f0 0x7f5e4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:52.845 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.843+0000 7f5e577fe640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5e4406c8f0 0x7f5e4406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e48000fd0 tx=0x7f5e48001150 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:52.847 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:52.847+0000 7f5e3ffff640 1 -- 172.21.15.27:0/3506725825 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5e5007c130 con 0x7f5e58159260 2024-04-20T11:00:53.077 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.076+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e581468a0 con 0x7f5e58159260 2024-04-20T11:00:53.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.076+0000 7f5e3ffff640 1 -- 172.21.15.27:0/3506725825 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6112 (secure 0 0 0) 0x7f5e50080430 con 0x7f5e58159260 2024-04-20T11:00:53.078 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:53.078 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:00:52.522476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4278,4293,4894,5029]},"id":1}]} 2024-04-20T11:00:53.078 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2024-04-20T11:00:53.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5e4406c8f0 msgr2=0x7f5e4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5e4406c8f0 0x7f5e4406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e48000fd0 tx=0x7f5e48001150 comp rx=0 tx=0).stop 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58159260 msgr2=0x7f5e580cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58159260 0x7f5e580cb230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009eb0 tx=0x7f5e50004a40 comp rx=0 tx=0).stop 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 shutdown_connections 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5e4406c8f0 0x7f5e4406edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e58159260 0x7f5e580cb230 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e581537f0 0x7f5e580c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 --2- 172.21.15.27:0/3506725825 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5e58153410 0x7f5e581f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 >> 172.21.15.27:0/3506725825 conn(0x7f5e580ccbd0 msgr2=0x7f5e58156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 shutdown_connections 2024-04-20T11:00:53.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:53.079+0000 7f5e5e02c640 1 -- 172.21.15.27:0/3506725825 wait complete. 2024-04-20T11:00:53.090 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:00:55.091 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:55.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.212+0000 7ffa43f52640 1 Processor -- start 2024-04-20T11:00:55.212 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.213+0000 7ffa43f52640 1 -- start start 2024-04-20T11:00:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.213+0000 7ffa43f52640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa3c155950 0x7ffa3c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.213+0000 7ffa43f52640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 0x7ffa3c0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.213+0000 7ffa43f52640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa3c15b9c0 con 0x7ffa3c0cc330 2024-04-20T11:00:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.213+0000 7ffa43f52640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ffa3c15bca0 con 0x7ffa3c155950 2024-04-20T11:00:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.213+0000 7ffa43f52640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3c15bf80 con 0x7ffa3c0b5c00 2024-04-20T11:00:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.213+0000 7ffa41cee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa3c155950 0x7ffa3c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.214+0000 7ffa414ed640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ffa3c0cc330 0x7ffa3c0b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54716/0 (socket says 172.21.15.27:54716) 2024-04-20T11:00:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.214+0000 7ffa414ed640 1 -- 172.21.15.27:0/934398488 learned_addr learned my addr 172.21.15.27:0/934398488 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.214+0000 7ffa41cee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa3c155950 0x7ffa3c0cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43218/0 (socket says 172.21.15.27:43218) 2024-04-20T11:00:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.214+0000 7ffa424ef640 1 --2- 172.21.15.27:0/934398488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 0x7ffa3c0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.214+0000 7ffa41cee640 1 -- 172.21.15.27:0/934398488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 msgr2=0x7ffa3c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.214+0000 7ffa41cee640 1 --2- 172.21.15.27:0/934398488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 0x7ffa3c0d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.215 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.214+0000 7ffa41cee640 1 -- 172.21.15.27:0/934398488 >> v1:172.21.15.27:6789/0 conn(0x7ffa3c0cc330 legacy=0x7ffa3c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:55.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.214+0000 7ffa41cee640 1 -- 172.21.15.27:0/934398488 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c15c260 con 0x7ffa3c155950 2024-04-20T11:00:55.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa41cee640 1 --2- 172.21.15.27:0/934398488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 0x7ffa3c0cbdf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffa340097a0 tx=0x7ffa3402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b033c4c56cbe728 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa40cec640 1 -- 172.21.15.27:0/934398488 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34034070 con 0x7ffa3c155950 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa40cec640 1 -- 172.21.15.27:0/934398488 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa34030610 con 0x7ffa3c155950 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa40cec640 1 -- 172.21.15.27:0/934398488 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340309e0 con 0x7ffa3c155950 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa43f52640 1 -- 172.21.15.27:0/934398488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 msgr2=0x7ffa3c0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa43f52640 1 --2- 172.21.15.27:0/934398488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 0x7ffa3c0cbdf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffa340097a0 tx=0x7ffa3402b960 comp rx=0 tx=0).stop 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa43f52640 1 -- 172.21.15.27:0/934398488 shutdown_connections 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa43f52640 1 --2- 172.21.15.27:0/934398488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 0x7ffa3c0d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa43f52640 1 --2- 172.21.15.27:0/934398488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 0x7ffa3c0cbdf0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa43f52640 1 -- 172.21.15.27:0/934398488 >> 172.21.15.27:0/934398488 conn(0x7ffa3c0676c0 msgr2=0x7ffa3c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.215+0000 7ffa43f52640 1 -- 172.21.15.27:0/934398488 shutdown_connections 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.216+0000 7ffa43f52640 1 -- 172.21.15.27:0/934398488 wait complete. 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa43f52640 1 Processor -- start 2024-04-20T11:00:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa43f52640 1 -- start start 2024-04-20T11:00:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa43f52640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 0x7ffa3c1f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa43f52640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa3c0cc330 0x7ffa3c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa43f52640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 0x7ffa3c0c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa43f52640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c15fd10 con 0x7ffa3c155950 2024-04-20T11:00:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa43f52640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c15fb90 con 0x7ffa3c0cc330 2024-04-20T11:00:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa43f52640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3c155d30 con 0x7ffa3c0b5c00 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa424ef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 0x7ffa3c0c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa424ef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 0x7ffa3c0c5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43224/0 (socket says 172.21.15.27:43224) 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.217+0000 7ffa424ef640 1 -- 172.21.15.27:0/642017768 learned_addr learned my addr 172.21.15.27:0/642017768 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa41cee640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 0x7ffa3c1f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa414ed640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa3c0cc330 0x7ffa3c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa424ef640 1 -- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 msgr2=0x7ffa3c1f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa424ef640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 0x7ffa3c1f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa424ef640 1 -- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa3c0cc330 msgr2=0x7ffa3c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa424ef640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa3c0cc330 0x7ffa3c15df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa424ef640 1 -- 172.21.15.27:0/642017768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c1f5b20 con 0x7ffa3c155950 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa41cee640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 0x7ffa3c1f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa414ed640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa3c0cc330 0x7ffa3c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa424ef640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 0x7ffa3c0c5ba0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c007dc0 tx=0x7ffa2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa3a7fc640 1 -- 172.21.15.27:0/642017768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c014420 con 0x7ffa3c155950 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa3a7fc640 1 -- 172.21.15.27:0/642017768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa2c019070 con 0x7ffa3c155950 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c0b3ad0 con 0x7ffa3c155950 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa3a7fc640 1 -- 172.21.15.27:0/642017768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c014820 con 0x7ffa3c155950 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.218+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1f19c0 con 0x7ffa3c155950 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.219+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3c04fcf0 con 0x7ffa3c155950 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.220+0000 7ffa3a7fc640 1 -- 172.21.15.27:0/642017768 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa2c0149c0 con 0x7ffa3c155950 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.220+0000 7ffa3a7fc640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ffa2806c920 0x7ffa2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.220+0000 7ffa3a7fc640 1 -- 172.21.15.27:0/642017768 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7ffa2c08c530 con 0x7ffa3c155950 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.220+0000 7ffa41cee640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ffa2806c920 0x7ffa2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.221+0000 7ffa41cee640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ffa2806c920 0x7ffa2806ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa34036000 tx=0x7ffa34008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.225+0000 7ffa3a7fc640 1 -- 172.21.15.27:0/642017768 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffa2c05a790 con 0x7ffa3c155950 2024-04-20T11:00:55.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.448+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa3c15f3f0 con 0x7ffa3c155950 2024-04-20T11:00:55.448 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.448+0000 7ffa3a7fc640 1 -- 172.21.15.27:0/642017768 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6112 (secure 0 0 0) 0x7ffa2c05a090 con 0x7ffa3c155950 2024-04-20T11:00:55.449 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:55.449 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:00:52.522476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4278,4293,4894,5029]},"id":1}]} 2024-04-20T11:00:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2024-04-20T11:00:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ffa2806c920 msgr2=0x7ffa2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ffa2806c920 0x7ffa2806ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa34036000 tx=0x7ffa34008030 comp rx=0 tx=0).stop 2024-04-20T11:00:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 msgr2=0x7ffa3c0c5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 0x7ffa3c0c5ba0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c007dc0 tx=0x7ffa2c00e4b0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 shutdown_connections 2024-04-20T11:00:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ffa2806c920 0x7ffa2806ede0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa3c155950 0x7ffa3c0c5ba0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.451 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa3c0cc330 0x7ffa3c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 --2- 172.21.15.27:0/642017768 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffa3c0b5c00 0x7ffa3c1f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 >> 172.21.15.27:0/642017768 conn(0x7ffa3c0676c0 msgr2=0x7ffa3c153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 shutdown_connections 2024-04-20T11:00:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:55.450+0000 7ffa43f52640 1 -- 172.21.15.27:0/642017768 wait complete. 2024-04-20T11:00:55.462 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 38, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 38, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4278": { "addr": "172.21.15.27:6836/4195543922", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6833", "nonce": 4195543922, "type": "v2" }, { "addr": "172.21.15.27:6836", "nonce": 4195543922, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4278, "incarnation": 10, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4894": { "addr": "172.21.15.137:6838/2896327439", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6836", "nonce": 2896327439, "type": "v2" }, { "addr": "172.21.15.137:6838", "nonce": 2896327439, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4894, "incarnation": 24, "join_fscid": -1, "name": "b", "rank": 4, "state": "up:active", "state_seq": 17 }, "gid_5029": { "addr": "172.21.15.137:6835/4235742413", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6834", "nonce": 4235742413, "type": "v2" }, { "addr": "172.21.15.137:6835", "nonce": 4235742413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5029, "incarnation": 30, "join_fscid": -1, "name": "f", "rank": 3, "state": "up:active", "state_seq": 22 }, "gid_5143": { "addr": "172.21.15.27:6835/2020174232", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6832", "nonce": 2020174232, "type": "v2" }, { "addr": "172.21.15.27:6835", "nonce": 2020174232, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5143, "incarnation": 36, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:reconnect", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 26, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T11:00:52.522476+0000", "qdb_cluster": [ 4278, 4293, 4894, 5029 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 4278, "mds_2": 5143, "mds_3": 5029, "mds_4": 4894 } } } ], "standbys": [] } 2024-04-20T11:00:57.463 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:57.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.578+0000 7fe9944e0640 1 Processor -- start 2024-04-20T11:00:57.578 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.578+0000 7fe9944e0640 1 -- start start 2024-04-20T11:00:57.579 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.579+0000 7fe9944e0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe98c155950 0x7fe98c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.579+0000 7fe9944e0640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe98c0cc330 0x7fe98c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.579+0000 7fe9944e0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe98c15b9c0 con 0x7fe98c0b5a30 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.579+0000 7fe9944e0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe98c15bca0 con 0x7fe98c155950 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.579+0000 7fe9944e0640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fe98c15bf80 con 0x7fe98c0cc330 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.579+0000 7fe99227c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe98c155950 0x7fe98c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.579+0000 7fe99227c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe98c155950 0x7fe98c0cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43244/0 (socket says 172.21.15.27:43244) 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.579+0000 7fe99227c640 1 -- 172.21.15.27:0/4254497121 learned_addr learned my addr 172.21.15.27:0/4254497121 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.579+0000 7fe991a7b640 1 --2- 172.21.15.27:0/4254497121 >> v2:172.21.15.137:3300/0 conn(0x7fe98c0cc330 0x7fe98c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.580+0000 7fe99227c640 1 -- 172.21.15.27:0/4254497121 >> v2:172.21.15.137:3300/0 conn(0x7fe98c0cc330 msgr2=0x7fe98c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.580+0000 7fe99227c640 1 --2- 172.21.15.27:0/4254497121 >> v2:172.21.15.137:3300/0 conn(0x7fe98c0cc330 0x7fe98c0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.580+0000 7fe99227c640 1 -- 172.21.15.27:0/4254497121 >> v1:172.21.15.137:6789/0 conn(0x7fe98c0b5a30 legacy=0x7fe98c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:57.580 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.580+0000 7fe99227c640 1 -- 172.21.15.27:0/4254497121 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c15c260 con 0x7fe98c155950 2024-04-20T11:00:57.581 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe99227c640 1 --2- 172.21.15.27:0/4254497121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 0x7fe98c0cbdf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0b44e0 tx=0x7fe98402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e160b28ba154b824 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe99127a640 1 -- 172.21.15.27:0/4254497121 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98402f850 con 0x7fe98c155950 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe99127a640 1 -- 172.21.15.27:0/4254497121 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe98402f9f0 con 0x7fe98c155950 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe99127a640 1 -- 172.21.15.27:0/4254497121 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98402fdc0 con 0x7fe98c155950 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe9944e0640 1 -- 172.21.15.27:0/4254497121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 msgr2=0x7fe98c0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe9944e0640 1 --2- 172.21.15.27:0/4254497121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 0x7fe98c0cbdf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0b44e0 tx=0x7fe98402ba30 comp rx=0 tx=0).stop 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe9944e0640 1 -- 172.21.15.27:0/4254497121 shutdown_connections 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe9944e0640 1 --2- 172.21.15.27:0/4254497121 >> v2:172.21.15.137:3300/0 conn(0x7fe98c0cc330 0x7fe98c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe9944e0640 1 --2- 172.21.15.27:0/4254497121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 0x7fe98c0cbdf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.581+0000 7fe9944e0640 1 -- 172.21.15.27:0/4254497121 >> 172.21.15.27:0/4254497121 conn(0x7fe98c0676c0 msgr2=0x7fe98c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.582+0000 7fe9944e0640 1 -- 172.21.15.27:0/4254497121 shutdown_connections 2024-04-20T11:00:57.582 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.582+0000 7fe9944e0640 1 -- 172.21.15.27:0/4254497121 wait complete. 2024-04-20T11:00:57.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.583+0000 7fe9944e0640 1 Processor -- start 2024-04-20T11:00:57.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.583+0000 7fe9944e0640 1 -- start start 2024-04-20T11:00:57.584 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.583+0000 7fe9944e0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98c0b5a30 0x7fe98c153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.583+0000 7fe9944e0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c0cc330 0x7fe98c1508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.583+0000 7fe9944e0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 0x7fe98c1f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.583+0000 7fe9944e0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c15fb90 con 0x7fe98c155950 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.583+0000 7fe9944e0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c0b5e40 con 0x7fe98c0cc330 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe9944e0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe98c15fd10 con 0x7fe98c0b5a30 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe992a7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 0x7fe98c1f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe992a7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 0x7fe98c1f4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43246/0 (socket says 172.21.15.27:43246) 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe992a7d640 1 -- 172.21.15.27:0/3343559960 learned_addr learned my addr 172.21.15.27:0/3343559960 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe991a7b640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c0cc330 0x7fe98c1508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe99227c640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98c0b5a30 0x7fe98c153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe992a7d640 1 -- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98c0b5a30 msgr2=0x7fe98c153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe992a7d640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98c0b5a30 0x7fe98c153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe992a7d640 1 -- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c0cc330 msgr2=0x7fe98c1508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe992a7d640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c0cc330 0x7fe98c1508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.585 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe992a7d640 1 -- 172.21.15.27:0/3343559960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c1f1760 con 0x7fe98c155950 2024-04-20T11:00:57.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe99227c640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98c0b5a30 0x7fe98c153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:00:57.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe992a7d640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 0x7fe98c1f4c70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe98800c730 tx=0x7fe98800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:57.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe982ffd640 1 -- 172.21.15.27:0/3343559960 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988003b70 con 0x7fe98c155950 2024-04-20T11:00:57.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe982ffd640 1 -- 172.21.15.27:0/3343559960 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe988003d30 con 0x7fe98c155950 2024-04-20T11:00:57.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c1f1a40 con 0x7fe98c155950 2024-04-20T11:00:57.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe982ffd640 1 -- 172.21.15.27:0/3343559960 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988005310 con 0x7fe98c155950 2024-04-20T11:00:57.586 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.584+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c1f2000 con 0x7fe98c155950 2024-04-20T11:00:57.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.586+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98c04fcf0 con 0x7fe98c155950 2024-04-20T11:00:57.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.586+0000 7fe982ffd640 1 -- 172.21.15.27:0/3343559960 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9880056c0 con 0x7fe98c155950 2024-04-20T11:00:57.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.586+0000 7fe982ffd640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe97006c8f0 0x7fe97006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:57.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.587+0000 7fe982ffd640 1 -- 172.21.15.27:0/3343559960 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7fe98808bdd0 con 0x7fe98c155950 2024-04-20T11:00:57.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.587+0000 7fe99227c640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe97006c8f0 0x7fe97006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:57.587 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.587+0000 7fe99227c640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe97006c8f0 0x7fe97006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe984036000 tx=0x7fe984008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:57.590 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.591+0000 7fe982ffd640 1 -- 172.21.15.27:0/3343559960 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe988059fa0 con 0x7fe98c155950 2024-04-20T11:00:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.816+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe98c0cd0a0 con 0x7fe98c155950 2024-04-20T11:00:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.817+0000 7fe982ffd640 1 -- 172.21.15.27:0/3343559960 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6112 (secure 0 0 0) 0x7fe98808d020 con 0x7fe98c155950 2024-04-20T11:00:57.817 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:00:57.817 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:00:52.522476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4278,4293,4894,5029]},"id":1}]} 2024-04-20T11:00:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2024-04-20T11:00:57.818 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.818+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe97006c8f0 msgr2=0x7fe97006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.818+0000 7fe9944e0640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe97006c8f0 0x7fe97006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe984036000 tx=0x7fe984008030 comp rx=0 tx=0).stop 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.818+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 msgr2=0x7fe98c1f4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.818+0000 7fe9944e0640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 0x7fe98c1f4c70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe98800c730 tx=0x7fe98800cbf0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.819+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 shutdown_connections 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.819+0000 7fe9944e0640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe97006c8f0 0x7fe97006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.819+0000 7fe9944e0640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe98c155950 0x7fe98c1f4c70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.819+0000 7fe9944e0640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe98c0cc330 0x7fe98c1508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.819+0000 7fe9944e0640 1 --2- 172.21.15.27:0/3343559960 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe98c0b5a30 0x7fe98c153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.819+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 >> 172.21.15.27:0/3343559960 conn(0x7fe98c0676c0 msgr2=0x7fe98c0cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.819+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 shutdown_connections 2024-04-20T11:00:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:57.819+0000 7fe9944e0640 1 -- 172.21.15.27:0/3343559960 wait complete. 2024-04-20T11:00:57.829 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:00:59.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:00:59.944 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.945+0000 7fbd17123640 1 Processor -- start 2024-04-20T11:00:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.945+0000 7fbd17123640 1 -- start start 2024-04-20T11:00:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd17123640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10151f40 0x7fbd1006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd17123640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fbd10152320 0x7fbd1006fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd17123640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd100c9970 con 0x7fbd1015e990 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd17123640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fbd10070210 con 0x7fbd10152320 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd17123640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbd1015b750 con 0x7fbd10151f40 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd156c0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbd1015e990 0x7fbd1006ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54726/0 (socket says 172.21.15.27:54726) 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd156c0640 1 -- 172.21.15.27:0/3916659066 learned_addr learned my addr 172.21.15.27:0/3916659066 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd14ebf640 1 --2- 172.21.15.27:0/3916659066 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10151f40 0x7fbd1006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd0ffff640 1 --2- 172.21.15.27:0/3916659066 >> v2:172.21.15.137:3300/0 conn(0x7fbd10152320 0x7fbd1006fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd0f7fe640 1 -- 172.21.15.27:0/3916659066 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 931891484 0 0) 0x7fbd100c9970 con 0x7fbd1015e990 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd0f7fe640 1 -- 172.21.15.27:0/3916659066 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd00003520 con 0x7fbd1015e990 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd0ffff640 1 -- 172.21.15.27:0/3916659066 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10151f40 msgr2=0x7fbd1006f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd0ffff640 1 --2- 172.21.15.27:0/3916659066 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10151f40 0x7fbd1006f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd0ffff640 1 -- 172.21.15.27:0/3916659066 >> v1:172.21.15.27:6789/0 conn(0x7fbd1015e990 legacy=0x7fbd1006ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd0ffff640 1 -- 172.21.15.27:0/3916659066 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1015ba30 con 0x7fbd10152320 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.946+0000 7fbd14ebf640 1 --2- 172.21.15.27:0/3916659066 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10151f40 0x7fbd1006f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:00:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.947+0000 7fbd0ffff640 1 --2- 172.21.15.27:0/3916659066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd10152320 0x7fbd1006fb00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0097a0 tx=0x7fbcfc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e855b38583a6f58b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.947+0000 7fbd0f7fe640 1 -- 172.21.15.27:0/3916659066 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc034070 con 0x7fbd10152320 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.947+0000 7fbd0f7fe640 1 -- 172.21.15.27:0/3916659066 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcfc030610 con 0x7fbd10152320 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.947+0000 7fbd0f7fe640 1 -- 172.21.15.27:0/3916659066 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc0309c0 con 0x7fbd10152320 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.948+0000 7fbd17123640 1 -- 172.21.15.27:0/3916659066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd10152320 msgr2=0x7fbd1006fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.948+0000 7fbd17123640 1 --2- 172.21.15.27:0/3916659066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd10152320 0x7fbd1006fb00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0097a0 tx=0x7fbcfc02b960 comp rx=0 tx=0).stop 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.948+0000 7fbd17123640 1 -- 172.21.15.27:0/3916659066 shutdown_connections 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.948+0000 7fbd17123640 1 --2- 172.21.15.27:0/3916659066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd10152320 0x7fbd1006fb00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.948+0000 7fbd17123640 1 --2- 172.21.15.27:0/3916659066 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10151f40 0x7fbd1006f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.948+0000 7fbd17123640 1 -- 172.21.15.27:0/3916659066 >> 172.21.15.27:0/3916659066 conn(0x7fbd100cee90 msgr2=0x7fbd100cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.948+0000 7fbd17123640 1 -- 172.21.15.27:0/3916659066 shutdown_connections 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.948+0000 7fbd17123640 1 -- 172.21.15.27:0/3916659066 wait complete. 2024-04-20T11:00:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.949+0000 7fbd17123640 1 Processor -- start 2024-04-20T11:00:59.950 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.949+0000 7fbd17123640 1 -- start start 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd17123640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd10151f40 0x7fbd100c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd17123640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10152320 0x7fbd100c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd17123640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd1015e990 0x7fbd1014b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd17123640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1015c2d0 con 0x7fbd10151f40 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd17123640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1015bff0 con 0x7fbd1015e990 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd17123640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbd1006f0a0 con 0x7fbd10152320 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd14ebf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd10151f40 0x7fbd100c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd0ffff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10152320 0x7fbd100c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd0ffff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10152320 0x7fbd100c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:37452/0 (socket says 172.21.15.27:37452) 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd14ebf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd10151f40 0x7fbd100c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43252/0 (socket says 172.21.15.27:43252) 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd0ffff640 1 -- 172.21.15.27:0/323141706 learned_addr learned my addr 172.21.15.27:0/323141706 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd156c0640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd1015e990 0x7fbd1014b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd14ebf640 1 -- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10152320 msgr2=0x7fbd100c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd14ebf640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10152320 0x7fbd100c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd14ebf640 1 -- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd1015e990 msgr2=0x7fbd1014b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd14ebf640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd1015e990 0x7fbd1014b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.950+0000 7fbd14ebf640 1 -- 172.21.15.27:0/323141706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1015a500 con 0x7fbd10151f40 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.951+0000 7fbd0ffff640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10152320 0x7fbd100c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.951+0000 7fbd14ebf640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd10151f40 0x7fbd100c5a30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400e900 tx=0x7fbd0400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.951+0000 7fbd0d7fa640 1 -- 172.21.15.27:0/323141706 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04017070 con 0x7fbd10151f40 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.951+0000 7fbd0d7fa640 1 -- 172.21.15.27:0/323141706 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd0400ca00 con 0x7fbd10151f40 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.951+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1015a220 con 0x7fbd10151f40 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.951+0000 7fbd0d7fa640 1 -- 172.21.15.27:0/323141706 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400cdf0 con 0x7fbd10151f40 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.951+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd101504f0 con 0x7fbd10151f40 2024-04-20T11:00:59.953 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.952+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1014c7d0 con 0x7fbd10151f40 2024-04-20T11:00:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.953+0000 7fbd0d7fa640 1 -- 172.21.15.27:0/323141706 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd0401a070 con 0x7fbd10151f40 2024-04-20T11:00:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.953+0000 7fbd0d7fa640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcf406c920 0x7fbcf406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:00:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.953+0000 7fbd0d7fa640 1 -- 172.21.15.27:0/323141706 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7fbd0408bb40 con 0x7fbd10151f40 2024-04-20T11:00:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.953+0000 7fbd0ffff640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcf406c920 0x7fbcf406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:00:59.954 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.954+0000 7fbd0ffff640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcf406c920 0x7fbcf406ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbd100c9de0 tx=0x7fbcfc03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:00:59.957 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:00:59.957+0000 7fbd0d7fa640 1 -- 172.21.15.27:0/323141706 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd1014c7d0 con 0x7fbd10151f40 2024-04-20T11:01:00.181 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.181+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd1004fcf0 con 0x7fbd10151f40 2024-04-20T11:01:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.182+0000 7fbd0d7fa640 1 -- 172.21.15.27:0/323141706 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+6112 (secure 0 0 0) 0x7fbd040596d0 con 0x7fbd10151f40 2024-04-20T11:01:00.182 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:00.182 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:00:52.522476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4278,4293,4894,5029]},"id":1}]} 2024-04-20T11:01:00.183 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2024-04-20T11:01:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcf406c920 msgr2=0x7fbcf406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcf406c920 0x7fbcf406ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbd100c9de0 tx=0x7fbcfc03b040 comp rx=0 tx=0).stop 2024-04-20T11:01:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd10151f40 msgr2=0x7fbd100c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd10151f40 0x7fbd100c5a30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400e900 tx=0x7fbd0400edc0 comp rx=0 tx=0).stop 2024-04-20T11:01:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 shutdown_connections 2024-04-20T11:01:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbcf406c920 0x7fbcf406ede0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd1015e990 0x7fbd1014b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbd10152320 0x7fbd100c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:00.184 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 --2- 172.21.15.27:0/323141706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd10151f40 0x7fbd100c5a30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 >> 172.21.15.27:0/323141706 conn(0x7fbd100cee90 msgr2=0x7fbd10070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 shutdown_connections 2024-04-20T11:01:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:00.184+0000 7fbd17123640 1 -- 172.21.15.27:0/323141706 wait complete. 2024-04-20T11:01:00.195 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:01:02.196 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:02.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.331+0000 7f0d03cce640 1 Processor -- start 2024-04-20T11:01:02.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.331+0000 7f0d03cce640 1 -- start start 2024-04-20T11:01:02.332 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d03cce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0cfc1522f0 0x7f0cfc0723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d03cce640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 0x7f0cfc158880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d03cce640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cfc072a40 con 0x7f0cfc158dc0 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d03cce640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0cfc1583f0 con 0x7f0cfc1522f0 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d03cce640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f0cfc072f40 con 0x7f0cfc15c150 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d01a6a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0cfc1522f0 0x7f0cfc0723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d0226b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0cfc158dc0 0x7f0cfc06f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54740/0 (socket says 172.21.15.27:54740) 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d01a6a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0cfc1522f0 0x7f0cfc0723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43254/0 (socket says 172.21.15.27:43254) 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d0226b640 1 -- 172.21.15.27:0/3372395009 learned_addr learned my addr 172.21.15.27:0/3372395009 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d01269640 1 --2- 172.21.15.27:0/3372395009 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 0x7f0cfc158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d01a6a640 1 -- 172.21.15.27:0/3372395009 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 msgr2=0x7f0cfc158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d01a6a640 1 --2- 172.21.15.27:0/3372395009 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 0x7f0cfc158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d01a6a640 1 -- 172.21.15.27:0/3372395009 >> v1:172.21.15.27:6789/0 conn(0x7f0cfc158dc0 legacy=0x7f0cfc06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:02.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d01a6a640 1 -- 172.21.15.27:0/3372395009 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc0731a0 con 0x7f0cfc1522f0 2024-04-20T11:01:02.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.332+0000 7f0d00a68640 1 -- 172.21.15.27:0/3372395009 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2163713279 0 0) 0x7f0cfc072a40 con 0x7f0cfc158dc0 2024-04-20T11:01:02.334 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.333+0000 7f0d01a6a640 1 --2- 172.21.15.27:0/3372395009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc1522f0 0x7f0cfc0723b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80097a0 tx=0x7f0cf802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146f0999ba02eb21 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.333+0000 7f0d00a68640 1 -- 172.21.15.27:0/3372395009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8035020 con 0x7f0cfc1522f0 2024-04-20T11:01:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.333+0000 7f0d00a68640 1 -- 172.21.15.27:0/3372395009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8031830 con 0x7f0cfc1522f0 2024-04-20T11:01:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.333+0000 7f0d00a68640 1 -- 172.21.15.27:0/3372395009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8031c00 con 0x7f0cfc1522f0 2024-04-20T11:01:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.334+0000 7f0d03cce640 1 -- 172.21.15.27:0/3372395009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc1522f0 msgr2=0x7f0cfc0723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.334+0000 7f0d03cce640 1 --2- 172.21.15.27:0/3372395009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc1522f0 0x7f0cfc0723b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80097a0 tx=0x7f0cf802b8d0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.334+0000 7f0d03cce640 1 -- 172.21.15.27:0/3372395009 shutdown_connections 2024-04-20T11:01:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.334+0000 7f0d03cce640 1 --2- 172.21.15.27:0/3372395009 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 0x7f0cfc158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.334+0000 7f0d03cce640 1 --2- 172.21.15.27:0/3372395009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc1522f0 0x7f0cfc0723b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.334+0000 7f0d03cce640 1 -- 172.21.15.27:0/3372395009 >> 172.21.15.27:0/3372395009 conn(0x7f0cfc0cd9c0 msgr2=0x7f0cfc0cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.334+0000 7f0d03cce640 1 -- 172.21.15.27:0/3372395009 shutdown_connections 2024-04-20T11:01:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.334+0000 7f0d03cce640 1 -- 172.21.15.27:0/3372395009 wait complete. 2024-04-20T11:01:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.335+0000 7f0d03cce640 1 Processor -- start 2024-04-20T11:01:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.335+0000 7f0d03cce640 1 -- start start 2024-04-20T11:01:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d03cce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc1522f0 0x7f0cfc1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d03cce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc158dc0 0x7f0cfc145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d03cce640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 0x7f0cfc1eab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d03cce640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc15c580 con 0x7f0cfc158dc0 2024-04-20T11:01:02.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d03cce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc0c49e0 con 0x7f0cfc1522f0 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d03cce640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f0cfc1526d0 con 0x7f0cfc15c150 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01269640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc158dc0 0x7f0cfc145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01269640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc158dc0 0x7f0cfc145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43268/0 (socket says 172.21.15.27:43268) 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01269640 1 -- 172.21.15.27:0/22091486 learned_addr learned my addr 172.21.15.27:0/22091486 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d0226b640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 0x7f0cfc1eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01a6a640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc1522f0 0x7f0cfc1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01269640 1 -- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 msgr2=0x7f0cfc1eab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01269640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 0x7f0cfc1eab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01269640 1 -- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc1522f0 msgr2=0x7f0cfc1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01269640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc1522f0 0x7f0cfc1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01269640 1 -- 172.21.15.27:0/22091486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc158020 con 0x7f0cfc158dc0 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.336+0000 7f0d01a6a640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc1522f0 0x7f0cfc1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.337+0000 7f0d01269640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc158dc0 0x7f0cfc145180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e8a0 tx=0x7f0cec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.337+0000 7f0cf27fc640 1 -- 172.21.15.27:0/22091486 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec017070 con 0x7f0cfc158dc0 2024-04-20T11:01:02.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.337+0000 7f0cf27fc640 1 -- 172.21.15.27:0/22091486 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cec00c940 con 0x7f0cfc158dc0 2024-04-20T11:01:02.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.337+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc1f14a0 con 0x7f0cfc158dc0 2024-04-20T11:01:02.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.337+0000 7f0cf27fc640 1 -- 172.21.15.27:0/22091486 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00cd30 con 0x7f0cfc158dc0 2024-04-20T11:01:02.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.337+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1f1a60 con 0x7f0cfc158dc0 2024-04-20T11:01:02.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.338+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cfc04fcf0 con 0x7f0cfc158dc0 2024-04-20T11:01:02.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.338+0000 7f0cf27fc640 1 -- 172.21.15.27:0/22091486 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cec01a070 con 0x7f0cfc158dc0 2024-04-20T11:01:02.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.339+0000 7f0cf27fc640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0ce006c920 0x7f0ce006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:02.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.339+0000 7f0cf27fc640 1 -- 172.21.15.27:0/22091486 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f0cec08be90 con 0x7f0cfc158dc0 2024-04-20T11:01:02.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.339+0000 7f0d01a6a640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0ce006c920 0x7f0ce006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:02.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.339+0000 7f0d01a6a640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0ce006c920 0x7f0ce006ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8036000 tx=0x7f0cf8008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:02.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.343+0000 7f0cf27fc640 1 -- 172.21.15.27:0/22091486 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0cec059aa0 con 0x7f0cfc158dc0 2024-04-20T11:01:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.567+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cfc0c7b70 con 0x7f0cfc158dc0 2024-04-20T11:01:02.567 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.568+0000 7f0cf27fc640 1 -- 172.21.15.27:0/22091486 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+6109 (secure 0 0 0) 0x7f0cec05dda0 con 0x7f0cfc158dc0 2024-04-20T11:01:02.568 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:02.568 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:01.533340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:02.568 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0ce006c920 msgr2=0x7f0ce006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0ce006c920 0x7f0ce006ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8036000 tx=0x7f0cf8008030 comp rx=0 tx=0).stop 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc158dc0 msgr2=0x7f0cfc145180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc158dc0 0x7f0cfc145180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e8a0 tx=0x7f0cec00ed60 comp rx=0 tx=0).stop 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 shutdown_connections 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0ce006c920 0x7f0ce006ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0cfc15c150 0x7f0cfc1eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cfc158dc0 0x7f0cfc145180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 --2- 172.21.15.27:0/22091486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc1522f0 0x7f0cfc1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 >> 172.21.15.27:0/22091486 conn(0x7f0cfc0cd9c0 msgr2=0x7f0cfc0cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:02.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 shutdown_connections 2024-04-20T11:01:02.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:02.570+0000 7f0d03cce640 1 -- 172.21.15.27:0/22091486 wait complete. 2024-04-20T11:01:02.580 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:01:04.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.697+0000 7fe0c2c81640 1 Processor -- start 2024-04-20T11:01:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.697+0000 7fe0c2c81640 1 -- start start 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.698+0000 7fe0c2c81640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc0719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.698+0000 7fe0c2c81640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe0bc152640 0x7fe0bc0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.698+0000 7fe0c2c81640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0bc15ba80 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.698+0000 7fe0c2c81640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fe0bc15bd60 con 0x7fe0bc152640 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.698+0000 7fe0c2c81640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc15c040 con 0x7fe0bc152260 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0c0a1d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc0719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0bbfff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe0bc152640 0x7fe0bc0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0c0a1d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc0719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:55362/0 (socket says 172.21.15.27:55362) 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0c0a1d640 1 -- 172.21.15.27:0/3366806849 learned_addr learned my addr 172.21.15.27:0/3366806849 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0c121e640 1 --1- 172.21.15.27:0/3366806849 >> v1:172.21.15.137:6789/0 conn(0x7fe0bc0cd6f0 0x7fe0bc158760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:48036/0 (socket says 172.21.15.27:48036) 2024-04-20T11:01:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0c0a1d640 1 -- 172.21.15.27:0/3366806849 >> v2:172.21.15.137:3300/0 conn(0x7fe0bc152640 msgr2=0x7fe0bc0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:04.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0c0a1d640 1 --2- 172.21.15.27:0/3366806849 >> v2:172.21.15.137:3300/0 conn(0x7fe0bc152640 0x7fe0bc0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0c0a1d640 1 -- 172.21.15.27:0/3366806849 >> v1:172.21.15.137:6789/0 conn(0x7fe0bc0cd6f0 legacy=0x7fe0bc158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:04.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0c0a1d640 1 -- 172.21.15.27:0/3366806849 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc15c320 con 0x7fe0bc152260 2024-04-20T11:01:04.700 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.699+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/3366806849 >> v2:172.21.15.137:3300/0 conn(0x7fe0bc152640 0x7fe0bc0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:01:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.700+0000 7fe0c0a1d640 1 --2- 172.21.15.27:0/3366806849 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc0719c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e50 tx=0x7fe0b402e1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f66c33c099d7b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.700+0000 7fe0bb7fe640 1 -- 172.21.15.27:0/3366806849 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b402ed90 con 0x7fe0bc152260 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.700+0000 7fe0bb7fe640 1 -- 172.21.15.27:0/3366806849 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b40363f0 con 0x7fe0bc152260 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.700+0000 7fe0bb7fe640 1 -- 172.21.15.27:0/3366806849 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40367c0 con 0x7fe0bc152260 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.700+0000 7fe0c2c81640 1 -- 172.21.15.27:0/3366806849 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 msgr2=0x7fe0bc0719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.700+0000 7fe0c2c81640 1 --2- 172.21.15.27:0/3366806849 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc0719c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e50 tx=0x7fe0b402e1f0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.701+0000 7fe0c2c81640 1 -- 172.21.15.27:0/3366806849 shutdown_connections 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.701+0000 7fe0c2c81640 1 --2- 172.21.15.27:0/3366806849 >> v2:172.21.15.137:3300/0 conn(0x7fe0bc152640 0x7fe0bc0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.701+0000 7fe0c2c81640 1 --2- 172.21.15.27:0/3366806849 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc0719c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.701+0000 7fe0c2c81640 1 -- 172.21.15.27:0/3366806849 >> 172.21.15.27:0/3366806849 conn(0x7fe0bc0ccbd0 msgr2=0x7fe0bc0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.701+0000 7fe0c2c81640 1 -- 172.21.15.27:0/3366806849 shutdown_connections 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.701+0000 7fe0c2c81640 1 -- 172.21.15.27:0/3366806849 wait complete. 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.702+0000 7fe0c2c81640 1 Processor -- start 2024-04-20T11:01:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.702+0000 7fe0c2c81640 1 -- start start 2024-04-20T11:01:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c2c81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0cd6f0 0x7fe0bc14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:04.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c2c81640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc1f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c2c81640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0bc152640 0x7fe0bc146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c2c81640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc15c600 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c2c81640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc074260 con 0x7fe0bc152640 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c2c81640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc15b720 con 0x7fe0bc152260 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c0a1d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0cd6f0 0x7fe0bc14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c0a1d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0cd6f0 0x7fe0bc14c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57752/0 (socket says 172.21.15.27:57752) 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c0a1d640 1 -- 172.21.15.27:0/1851661347 learned_addr learned my addr 172.21.15.27:0/1851661347 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c121e640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0bc152640 0x7fe0bc146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc1f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c0a1d640 1 -- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 msgr2=0x7fe0bc1f09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:04.705 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c0a1d640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc1f09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c0a1d640 1 -- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0bc152640 msgr2=0x7fe0bc146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c0a1d640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0bc152640 0x7fe0bc146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c0a1d640 1 -- 172.21.15.27:0/1851661347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc1f0f00 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.703+0000 7fe0c0a1d640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0cd6f0 0x7fe0bc14c9d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4038040 tx=0x7fe0b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.704+0000 7fe0b97fa640 1 -- 172.21.15.27:0/1851661347 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4045020 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.704+0000 7fe0b97fa640 1 -- 172.21.15.27:0/1851661347 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b4040450 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.704+0000 7fe0b97fa640 1 -- 172.21.15.27:0/1851661347 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4009510 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.704+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc14fa80 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.704+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc150040 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.705+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0bc04fcf0 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.705+0000 7fe0b97fa640 1 -- 172.21.15.27:0/1851661347 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0b4048020 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.706+0000 7fe0b97fa640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe0a806c920 0x7fe0a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:04.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.706+0000 7fe0b97fa640 1 -- 172.21.15.27:0/1851661347 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7fe0b40b04c0 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.706+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe0a806c920 0x7fe0a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:04.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.706+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe0a806c920 0x7fe0a806ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc14c590 tx=0x7fe0b0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:04.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.710+0000 7fe0b97fa640 1 -- 172.21.15.27:0/1851661347 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe0b407e0d0 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.934+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0bc072880 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.935+0000 7fe0b97fa640 1 -- 172.21.15.27:0/1851661347 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+6109 (secure 0 0 0) 0x7fe0b40823d0 con 0x7fe0bc0cd6f0 2024-04-20T11:01:04.935 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:04.935 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:01.533340+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:04.935 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 39 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe0a806c920 msgr2=0x7fe0a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe0a806c920 0x7fe0a806ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc14c590 tx=0x7fe0b0009190 comp rx=0 tx=0).stop 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0cd6f0 msgr2=0x7fe0bc14c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0cd6f0 0x7fe0bc14c9d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4038040 tx=0x7fe0b4004a40 comp rx=0 tx=0).stop 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 shutdown_connections 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe0a806c920 0x7fe0a806ede0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0bc152640 0x7fe0bc146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe0bc152260 0x7fe0bc1f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 --2- 172.21.15.27:0/1851661347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0cd6f0 0x7fe0bc14c9d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 >> 172.21.15.27:0/1851661347 conn(0x7fe0bc0ccbd0 msgr2=0x7fe0bc147800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 shutdown_connections 2024-04-20T11:01:04.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:04.937+0000 7fe0c2c81640 1 -- 172.21.15.27:0/1851661347 wait complete. 2024-04-20T11:01:04.948 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:01:06.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.062+0000 7f6d51a31640 1 Processor -- start 2024-04-20T11:01:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.062+0000 7f6d51a31640 1 -- start start 2024-04-20T11:01:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d51a31640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6d4c153480 0x7f6d4c0c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d51a31640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6d4c1592e0 0x7f6d4c0c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d51a31640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c0cc2a0 0x7f6d4c0c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d51a31640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4c071520 con 0x7f6d4c1592e0 2024-04-20T11:01:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d51a31640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4c070a60 con 0x7f6d4c153480 2024-04-20T11:01:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d51a31640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6d4c071770 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d4a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6d4c1592e0 0x7f6d4c0c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d4affd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6d4c153480 0x7f6d4c0c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d4a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6d4c1592e0 0x7f6d4c0c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57760/0 (socket says 172.21.15.27:57760) 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d4a7fc640 1 -- 172.21.15.27:0/1450263981 learned_addr learned my addr 172.21.15.27:0/1450263981 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d4b7fe640 1 --2- 172.21.15.27:0/1450263981 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c0cc2a0 0x7f6d4c0c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d4a7fc640 1 -- 172.21.15.27:0/1450263981 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c0cc2a0 msgr2=0x7f6d4c0c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d4a7fc640 1 --2- 172.21.15.27:0/1450263981 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c0cc2a0 0x7f6d4c0c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d4a7fc640 1 -- 172.21.15.27:0/1450263981 >> v2:172.21.15.137:3300/0 conn(0x7f6d4c153480 msgr2=0x7f6d4c0c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.063+0000 7f6d4a7fc640 1 --2- 172.21.15.27:0/1450263981 >> v2:172.21.15.137:3300/0 conn(0x7f6d4c153480 0x7f6d4c0c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.064+0000 7f6d4a7fc640 1 -- 172.21.15.27:0/1450263981 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c070d00 con 0x7f6d4c1592e0 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.064+0000 7f6d4a7fc640 1 --2- 172.21.15.27:0/1450263981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c1592e0 0x7f6d4c0c9800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c06fca0 tx=0x7f6d3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834779c585fb95cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.064+0000 7f6d49ffb640 1 -- 172.21.15.27:0/1450263981 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34035020 con 0x7f6d4c1592e0 2024-04-20T11:01:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.064+0000 7f6d49ffb640 1 -- 172.21.15.27:0/1450263981 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d34031870 con 0x7f6d4c1592e0 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.064+0000 7f6d49ffb640 1 -- 172.21.15.27:0/1450263981 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34031c40 con 0x7f6d4c1592e0 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.065+0000 7f6d51a31640 1 -- 172.21.15.27:0/1450263981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c1592e0 msgr2=0x7f6d4c0c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.065+0000 7f6d51a31640 1 --2- 172.21.15.27:0/1450263981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c1592e0 0x7f6d4c0c9800 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c06fca0 tx=0x7f6d3402b910 comp rx=0 tx=0).stop 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.065+0000 7f6d51a31640 1 -- 172.21.15.27:0/1450263981 shutdown_connections 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.065+0000 7f6d51a31640 1 --2- 172.21.15.27:0/1450263981 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c0cc2a0 0x7f6d4c0c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.065+0000 7f6d51a31640 1 --2- 172.21.15.27:0/1450263981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c1592e0 0x7f6d4c0c9800 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.065+0000 7f6d51a31640 1 --2- 172.21.15.27:0/1450263981 >> v2:172.21.15.137:3300/0 conn(0x7f6d4c153480 0x7f6d4c0c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.065+0000 7f6d51a31640 1 -- 172.21.15.27:0/1450263981 >> 172.21.15.27:0/1450263981 conn(0x7f6d4c067ae0 msgr2=0x7f6d4c067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.065+0000 7f6d51a31640 1 -- 172.21.15.27:0/1450263981 shutdown_connections 2024-04-20T11:01:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.065+0000 7f6d51a31640 1 -- 172.21.15.27:0/1450263981 wait complete. 2024-04-20T11:01:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.066+0000 7f6d51a31640 1 Processor -- start 2024-04-20T11:01:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d51a31640 1 -- start start 2024-04-20T11:01:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d51a31640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c0cc2a0 0x7f6d4c1526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d51a31640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c153480 0x7f6d4c0c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d51a31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d4c1592e0 0x7f6d4c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d51a31640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c0c5f20 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d51a31640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c071a50 con 0x7f6d4c1592e0 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d51a31640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6d4c0cf350 con 0x7f6d4c153480 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d4affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c0cc2a0 0x7f6d4c1526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d4affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c0cc2a0 0x7f6d4c1526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57768/0 (socket says 172.21.15.27:57768) 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d4affd640 1 -- 172.21.15.27:0/4039072159 learned_addr learned my addr 172.21.15.27:0/4039072159 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d4b7fe640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d4c1592e0 0x7f6d4c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.067+0000 7f6d4a7fc640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c153480 0x7f6d4c0c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d4affd640 1 -- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c153480 msgr2=0x7f6d4c0c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d4affd640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c153480 0x7f6d4c0c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d4affd640 1 -- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d4c1592e0 msgr2=0x7f6d4c06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d4affd640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d4c1592e0 0x7f6d4c06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d4affd640 1 -- 172.21.15.27:0/4039072159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c146ef0 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d4affd640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c0cc2a0 0x7f6d4c1526c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400e9b0 tx=0x7f6d4400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d43fff640 1 -- 172.21.15.27:0/4039072159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44017070 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d43fff640 1 -- 172.21.15.27:0/4039072159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d4400c9a0 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c1471d0 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d43fff640 1 -- 172.21.15.27:0/4039072159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400cd90 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.068+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c0c80d0 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.069+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4c0688e0 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.070+0000 7f6d43fff640 1 -- 172.21.15.27:0/4039072159 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d4401a070 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.070+0000 7f6d43fff640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6d2c06c920 0x7f6d2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.070+0000 7f6d43fff640 1 -- 172.21.15.27:0/4039072159 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f6d4408bed0 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.070+0000 7f6d4a7fc640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6d2c06c920 0x7f6d2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.071+0000 7f6d4a7fc640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6d2c06c920 0x7f6d2c06ede0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d34036000 tx=0x7f6d3403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.075+0000 7f6d43fff640 1 -- 172.21.15.27:0/4039072159 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6d44059ae0 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.302+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d4c04fcf0 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.302 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.303+0000 7f6d43fff640 1 -- 172.21.15.27:0/4039072159 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+6114 (secure 0 0 0) 0x7f6d4405dde0 con 0x7f6d4c0cc2a0 2024-04-20T11:01:07.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:07.303 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4278,4293,4894,5029,5143]},"id":1}]} 2024-04-20T11:01:07.303 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 40 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6d2c06c920 msgr2=0x7f6d2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6d2c06c920 0x7f6d2c06ede0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d34036000 tx=0x7f6d3403c040 comp rx=0 tx=0).stop 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c0cc2a0 msgr2=0x7f6d4c1526c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c0cc2a0 0x7f6d4c1526c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400e9b0 tx=0x7f6d4400ee70 comp rx=0 tx=0).stop 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 shutdown_connections 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6d2c06c920 0x7f6d2c06ede0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d4c1592e0 0x7f6d4c06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6d4c153480 0x7f6d4c0c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 --2- 172.21.15.27:0/4039072159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d4c0cc2a0 0x7f6d4c1526c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 >> 172.21.15.27:0/4039072159 conn(0x7f6d4c067ae0 msgr2=0x7f6d4c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 shutdown_connections 2024-04-20T11:01:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:07.305+0000 7f6d51a31640 1 -- 172.21.15.27:0/4039072159 wait complete. 2024-04-20T11:01:07.316 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=2, replacing gid=4275 2024-04-20T11:01:07.317 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.e 2024-04-20T11:01:11.318 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-04-20T11:01:11.318 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-20T11:01:11.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-20T11:01:11.321 INFO:tasks.ceph.mds.e:Started 2024-04-20T11:01:11.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:11.408 INFO:tasks.ceph.mds.e.smithi027.stdout:starting mds.e at 2024-04-20T11:01:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.433+0000 7fd5c1ad3640 1 Processor -- start 2024-04-20T11:01:11.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.433+0000 7fd5c1ad3640 1 -- start start 2024-04-20T11:01:11.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5c1ad3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd5bc155950 0x7fd5bc0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:11.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5c1ad3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 0x7fd5bc0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:11.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5c1ad3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5bc0b5ea0 con 0x7fd5bc0b5a30 2024-04-20T11:01:11.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5c1ad3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fd5bc15b9c0 con 0x7fd5bc155950 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5c1ad3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc15bca0 con 0x7fd5bc0cc330 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5bb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd5bc155950 0x7fd5bc0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5bbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd5bc0b5a30 0x7fd5bc0d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40684/0 (socket says 172.21.15.27:40684) 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5bb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd5bc155950 0x7fd5bc0cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57792/0 (socket says 172.21.15.27:57792) 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5bbfff640 1 -- 172.21.15.27:0/3012863173 learned_addr learned my addr 172.21.15.27:0/3012863173 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.434+0000 7fd5baffd640 1 --2- 172.21.15.27:0/3012863173 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 0x7fd5bc0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.435+0000 7fd5bb7fe640 1 -- 172.21.15.27:0/3012863173 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 msgr2=0x7fd5bc0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.435+0000 7fd5bb7fe640 1 --2- 172.21.15.27:0/3012863173 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 0x7fd5bc0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.435+0000 7fd5bb7fe640 1 -- 172.21.15.27:0/3012863173 >> v1:172.21.15.27:6789/0 conn(0x7fd5bc0b5a30 legacy=0x7fd5bc0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.435+0000 7fd5bb7fe640 1 -- 172.21.15.27:0/3012863173 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc15bf80 con 0x7fd5bc155950 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.435+0000 7fd5ba7fc640 1 -- 172.21.15.27:0/3012863173 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109245637 0 0) 0x7fd5bc0b5ea0 con 0x7fd5bc0b5a30 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5bb7fe640 1 --2- 172.21.15.27:0/3012863173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 0x7fd5bc0cbdf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0097a0 tx=0x7fd5ac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7df2770b7b2b167 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:11.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5ba7fc640 1 -- 172.21.15.27:0/3012863173 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac034070 con 0x7fd5bc155950 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5ba7fc640 1 -- 172.21.15.27:0/3012863173 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ac030610 con 0x7fd5bc155950 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5ba7fc640 1 -- 172.21.15.27:0/3012863173 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0309e0 con 0x7fd5bc155950 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/3012863173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 msgr2=0x7fd5bc0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5c1ad3640 1 --2- 172.21.15.27:0/3012863173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 0x7fd5bc0cbdf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0097a0 tx=0x7fd5ac02b960 comp rx=0 tx=0).stop 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/3012863173 shutdown_connections 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5c1ad3640 1 --2- 172.21.15.27:0/3012863173 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 0x7fd5bc0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5c1ad3640 1 --2- 172.21.15.27:0/3012863173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 0x7fd5bc0cbdf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/3012863173 >> 172.21.15.27:0/3012863173 conn(0x7fd5bc0676c0 msgr2=0x7fd5bc067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/3012863173 shutdown_connections 2024-04-20T11:01:11.439 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.436+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/3012863173 wait complete. 2024-04-20T11:01:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.437+0000 7fd5c1ad3640 1 Processor -- start 2024-04-20T11:01:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.438+0000 7fd5c1ad3640 1 -- start start 2024-04-20T11:01:11.442 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.438+0000 7fd5c1ad3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5bc0b5a30 0x7fd5bc15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.438+0000 7fd5c1ad3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 0x7fd5bc06d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.438+0000 7fd5c1ad3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 0x7fd5bc1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.438+0000 7fd5c1ad3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc15fb90 con 0x7fd5bc155950 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.438+0000 7fd5c1ad3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc15c540 con 0x7fd5bc0b5a30 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.438+0000 7fd5c1ad3640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc15fd10 con 0x7fd5bc0cc330 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.438+0000 7fd5bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 0x7fd5bc1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 0x7fd5bc1f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57796/0 (socket says 172.21.15.27:57796) 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5bbfff640 1 -- 172.21.15.27:0/2737994155 learned_addr learned my addr 172.21.15.27:0/2737994155 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5baffd640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 0x7fd5bc06d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5bb7fe640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5bc0b5a30 0x7fd5bc15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5bbfff640 1 -- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 msgr2=0x7fd5bc06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5bbfff640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 0x7fd5bc06d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5bbfff640 1 -- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5bc0b5a30 msgr2=0x7fd5bc15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:11.443 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5bbfff640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5bc0b5a30 0x7fd5bc15b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5bbfff640 1 -- 172.21.15.27:0/2737994155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc15cbc0 con 0x7fd5bc155950 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5bbfff640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 0x7fd5bc1f1c40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400c7b0 tx=0x7fd5b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5a3fff640 1 -- 172.21.15.27:0/2737994155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4018070 con 0x7fd5bc155950 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5a3fff640 1 -- 172.21.15.27:0/2737994155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b400a640 con 0x7fd5bc155950 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5a3fff640 1 -- 172.21.15.27:0/2737994155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b400ce60 con 0x7fd5bc155950 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc15cea0 con 0x7fd5bc155950 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.439+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc15d460 con 0x7fd5bc155950 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.440+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc04fcf0 con 0x7fd5bc155950 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.441+0000 7fd5a3fff640 1 -- 172.21.15.27:0/2737994155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5b401b070 con 0x7fd5bc155950 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.441+0000 7fd5a3fff640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd5a806c8f0 0x7fd5a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.441+0000 7fd5a3fff640 1 -- 172.21.15.27:0/2737994155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7fd5b408b300 con 0x7fd5bc155950 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.441+0000 7fd5bb7fe640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd5a806c8f0 0x7fd5a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:11.444 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.442+0000 7fd5bb7fe640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd5a806c8f0 0x7fd5a806edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac036000 tx=0x7fd5ac008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:11.446 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.446+0000 7fd5a3fff640 1 -- 172.21.15.27:0/2737994155 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd5b4058e80 con 0x7fd5bc155950 2024-04-20T11:01:11.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.676+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5bc0cc930 con 0x7fd5bc155950 2024-04-20T11:01:11.677 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.677+0000 7fd5a3fff640 1 -- 172.21.15.27:0/2737994155 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7fd5b405d180 con 0x7fd5bc155950 2024-04-20T11:01:11.678 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:11.678 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:11.678 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:11.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd5a806c8f0 msgr2=0x7fd5a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:11.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd5a806c8f0 0x7fd5a806edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac036000 tx=0x7fd5ac008030 comp rx=0 tx=0).stop 2024-04-20T11:01:11.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 msgr2=0x7fd5bc1f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:11.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 0x7fd5bc1f1c40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400c7b0 tx=0x7fd5b400cc70 comp rx=0 tx=0).stop 2024-04-20T11:01:11.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 shutdown_connections 2024-04-20T11:01:11.681 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd5a806c8f0 0x7fd5a806edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5bc155950 0x7fd5bc1f1c40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd5bc0cc330 0x7fd5bc06d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 --2- 172.21.15.27:0/2737994155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5bc0b5a30 0x7fd5bc15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:11.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 >> 172.21.15.27:0/2737994155 conn(0x7fd5bc0676c0 msgr2=0x7fd5bc151830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:11.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 shutdown_connections 2024-04-20T11:01:11.682 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:11.679+0000 7fd5c1ad3640 1 -- 172.21.15.27:0/2737994155 wait complete. 2024-04-20T11:01:11.690 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-04-20T11:01:11.691 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 22 secs before thrashing 2024-04-20T11:01:12.679 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:01:12.682 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:01:33.691 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.807+0000 7f1b1cd38640 1 Processor -- start 2024-04-20T11:01:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.807+0000 7f1b1cd38640 1 -- start start 2024-04-20T11:01:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b1cd38640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b18153410 0x7f1b1815d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b1cd38640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b181537f0 0x7f1b180c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b1cd38640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1815d6b0 con 0x7f1b18159260 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b1cd38640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1b1806cad0 con 0x7f1b181537f0 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b1cd38640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f1b180c7cd0 con 0x7f1b18153410 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b0ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b181537f0 0x7f1b180c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b0ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1b181537f0 0x7f1b180c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47328/0 (socket says 172.21.15.27:47328) 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b0ffff640 1 -- 172.21.15.27:0/1324697067 learned_addr learned my addr 172.21.15.27:0/1324697067 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b1659c640 1 --2- 172.21.15.27:0/1324697067 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b18153410 0x7f1b1815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b0ffff640 1 -- 172.21.15.27:0/1324697067 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b18153410 msgr2=0x7f1b1815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b0ffff640 1 --2- 172.21.15.27:0/1324697067 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b18153410 0x7f1b1815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.808+0000 7f1b0ffff640 1 -- 172.21.15.27:0/1324697067 >> v1:172.21.15.137:6789/0 conn(0x7f1b18159260 legacy=0x7f1b1806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.809+0000 7f1b0ffff640 1 -- 172.21.15.27:0/1324697067 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b180c7fb0 con 0x7f1b181537f0 2024-04-20T11:01:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.809+0000 7f1b1659c640 1 --2- 172.21.15.27:0/1324697067 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b18153410 0x7f1b1815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:01:33.810 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.809+0000 7f1b0ffff640 1 --2- 172.21.15.27:0/1324697067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b181537f0 0x7f1b180c6730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009870 tx=0x7f1b0002ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbaae411321daf40 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.809+0000 7f1b15d9b640 1 -- 172.21.15.27:0/1324697067 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0002f830 con 0x7f1b181537f0 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b15d9b640 1 -- 172.21.15.27:0/1324697067 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b0002f9d0 con 0x7f1b181537f0 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b15d9b640 1 -- 172.21.15.27:0/1324697067 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0002fda0 con 0x7f1b181537f0 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b1cd38640 1 -- 172.21.15.27:0/1324697067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b181537f0 msgr2=0x7f1b180c6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b1cd38640 1 --2- 172.21.15.27:0/1324697067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b181537f0 0x7f1b180c6730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009870 tx=0x7f1b0002ba10 comp rx=0 tx=0).stop 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b1cd38640 1 -- 172.21.15.27:0/1324697067 shutdown_connections 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b1cd38640 1 --2- 172.21.15.27:0/1324697067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b181537f0 0x7f1b180c6730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b1cd38640 1 --2- 172.21.15.27:0/1324697067 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b18153410 0x7f1b1815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b1cd38640 1 -- 172.21.15.27:0/1324697067 >> 172.21.15.27:0/1324697067 conn(0x7f1b180ccbd0 msgr2=0x7f1b180ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b1cd38640 1 -- 172.21.15.27:0/1324697067 shutdown_connections 2024-04-20T11:01:33.811 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.810+0000 7f1b1cd38640 1 -- 172.21.15.27:0/1324697067 wait complete. 2024-04-20T11:01:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.812+0000 7f1b1cd38640 1 Processor -- start 2024-04-20T11:01:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.812+0000 7f1b1cd38640 1 -- start start 2024-04-20T11:01:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b1cd38640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18153410 0x7f1b18072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b1cd38640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b181537f0 0x7f1b180cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b1cd38640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b18159260 0x7f1b18146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b1cd38640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1815cc50 con 0x7f1b18159260 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b1cd38640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1815c970 con 0x7f1b18153410 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b1cd38640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f1b180c75d0 con 0x7f1b181537f0 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b16d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b18159260 0x7f1b18146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b16d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b18159260 0x7f1b18146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47340/0 (socket says 172.21.15.27:47340) 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b16d9d640 1 -- 172.21.15.27:0/737282483 learned_addr learned my addr 172.21.15.27:0/737282483 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b1659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18153410 0x7f1b18072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b0ffff640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b181537f0 0x7f1b180cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b16d9d640 1 -- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b181537f0 msgr2=0x7f1b180cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b16d9d640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b181537f0 0x7f1b180cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b16d9d640 1 -- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18153410 msgr2=0x7f1b18072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b16d9d640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18153410 0x7f1b18072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b16d9d640 1 -- 172.21.15.27:0/737282483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b18141eb0 con 0x7f1b18159260 2024-04-20T11:01:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b16d9d640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b18159260 0x7f1b18146b70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c6b0 tx=0x7f1b1000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b0effd640 1 -- 172.21.15.27:0/737282483 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10003b70 con 0x7f1b18159260 2024-04-20T11:01:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b0effd640 1 -- 172.21.15.27:0/737282483 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b10003d10 con 0x7f1b18159260 2024-04-20T11:01:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b18142190 con 0x7f1b18159260 2024-04-20T11:01:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.813+0000 7f1b0effd640 1 -- 172.21.15.27:0/737282483 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10005310 con 0x7f1b18159260 2024-04-20T11:01:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.814+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b180cc740 con 0x7f1b18159260 2024-04-20T11:01:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.814+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b180caa90 con 0x7f1b18159260 2024-04-20T11:01:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.815+0000 7f1b0effd640 1 -- 172.21.15.27:0/737282483 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b100054b0 con 0x7f1b18159260 2024-04-20T11:01:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.815+0000 7f1b0effd640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1af806c8f0 0x7f1af806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.816+0000 7f1b0effd640 1 -- 172.21.15.27:0/737282483 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f1b1008b9f0 con 0x7f1b18159260 2024-04-20T11:01:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.816+0000 7f1b1659c640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1af806c8f0 0x7f1af806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.816+0000 7f1b1659c640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1af806c8f0 0x7f1af806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800ab60 tx=0x7f1b08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:33.820+0000 7f1b0effd640 1 -- 172.21.15.27:0/737282483 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1b180caa90 con 0x7f1b18159260 2024-04-20T11:01:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.048+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b1804fcf0 con 0x7f1b18159260 2024-04-20T11:01:34.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.048+0000 7f1b0effd640 1 -- 172.21.15.27:0/737282483 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7f1b10059600 con 0x7f1b18159260 2024-04-20T11:01:34.049 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:34.049 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:34.049 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1af806c8f0 msgr2=0x7f1af806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1af806c8f0 0x7f1af806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800ab60 tx=0x7f1b08008040 comp rx=0 tx=0).stop 2024-04-20T11:01:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b18159260 msgr2=0x7f1b18146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b18159260 0x7f1b18146b70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c6b0 tx=0x7f1b1000cb70 comp rx=0 tx=0).stop 2024-04-20T11:01:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 shutdown_connections 2024-04-20T11:01:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1af806c8f0 0x7f1af806edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b18159260 0x7f1b18146b70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1b181537f0 0x7f1b180cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 --2- 172.21.15.27:0/737282483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18153410 0x7f1b18072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 >> 172.21.15.27:0/737282483 conn(0x7f1b180ccbd0 msgr2=0x7f1b181582d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 shutdown_connections 2024-04-20T11:01:34.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.051+0000 7f1b1cd38640 1 -- 172.21.15.27:0/737282483 wait complete. 2024-04-20T11:01:34.061 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=1) 2024-04-20T11:01:34.062 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:01:34.062 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:01:34.074 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:01:34.074 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.216+0000 7f0c2ed46640 1 Processor -- start 2024-04-20T11:01:34.216 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.216+0000 7f0c2ed46640 1 -- start start 2024-04-20T11:01:34.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2ed46640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0c28155950 0x7f0c280cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:34.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2ed46640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c280cc330 0x7f0c280b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:34.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2ed46640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f0c280b5a30 0x7f0c280d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:34.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2ed46640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f0c280b5ea0 con 0x7f0c28155950 2024-04-20T11:01:34.219 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2ed46640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f0c2815b9c0 con 0x7f0c280b5a30 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2ed46640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2815bca0 con 0x7f0c280cc330 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2cae2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0c28155950 0x7f0c280cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2cae2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0c28155950 0x7f0c280cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47356/0 (socket says 172.21.15.27:47356) 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2cae2640 1 -- 172.21.15.27:0/314873023 learned_addr learned my addr 172.21.15.27:0/314873023 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c27fff640 1 --2- 172.21.15.27:0/314873023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c280cc330 0x7f0c280b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.217+0000 7f0c2d2e3640 1 --2- 172.21.15.27:0/314873023 >> v2:172.21.15.137:3300/0 conn(0x7f0c280b5a30 0x7f0c280d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.218+0000 7f0c2cae2640 1 -- 172.21.15.27:0/314873023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c280cc330 msgr2=0x7f0c280b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.218+0000 7f0c2cae2640 1 --2- 172.21.15.27:0/314873023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c280cc330 0x7f0c280b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.218+0000 7f0c2cae2640 1 -- 172.21.15.27:0/314873023 >> v2:172.21.15.137:3300/0 conn(0x7f0c280b5a30 msgr2=0x7f0c280d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.218+0000 7f0c2cae2640 1 --2- 172.21.15.27:0/314873023 >> v2:172.21.15.137:3300/0 conn(0x7f0c280b5a30 0x7f0c280d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.220 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.218+0000 7f0c2cae2640 1 -- 172.21.15.27:0/314873023 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2815bf80 con 0x7f0c28155950 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.218+0000 7f0c2cae2640 1 --2- 172.21.15.27:0/314873023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c28155950 0x7f0c280cbdf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0c2808e4b0 tx=0x7f0c2002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0f17b23380a664 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c277fe640 1 -- 172.21.15.27:0/314873023 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2002cb00 con 0x7f0c28155950 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c277fe640 1 -- 172.21.15.27:0/314873023 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2002cca0 con 0x7f0c28155950 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c277fe640 1 -- 172.21.15.27:0/314873023 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20034610 con 0x7f0c28155950 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c2ed46640 1 -- 172.21.15.27:0/314873023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c28155950 msgr2=0x7f0c280cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/314873023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c28155950 0x7f0c280cbdf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0c2808e4b0 tx=0x7f0c2002bf60 comp rx=0 tx=0).stop 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c2ed46640 1 -- 172.21.15.27:0/314873023 shutdown_connections 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/314873023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c280cc330 0x7f0c280b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/314873023 >> v2:172.21.15.137:3300/0 conn(0x7f0c280b5a30 0x7f0c280d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.221 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/314873023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c28155950 0x7f0c280cbdf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c2ed46640 1 -- 172.21.15.27:0/314873023 >> 172.21.15.27:0/314873023 conn(0x7f0c280676c0 msgr2=0x7f0c28067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:34.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c2ed46640 1 -- 172.21.15.27:0/314873023 shutdown_connections 2024-04-20T11:01:34.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.219+0000 7f0c2ed46640 1 -- 172.21.15.27:0/314873023 wait complete. 2024-04-20T11:01:34.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.220+0000 7f0c2ed46640 1 Processor -- start 2024-04-20T11:01:34.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.220+0000 7f0c2ed46640 1 -- start start 2024-04-20T11:01:34.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2ed46640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c280b5a30 0x7f0c2815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:34.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2ed46640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c280cc330 0x7f0c281f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:34.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2ed46640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28155950 0x7f0c280c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:34.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2ed46640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2815fb90 con 0x7f0c280b5a30 2024-04-20T11:01:34.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2ed46640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2815c540 con 0x7f0c280cc330 2024-04-20T11:01:34.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2ed46640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2815fd10 con 0x7f0c28155950 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2cae2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c280b5a30 0x7f0c2815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2cae2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c280b5a30 0x7f0c2815e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47360/0 (socket says 172.21.15.27:47360) 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2cae2640 1 -- 172.21.15.27:0/464742313 learned_addr learned my addr 172.21.15.27:0/464742313 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c27fff640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c280cc330 0x7f0c281f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.221+0000 7f0c2d2e3640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28155950 0x7f0c280c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c2cae2640 1 -- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28155950 msgr2=0x7f0c280c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c2cae2640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28155950 0x7f0c280c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c2cae2640 1 -- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c280cc330 msgr2=0x7f0c281f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c2cae2640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c280cc330 0x7f0c281f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c2cae2640 1 -- 172.21.15.27:0/464742313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c281f4d00 con 0x7f0c280b5a30 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c27fff640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c280cc330 0x7f0c281f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:01:34.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c2cae2640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c280b5a30 0x7f0c2815e000 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0c20036040 tx=0x7f0c20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c257fa640 1 -- 172.21.15.27:0/464742313 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20043020 con 0x7f0c280b5a30 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c257fa640 1 -- 172.21.15.27:0/464742313 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2002ce50 con 0x7f0c280b5a30 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c257fa640 1 -- 172.21.15.27:0/464742313 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20009510 con 0x7f0c280b5a30 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2815ec70 con 0x7f0c280b5a30 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.222+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2815e990 con 0x7f0c280b5a30 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.223+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c2804fcf0 con 0x7f0c280b5a30 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.223+0000 7f0c257fa640 1 -- 172.21.15.27:0/464742313 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c20046020 con 0x7f0c280b5a30 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.224+0000 7f0c257fa640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0c0806c8f0 0x7f0c0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.224+0000 7f0c257fa640 1 -- 172.21.15.27:0/464742313 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f0c200ae1c0 con 0x7f0c280b5a30 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.224+0000 7f0c27fff640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0c0806c8f0 0x7f0c0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:34.227 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.225+0000 7f0c27fff640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0c0806c8f0 0x7f0c0806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0c28159540 tx=0x7f0c10009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:34.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.228+0000 7f0c257fa640 1 -- 172.21.15.27:0/464742313 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c2004d030 con 0x7f0c280b5a30 2024-04-20T11:01:34.468 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.469+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c280cc930 con 0x7f0c280b5a30 2024-04-20T11:01:34.469 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.469+0000 7f0c257fa640 1 -- 172.21.15.27:0/464742313 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7f0c2007bd40 con 0x7f0c280b5a30 2024-04-20T11:01:34.469 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:34.470 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:34.470 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:34.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.471+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0c0806c8f0 msgr2=0x7f0c0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:34.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.471+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0c0806c8f0 0x7f0c0806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0c28159540 tx=0x7f0c10009190 comp rx=0 tx=0).stop 2024-04-20T11:01:34.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.471+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c280b5a30 msgr2=0x7f0c2815e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:34.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.471+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c280b5a30 0x7f0c2815e000 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0c20036040 tx=0x7f0c20004a40 comp rx=0 tx=0).stop 2024-04-20T11:01:34.472 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.472+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 shutdown_connections 2024-04-20T11:01:34.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.472+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f0c0806c8f0 0x7f0c0806edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.472+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0c28155950 0x7f0c280c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.472+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c280cc330 0x7f0c281f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.472+0000 7f0c2ed46640 1 --2- 172.21.15.27:0/464742313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c280b5a30 0x7f0c2815e000 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:34.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.472+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 >> 172.21.15.27:0/464742313 conn(0x7f0c280676c0 msgr2=0x7f0c2806d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:34.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.472+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 shutdown_connections 2024-04-20T11:01:34.473 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:34.472+0000 7f0c2ed46640 1 -- 172.21.15.27:0/464742313 wait complete. 2024-04-20T11:01:34.483 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T11:01:36.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.593+0000 7efe69cc0640 1 Processor -- start 2024-04-20T11:01:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.593+0000 7efe69cc0640 1 -- start start 2024-04-20T11:01:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.594+0000 7efe69cc0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe640b54f0 0x7efe640b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.594+0000 7efe69cc0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe640b5e70 con 0x7efe64155950 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.594+0000 7efe69cc0640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6415b9c0 con 0x7efe640d0c90 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.594+0000 7efe69cc0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7efe6415bca0 con 0x7efe640b54f0 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.594+0000 7efe637fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efe64155950 0x7efe640cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58848/0 (socket says 172.21.15.27:58848) 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.594+0000 7efe637fe640 1 -- 172.21.15.27:0/3589707853 learned_addr learned my addr 172.21.15.27:0/3589707853 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe62ffd640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe640b54f0 0x7efe640b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2866228853 0 0) 0x7efe640b5e70 con 0x7efe64155950 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe4c003520 con 0x7efe64155950 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2582574162 0 0) 0x7efe4c003520 con 0x7efe64155950 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe640b54f0 msgr2=0x7efe640b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe640b54f0 0x7efe640b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 >> v1:172.21.15.137:6789/0 conn(0x7efe640d0c90 legacy=0x7efe640d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6415bf80 con 0x7efe64155950 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7efe540019f0 con 0x7efe64155950 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 >> v1:172.21.15.27:6789/0 conn(0x7efe64155950 legacy=0x7efe640cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe4c0039e0 0x7efe4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe640b54f0 0x7efe4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4c004d40 0x7efe4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efe6415bca0 con 0x7efe4c004d40 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c00bb00 con 0x7efe4c0039e0 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7efe4c00bd00 con 0x7efe640b54f0 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe637fe640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe4c0039e0 0x7efe4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe63fff640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4c004d40 0x7efe4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe62ffd640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe640b54f0 0x7efe4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.595+0000 7efe637fe640 1 -- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe640b54f0 msgr2=0x7efe4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.596+0000 7efe637fe640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe640b54f0 0x7efe4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.596+0000 7efe637fe640 1 -- 172.21.15.27:0/3589707853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4c004d40 msgr2=0x7efe4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.596+0000 7efe637fe640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4c004d40 0x7efe4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.596+0000 7efe637fe640 1 -- 172.21.15.27:0/3589707853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c003520 con 0x7efe4c0039e0 2024-04-20T11:01:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.596+0000 7efe63fff640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe4c004d40 0x7efe4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:01:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.596+0000 7efe637fe640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe4c0039e0 0x7efe4c003e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe5400a830 tx=0x7efe4c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec34daed53529594 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.596+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5400a480 con 0x7efe4c0039e0 2024-04-20T11:01:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.596+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe54034630 con 0x7efe4c0039e0 2024-04-20T11:01:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.597+0000 7efe627fc640 1 -- 172.21.15.27:0/3589707853 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54034a60 con 0x7efe4c0039e0 2024-04-20T11:01:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.597+0000 7efe69cc0640 1 -- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe4c0039e0 msgr2=0x7efe4c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.597+0000 7efe69cc0640 1 --2- 172.21.15.27:0/3589707853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe4c0039e0 0x7efe4c003e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe5400a830 tx=0x7efe4c005830 comp rx=0 tx=0).stop 2024-04-20T11:01:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.597+0000 7efe63fff640 1 -- 172.21.15.27:0/3589707853 reap_dead start 2024-04-20T11:01:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.597+0000 7efe69cc0640 1 -- 172.21.15.27:0/3589707853 shutdown_connections 2024-04-20T11:01:36.601 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.597+0000 7efe69cc0640 1 -- 172.21.15.27:0/3589707853 >> 172.21.15.27:0/3589707853 conn(0x7efe640676c0 msgr2=0x7efe64067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.597+0000 7efe69cc0640 1 -- 172.21.15.27:0/3589707853 shutdown_connections 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.597+0000 7efe69cc0640 1 -- 172.21.15.27:0/3589707853 wait complete. 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.598+0000 7efe69cc0640 1 Processor -- start 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.598+0000 7efe69cc0640 1 -- start start 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe69cc0640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe4c0039e0 0x7efe640c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe69cc0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe640c7640 0x7efe6415e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe69cc0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe641f1740 0x7efe640c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe69cc0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efe6415fb90 con 0x7efe641f1740 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe69cc0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe6415c540 con 0x7efe640c7640 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe69cc0640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7efe6415fd10 con 0x7efe4c0039e0 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe63fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe641f1740 0x7efe640c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe63fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe641f1740 0x7efe640c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51224/0 (socket says 172.21.15.27:51224) 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe63fff640 1 -- 172.21.15.27:0/3259936315 learned_addr learned my addr 172.21.15.27:0/3259936315 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:36.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe62ffd640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe640c7640 0x7efe6415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.599+0000 7efe637fe640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe4c0039e0 0x7efe640c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe63fff640 1 -- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe4c0039e0 msgr2=0x7efe640c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe63fff640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe4c0039e0 0x7efe640c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe63fff640 1 -- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe640c7640 msgr2=0x7efe6415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe63fff640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe640c7640 0x7efe6415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe63fff640 1 -- 172.21.15.27:0/3259936315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe640c6f60 con 0x7efe641f1740 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe63fff640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe641f1740 0x7efe640c6010 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efe640cb2e0 tx=0x7efe5c00fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe4bfff640 1 -- 172.21.15.27:0/3259936315 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c001370 con 0x7efe641f1740 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe4bfff640 1 -- 172.21.15.27:0/3259936315 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c018070 con 0x7efe641f1740 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe640ca940 con 0x7efe641f1740 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe4bfff640 1 -- 172.21.15.27:0/3259936315 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0052f0 con 0x7efe641f1740 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.600+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe640caf00 con 0x7efe641f1740 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.601+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6404fcf0 con 0x7efe641f1740 2024-04-20T11:01:36.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.601+0000 7efe4bfff640 1 -- 172.21.15.27:0/3259936315 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe5c00d6b0 con 0x7efe641f1740 2024-04-20T11:01:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.602+0000 7efe4bfff640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efe5006c8f0 0x7efe5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.602+0000 7efe4bfff640 1 -- 172.21.15.27:0/3259936315 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7efe5c08b7e0 con 0x7efe641f1740 2024-04-20T11:01:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.602+0000 7efe637fe640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efe5006c8f0 0x7efe5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.602+0000 7efe637fe640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efe5006c8f0 0x7efe5006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efe54034000 tx=0x7efe5402f000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:36.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.606+0000 7efe4bfff640 1 -- 172.21.15.27:0/3259936315 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe5c0599b0 con 0x7efe641f1740 2024-04-20T11:01:36.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.831+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe640cd0a0 con 0x7efe641f1740 2024-04-20T11:01:36.832 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.832+0000 7efe4bfff640 1 -- 172.21.15.27:0/3259936315 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7efe5c0592b0 con 0x7efe641f1740 2024-04-20T11:01:36.832 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:36.832 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:36.832 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:36.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efe5006c8f0 msgr2=0x7efe5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:36.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efe5006c8f0 0x7efe5006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efe54034000 tx=0x7efe5402f000 comp rx=0 tx=0).stop 2024-04-20T11:01:36.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe641f1740 msgr2=0x7efe640c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:36.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe641f1740 0x7efe640c6010 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efe640cb2e0 tx=0x7efe5c00fb40 comp rx=0 tx=0).stop 2024-04-20T11:01:36.835 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 shutdown_connections 2024-04-20T11:01:36.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efe5006c8f0 0x7efe5006edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:36.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe641f1740 0x7efe640c6010 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:36.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe640c7640 0x7efe6415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:36.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 --2- 172.21.15.27:0/3259936315 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efe4c0039e0 0x7efe640c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:36.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 >> 172.21.15.27:0/3259936315 conn(0x7efe640676c0 msgr2=0x7efe640cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:36.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 shutdown_connections 2024-04-20T11:01:36.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:36.834+0000 7efe69cc0640 1 -- 172.21.15.27:0/3259936315 wait complete. 2024-04-20T11:01:36.845 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T11:01:38.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:38.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.960+0000 7fa95b443640 1 Processor -- start 2024-04-20T11:01:38.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.960+0000 7fa95b443640 1 -- start start 2024-04-20T11:01:38.960 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa95b443640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa95406ca40 0x7fa95415c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:38.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa95b443640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95415ec70 con 0x7fa95415f710 2024-04-20T11:01:38.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa95b443640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95406e510 con 0x7fa9541556f0 2024-04-20T11:01:38.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa95b443640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa95406ead0 con 0x7fa95406ca40 2024-04-20T11:01:38.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa9589de640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa95415f710 0x7fa954163e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58854/0 (socket says 172.21.15.27:58854) 2024-04-20T11:01:38.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa9589de640 1 -- 172.21.15.27:0/1981412647 learned_addr learned my addr 172.21.15.27:0/1981412647 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:01:38.961 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa9599e0640 1 --2- 172.21.15.27:0/1981412647 >> v2:172.21.15.27:3300/0 conn(0x7fa95406ca40 0x7fa95415c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:38.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa9599e0640 1 -- 172.21.15.27:0/1981412647 >> v1:172.21.15.137:6789/0 conn(0x7fa9541556f0 legacy=0x7fa9540cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:38.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa9599e0640 1 -- 172.21.15.27:0/1981412647 >> v1:172.21.15.27:6789/0 conn(0x7fa95415f710 legacy=0x7fa954163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:38.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.961+0000 7fa9599e0640 1 -- 172.21.15.27:0/1981412647 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95406e7f0 con 0x7fa95406ca40 2024-04-20T11:01:38.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.962+0000 7fa9599e0640 1 --2- 172.21.15.27:0/1981412647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 0x7fa95415c9d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009d90 tx=0x7fa94c02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ebe465ea13a67b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:38.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.962+0000 7fa94bfff640 1 -- 172.21.15.27:0/1981412647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c039070 con 0x7fa95406ca40 2024-04-20T11:01:38.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.962+0000 7fa94bfff640 1 -- 172.21.15.27:0/1981412647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c02cd80 con 0x7fa95406ca40 2024-04-20T11:01:38.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.962+0000 7fa94bfff640 1 -- 172.21.15.27:0/1981412647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c034660 con 0x7fa95406ca40 2024-04-20T11:01:38.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.963+0000 7fa95b443640 1 -- 172.21.15.27:0/1981412647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 msgr2=0x7fa95415c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:38.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.963+0000 7fa95b443640 1 --2- 172.21.15.27:0/1981412647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 0x7fa95415c9d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009d90 tx=0x7fa94c02bfd0 comp rx=0 tx=0).stop 2024-04-20T11:01:38.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.963+0000 7fa95b443640 1 -- 172.21.15.27:0/1981412647 shutdown_connections 2024-04-20T11:01:38.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.963+0000 7fa95b443640 1 --2- 172.21.15.27:0/1981412647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 0x7fa95415c9d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:38.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.963+0000 7fa95b443640 1 -- 172.21.15.27:0/1981412647 >> 172.21.15.27:0/1981412647 conn(0x7fa954067ae0 msgr2=0x7fa954067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:38.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.963+0000 7fa95b443640 1 -- 172.21.15.27:0/1981412647 shutdown_connections 2024-04-20T11:01:38.964 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.963+0000 7fa95b443640 1 -- 172.21.15.27:0/1981412647 wait complete. 2024-04-20T11:01:38.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.964+0000 7fa95b443640 1 Processor -- start 2024-04-20T11:01:38.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.964+0000 7fa95b443640 1 -- start start 2024-04-20T11:01:38.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa95b443640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 0x7fa954152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:38.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa95b443640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9541556f0 0x7fa95414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:38.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa95b443640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa95415f710 0x7fa954152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:38.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa95b443640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540c9800 con 0x7fa95406ca40 2024-04-20T11:01:38.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa95b443640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa95415f1f0 con 0x7fa95415f710 2024-04-20T11:01:38.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa95b443640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fa9540c9980 con 0x7fa9541556f0 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9591df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 0x7fa954152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9591df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 0x7fa954152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51232/0 (socket says 172.21.15.27:51232) 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9591df640 1 -- 172.21.15.27:0/8840568 learned_addr learned my addr 172.21.15.27:0/8840568 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9589de640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9541556f0 0x7fa95414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9599e0640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa95415f710 0x7fa954152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9591df640 1 -- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9541556f0 msgr2=0x7fa95414e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9591df640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9541556f0 0x7fa95414e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9591df640 1 -- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa95415f710 msgr2=0x7fa954152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9591df640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa95415f710 0x7fa954152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9591df640 1 -- 172.21.15.27:0/8840568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9541f4d10 con 0x7fa95406ca40 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9599e0640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa95415f710 0x7fa954152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.965+0000 7fa9589de640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9541556f0 0x7fa95414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.966+0000 7fa9591df640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 0x7fa954152b10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa95000c400 tx=0x7fa95000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.966+0000 7fa949ffb640 1 -- 172.21.15.27:0/8840568 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9500052d0 con 0x7fa95406ca40 2024-04-20T11:01:38.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.966+0000 7fa949ffb640 1 -- 172.21.15.27:0/8840568 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa950005490 con 0x7fa95406ca40 2024-04-20T11:01:38.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.966+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541f4ff0 con 0x7fa95406ca40 2024-04-20T11:01:38.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.966+0000 7fa949ffb640 1 -- 172.21.15.27:0/8840568 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950007260 con 0x7fa95406ca40 2024-04-20T11:01:38.968 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.966+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541f55b0 con 0x7fa95406ca40 2024-04-20T11:01:38.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.967+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9540688e0 con 0x7fa95406ca40 2024-04-20T11:01:38.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.967+0000 7fa949ffb640 1 -- 172.21.15.27:0/8840568 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa950007400 con 0x7fa95406ca40 2024-04-20T11:01:38.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.968+0000 7fa949ffb640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa93806c8f0 0x7fa93806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:38.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.968+0000 7fa949ffb640 1 -- 172.21.15.27:0/8840568 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7fa95008b230 con 0x7fa95406ca40 2024-04-20T11:01:38.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.968+0000 7fa9589de640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa93806c8f0 0x7fa93806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:38.969 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.969+0000 7fa9589de640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa93806c8f0 0x7fa93806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa9541e7160 tx=0x7fa944009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:38.972 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:38.972+0000 7fa949ffb640 1 -- 172.21.15.27:0/8840568 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa950059490 con 0x7fa95406ca40 2024-04-20T11:01:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.197+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa95404fcf0 con 0x7fa95406ca40 2024-04-20T11:01:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.198+0000 7fa949ffb640 1 -- 172.21.15.27:0/8840568 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7fa950058d90 con 0x7fa95406ca40 2024-04-20T11:01:39.197 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:39.198 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:39.198 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa93806c8f0 msgr2=0x7fa93806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa93806c8f0 0x7fa93806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa9541e7160 tx=0x7fa944009040 comp rx=0 tx=0).stop 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 msgr2=0x7fa954152b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 0x7fa954152b10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa95000c400 tx=0x7fa95000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 shutdown_connections 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa93806c8f0 0x7fa93806edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa95415f710 0x7fa954152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa9541556f0 0x7fa95414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 --2- 172.21.15.27:0/8840568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa95406ca40 0x7fa954152b10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 >> 172.21.15.27:0/8840568 conn(0x7fa954067ae0 msgr2=0x7fa9540cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 shutdown_connections 2024-04-20T11:01:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:39.200+0000 7fa95b443640 1 -- 172.21.15.27:0/8840568 wait complete. 2024-04-20T11:01:39.211 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T11:01:41.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:41.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.333+0000 7fca4cc70640 1 Processor -- start 2024-04-20T11:01:41.333 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.333+0000 7fca4cc70640 1 -- start start 2024-04-20T11:01:41.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca4cc70640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fca4815c860 0x7fca4815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:41.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca4cc70640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4813e330 0x7fca4813e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:41.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca4cc70640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca4813f2c0 con 0x7fca4813bab0 2024-04-20T11:01:41.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca4cc70640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fca480cc160 con 0x7fca4815c860 2024-04-20T11:01:41.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca4cc70640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fca4815d1e0 con 0x7fca4813e330 2024-04-20T11:01:41.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca46d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4813e330 0x7fca4813e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca3ffff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fca4815c860 0x7fca4815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca4659c640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fca4813bab0 0x7fca48070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:50748/0 (socket says 172.21.15.27:50748) 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca3ffff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fca4815c860 0x7fca4815cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:43538/0 (socket says 172.21.15.27:43538) 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca46d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4813e330 0x7fca4813e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60000/0 (socket says 172.21.15.27:60000) 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca4659c640 1 -- 172.21.15.27:0/1218009533 learned_addr learned my addr 172.21.15.27:0/1218009533 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca3ffff640 1 -- 172.21.15.27:0/1218009533 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4813e330 msgr2=0x7fca4813e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca3ffff640 1 --2- 172.21.15.27:0/1218009533 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4813e330 0x7fca4813e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca3ffff640 1 -- 172.21.15.27:0/1218009533 >> v1:172.21.15.137:6789/0 conn(0x7fca4813bab0 legacy=0x7fca48070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca3ffff640 1 -- 172.21.15.27:0/1218009533 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca48159360 con 0x7fca4815c860 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.334+0000 7fca45d9b640 1 -- 172.21.15.27:0/1218009533 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771692034 0 0) 0x7fca4813f2c0 con 0x7fca4813bab0 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.335+0000 7fca3ffff640 1 --2- 172.21.15.27:0/1218009533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4815c860 0x7fca4815cca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fca300097a0 tx=0x7fca3002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=690159140afaafbc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.335+0000 7fca45d9b640 1 -- 172.21.15.27:0/1218009533 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30035020 con 0x7fca4815c860 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.335+0000 7fca45d9b640 1 -- 172.21.15.27:0/1218009533 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca30031830 con 0x7fca4815c860 2024-04-20T11:01:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.335+0000 7fca45d9b640 1 -- 172.21.15.27:0/1218009533 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30031c00 con 0x7fca4815c860 2024-04-20T11:01:41.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.336+0000 7fca4cc70640 1 -- 172.21.15.27:0/1218009533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4815c860 msgr2=0x7fca4815cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:41.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.336+0000 7fca4cc70640 1 --2- 172.21.15.27:0/1218009533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4815c860 0x7fca4815cca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fca300097a0 tx=0x7fca3002b8d0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.336+0000 7fca4cc70640 1 -- 172.21.15.27:0/1218009533 shutdown_connections 2024-04-20T11:01:41.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.336+0000 7fca4cc70640 1 --2- 172.21.15.27:0/1218009533 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4813e330 0x7fca4813e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.336+0000 7fca4cc70640 1 --2- 172.21.15.27:0/1218009533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4815c860 0x7fca4815cca0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.336+0000 7fca4cc70640 1 -- 172.21.15.27:0/1218009533 >> 172.21.15.27:0/1218009533 conn(0x7fca4806c070 msgr2=0x7fca480676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.336+0000 7fca4cc70640 1 -- 172.21.15.27:0/1218009533 shutdown_connections 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.336+0000 7fca4cc70640 1 -- 172.21.15.27:0/1218009533 wait complete. 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4cc70640 1 Processor -- start 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4cc70640 1 -- start start 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4cc70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4813bab0 0x7fca48153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4cc70640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4813e330 0x7fca48139ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4cc70640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4815c860 0x7fca481eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4cc70640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca48159c00 con 0x7fca4813e330 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4cc70640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fca48159920 con 0x7fca4813bab0 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4cc70640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fca4813be90 con 0x7fca4815c860 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4813bab0 0x7fca48153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca3ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4813e330 0x7fca48139ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca46d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4815c860 0x7fca481eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca3ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4813e330 0x7fca48139ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51246/0 (socket says 172.21.15.27:51246) 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca3ffff640 1 -- 172.21.15.27:0/2282847563 learned_addr learned my addr 172.21.15.27:0/2282847563 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca4659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4813bab0 0x7fca48153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:43546/0 (socket says 172.21.15.27:43546) 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.338+0000 7fca46d9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4815c860 0x7fca481eefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:60010/0 (socket says 172.21.15.27:60010) 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca3ffff640 1 -- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4815c860 msgr2=0x7fca481eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca3ffff640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4815c860 0x7fca481eefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca3ffff640 1 -- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4813bab0 msgr2=0x7fca48153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca3ffff640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4813bab0 0x7fca48153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca3ffff640 1 -- 172.21.15.27:0/2282847563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca481eead0 con 0x7fca4813e330 2024-04-20T11:01:41.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca4659c640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4813bab0 0x7fca48153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca3ffff640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4813e330 0x7fca48139ee0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fca300097a0 tx=0x7fca300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca3effd640 1 -- 172.21.15.27:0/2282847563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30035040 con 0x7fca4813e330 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca3effd640 1 -- 172.21.15.27:0/2282847563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca30004230 con 0x7fca4813e330 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca3effd640 1 -- 172.21.15.27:0/2282847563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3003a640 con 0x7fca4813e330 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca481559f0 con 0x7fca4813e330 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.339+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca48155fb0 con 0x7fca4813e330 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.340+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca480756c0 con 0x7fca4813e330 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.341+0000 7fca3effd640 1 -- 172.21.15.27:0/2282847563 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca30038020 con 0x7fca4813e330 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.341+0000 7fca3effd640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca2806c8f0 0x7fca2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.341+0000 7fca3effd640 1 -- 172.21.15.27:0/2282847563 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7fca300ad9b0 con 0x7fca4813e330 2024-04-20T11:01:41.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.342+0000 7fca4659c640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca2806c8f0 0x7fca2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:41.343 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.342+0000 7fca4659c640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca2806c8f0 0x7fca2806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fca38000f60 tx=0x7fca38007d50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.346+0000 7fca3effd640 1 -- 172.21.15.27:0/2282847563 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fca480756c0 con 0x7fca4813e330 2024-04-20T11:01:41.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.571+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca481eeca0 con 0x7fca4813e330 2024-04-20T11:01:41.571 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.571+0000 7fca3effd640 1 -- 172.21.15.27:0/2282847563 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7fca3007b530 con 0x7fca4813e330 2024-04-20T11:01:41.571 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:41.571 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:41.572 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca2806c8f0 msgr2=0x7fca2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca2806c8f0 0x7fca2806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fca38000f60 tx=0x7fca38007d50 comp rx=0 tx=0).stop 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4813e330 msgr2=0x7fca48139ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4813e330 0x7fca48139ee0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fca300097a0 tx=0x7fca300046c0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 shutdown_connections 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fca2806c8f0 0x7fca2806edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fca4815c860 0x7fca481eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca4813e330 0x7fca48139ee0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 --2- 172.21.15.27:0/2282847563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca4813bab0 0x7fca48153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 >> 172.21.15.27:0/2282847563 conn(0x7fca4806c070 msgr2=0x7fca481530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 shutdown_connections 2024-04-20T11:01:41.574 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:41.574+0000 7fca4cc70640 1 -- 172.21.15.27:0/2282847563 wait complete. 2024-04-20T11:01:41.585 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T11:01:42.751 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:01:42.754 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:01:43.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.696+0000 7f8a136a8640 1 Processor -- start 2024-04-20T11:01:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.697+0000 7f8a136a8640 1 -- start start 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.697+0000 7f8a136a8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a0c15ed10 0x7f8a0c073970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.697+0000 7f8a136a8640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8a0c072130 0x7f8a0c0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.697+0000 7f8a136a8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c0c8d30 0x7f8a0c15f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.697+0000 7f8a136a8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0c0c3bc0 con 0x7f8a0c15ed10 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.697+0000 7f8a136a8640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0c0c4050 con 0x7f8a0c072130 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.697+0000 7f8a136a8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c073fc0 con 0x7f8a0c0c8d30 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a11444640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a0c15ed10 0x7f8a0c073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a11444640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a0c15ed10 0x7f8a0c073970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51256/0 (socket says 172.21.15.27:51256) 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a11444640 1 -- 172.21.15.27:0/967758090 learned_addr learned my addr 172.21.15.27:0/967758090 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a10c43640 1 --2- 172.21.15.27:0/967758090 >> v2:172.21.15.137:3300/0 conn(0x7f8a0c072130 0x7f8a0c0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a11c45640 1 --2- 172.21.15.27:0/967758090 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c0c8d30 0x7f8a0c15f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a11444640 1 -- 172.21.15.27:0/967758090 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c0c8d30 msgr2=0x7f8a0c15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a11444640 1 --2- 172.21.15.27:0/967758090 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c0c8d30 0x7f8a0c15f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a11444640 1 -- 172.21.15.27:0/967758090 >> v2:172.21.15.137:3300/0 conn(0x7f8a0c072130 msgr2=0x7f8a0c0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a11444640 1 --2- 172.21.15.27:0/967758090 >> v2:172.21.15.137:3300/0 conn(0x7f8a0c072130 0x7f8a0c0c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a11444640 1 -- 172.21.15.27:0/967758090 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c0c2fb0 con 0x7f8a0c15ed10 2024-04-20T11:01:43.699 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.698+0000 7f8a10c43640 1 --2- 172.21.15.27:0/967758090 >> v2:172.21.15.137:3300/0 conn(0x7f8a0c072130 0x7f8a0c0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:01:43.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a11444640 1 --2- 172.21.15.27:0/967758090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 0x7f8a0c073970 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c06f8e0 tx=0x7f8a0402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82b9603ea6022d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:43.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a03fff640 1 -- 172.21.15.27:0/967758090 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04035020 con 0x7f8a0c15ed10 2024-04-20T11:01:43.706 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a03fff640 1 -- 172.21.15.27:0/967758090 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04031640 con 0x7f8a0c15ed10 2024-04-20T11:01:43.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a03fff640 1 -- 172.21.15.27:0/967758090 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402d040 con 0x7f8a0c15ed10 2024-04-20T11:01:43.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a136a8640 1 -- 172.21.15.27:0/967758090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 msgr2=0x7f8a0c073970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:43.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a136a8640 1 --2- 172.21.15.27:0/967758090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 0x7f8a0c073970 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c06f8e0 tx=0x7f8a0402b8f0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a136a8640 1 -- 172.21.15.27:0/967758090 shutdown_connections 2024-04-20T11:01:43.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a136a8640 1 --2- 172.21.15.27:0/967758090 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c0c8d30 0x7f8a0c15f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a136a8640 1 --2- 172.21.15.27:0/967758090 >> v2:172.21.15.137:3300/0 conn(0x7f8a0c072130 0x7f8a0c0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.699+0000 7f8a136a8640 1 --2- 172.21.15.27:0/967758090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 0x7f8a0c073970 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.700+0000 7f8a136a8640 1 -- 172.21.15.27:0/967758090 >> 172.21.15.27:0/967758090 conn(0x7f8a0c0ca8f0 msgr2=0x7f8a0c0cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.700+0000 7f8a136a8640 1 -- 172.21.15.27:0/967758090 shutdown_connections 2024-04-20T11:01:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.700+0000 7f8a136a8640 1 -- 172.21.15.27:0/967758090 wait complete. 2024-04-20T11:01:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.701+0000 7f8a136a8640 1 Processor -- start 2024-04-20T11:01:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.701+0000 7f8a136a8640 1 -- start start 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.701+0000 7f8a136a8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c072130 0x7f8a0c14e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.701+0000 7f8a136a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a0c0c8d30 0x7f8a0c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a136a8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 0x7f8a0c0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a136a8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0c3850 con 0x7f8a0c15ed10 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a136a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0c3570 con 0x7f8a0c0c8d30 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a136a8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c15f630 con 0x7f8a0c072130 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11c45640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 0x7f8a0c0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11c45640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 0x7f8a0c0c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51260/0 (socket says 172.21.15.27:51260) 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11c45640 1 -- 172.21.15.27:0/192503491 learned_addr learned my addr 172.21.15.27:0/192503491 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11444640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c072130 0x7f8a0c14e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a10c43640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a0c0c8d30 0x7f8a0c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11c45640 1 -- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c072130 msgr2=0x7f8a0c14e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11c45640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c072130 0x7f8a0c14e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11c45640 1 -- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a0c0c8d30 msgr2=0x7f8a0c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11c45640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a0c0c8d30 0x7f8a0c15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11c45640 1 -- 172.21.15.27:0/192503491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c156ee0 con 0x7f8a0c15ed10 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a11c45640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 0x7f8a0c0c51f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800cda0 tx=0x7f8a0800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a01ffb640 1 -- 172.21.15.27:0/192503491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08003b90 con 0x7f8a0c15ed10 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a01ffb640 1 -- 172.21.15.27:0/192503491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a08003d30 con 0x7f8a0c15ed10 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c15bd30 con 0x7f8a0c15ed10 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.702+0000 7f8a01ffb640 1 -- 172.21.15.27:0/192503491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08004240 con 0x7f8a0c15ed10 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.703+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c1f5220 con 0x7f8a0c15ed10 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.704+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c15abc0 con 0x7f8a0c15ed10 2024-04-20T11:01:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.704+0000 7f8a01ffb640 1 -- 172.21.15.27:0/192503491 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a080043e0 con 0x7f8a0c15ed10 2024-04-20T11:01:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.704+0000 7f8a01ffb640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f89f006c8f0 0x7f89f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.705+0000 7f8a01ffb640 1 -- 172.21.15.27:0/192503491 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f8a0808c790 con 0x7f8a0c15ed10 2024-04-20T11:01:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.705+0000 7f8a11444640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f89f006c8f0 0x7f89f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.705+0000 7f8a11444640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f89f006c8f0 0x7f89f006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a04036000 tx=0x7f8a0403c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.709+0000 7f8a01ffb640 1 -- 172.21.15.27:0/192503491 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a0805a9f0 con 0x7f8a0c15ed10 2024-04-20T11:01:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.934+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a0c04fcf0 con 0x7f8a0c15ed10 2024-04-20T11:01:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.935+0000 7f8a01ffb640 1 -- 172.21.15.27:0/192503491 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7f8a0805a2f0 con 0x7f8a0c15ed10 2024-04-20T11:01:43.935 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:43.935 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:43.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f89f006c8f0 msgr2=0x7f89f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:43.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f89f006c8f0 0x7f89f006edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a04036000 tx=0x7f8a0403c040 comp rx=0 tx=0).stop 2024-04-20T11:01:43.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 msgr2=0x7f8a0c0c51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:43.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 0x7f8a0c0c51f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800cda0 tx=0x7f8a0800a660 comp rx=0 tx=0).stop 2024-04-20T11:01:43.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 shutdown_connections 2024-04-20T11:01:43.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f89f006c8f0 0x7f89f006edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a0c15ed10 0x7f8a0c0c51f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.937 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a0c0c8d30 0x7f8a0c15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 --2- 172.21.15.27:0/192503491 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8a0c072130 0x7f8a0c14e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:43.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 >> 172.21.15.27:0/192503491 conn(0x7f8a0c0ca8f0 msgr2=0x7f8a0c073800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:43.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 shutdown_connections 2024-04-20T11:01:43.938 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:43.937+0000 7f8a136a8640 1 -- 172.21.15.27:0/192503491 wait complete. 2024-04-20T11:01:43.948 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T11:01:45.950 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:46.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.052+0000 7f58dfbca640 1 Processor -- start 2024-04-20T11:01:46.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.052+0000 7f58dfbca640 1 -- start start 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dfbca640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58d8153410 0x7f58d815d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dfbca640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8159260 0x7f58d8163b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dfbca640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d815d6b0 con 0x7f58d81537f0 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dfbca640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f58d806cad0 con 0x7f58d8153410 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dfbca640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f58d80c7cd0 con 0x7f58d8159260 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dd966640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58d8153410 0x7f58d815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dd966640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58d8153410 0x7f58d815d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52072/0 (socket says 172.21.15.27:52072) 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dd966640 1 -- 172.21.15.27:0/3061280672 learned_addr learned my addr 172.21.15.27:0/3061280672 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58de167640 1 --2- 172.21.15.27:0/3061280672 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8159260 0x7f58d8163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dd966640 1 -- 172.21.15.27:0/3061280672 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8159260 msgr2=0x7f58d8163b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dd966640 1 --2- 172.21.15.27:0/3061280672 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8159260 0x7f58d8163b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dd966640 1 -- 172.21.15.27:0/3061280672 >> v1:172.21.15.27:6789/0 conn(0x7f58d81537f0 legacy=0x7f58d80c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.053+0000 7f58dd966640 1 -- 172.21.15.27:0/3061280672 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d80c7fb0 con 0x7f58d8153410 2024-04-20T11:01:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.054+0000 7f58dd966640 1 --2- 172.21.15.27:0/3061280672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d8153410 0x7f58d815d170 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009870 tx=0x7f58d402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ed5475c0d64458 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.054+0000 7f58dc964640 1 -- 172.21.15.27:0/3061280672 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d402f840 con 0x7f58d8153410 2024-04-20T11:01:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.054+0000 7f58dc964640 1 -- 172.21.15.27:0/3061280672 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d402f9e0 con 0x7f58d8153410 2024-04-20T11:01:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.054+0000 7f58dc964640 1 -- 172.21.15.27:0/3061280672 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d402fdb0 con 0x7f58d8153410 2024-04-20T11:01:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.055+0000 7f58dfbca640 1 -- 172.21.15.27:0/3061280672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d8153410 msgr2=0x7f58d815d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.055+0000 7f58dfbca640 1 --2- 172.21.15.27:0/3061280672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d8153410 0x7f58d815d170 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009870 tx=0x7f58d402ba20 comp rx=0 tx=0).stop 2024-04-20T11:01:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.055+0000 7f58dfbca640 1 -- 172.21.15.27:0/3061280672 shutdown_connections 2024-04-20T11:01:46.055 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.055+0000 7f58dfbca640 1 --2- 172.21.15.27:0/3061280672 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8159260 0x7f58d8163b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.055+0000 7f58dfbca640 1 --2- 172.21.15.27:0/3061280672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d8153410 0x7f58d815d170 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.055+0000 7f58dfbca640 1 -- 172.21.15.27:0/3061280672 >> 172.21.15.27:0/3061280672 conn(0x7f58d80ccbd0 msgr2=0x7f58d80ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:46.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.055+0000 7f58dfbca640 1 -- 172.21.15.27:0/3061280672 shutdown_connections 2024-04-20T11:01:46.056 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.055+0000 7f58dfbca640 1 -- 172.21.15.27:0/3061280672 wait complete. 2024-04-20T11:01:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.056+0000 7f58dfbca640 1 Processor -- start 2024-04-20T11:01:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.056+0000 7f58dfbca640 1 -- start start 2024-04-20T11:01:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dfbca640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8153410 0x7f58d815b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dfbca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d81537f0 0x7f58d8153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dfbca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58d8159260 0x7f58d8154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dfbca640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58d815cc50 con 0x7f58d81537f0 2024-04-20T11:01:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dfbca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f58d815c970 con 0x7f58d8159260 2024-04-20T11:01:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dfbca640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f58d80c75d0 con 0x7f58d8153410 2024-04-20T11:01:46.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd165640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d81537f0 0x7f58d8153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd165640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d81537f0 0x7f58d8153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52076/0 (socket says 172.21.15.27:52076) 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd165640 1 -- 172.21.15.27:0/2114792910 learned_addr learned my addr 172.21.15.27:0/2114792910 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd966640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8153410 0x7f58d815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58de167640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58d8159260 0x7f58d8154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd165640 1 -- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8153410 msgr2=0x7f58d815b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd165640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8153410 0x7f58d815b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd165640 1 -- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58d8159260 msgr2=0x7f58d8154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd165640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58d8159260 0x7f58d8154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd165640 1 -- 172.21.15.27:0/2114792910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d81ef2b0 con 0x7f58d81537f0 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd966640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8153410 0x7f58d815b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58de167640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58d8159260 0x7f58d8154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dd165640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d81537f0 0x7f58d8153bd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c430 tx=0x7f58c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58ce7fc640 1 -- 172.21.15.27:0/2114792910 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80052d0 con 0x7f58d81537f0 2024-04-20T11:01:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58ce7fc640 1 -- 172.21.15.27:0/2114792910 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c8005490 con 0x7f58d81537f0 2024-04-20T11:01:46.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d81ef590 con 0x7f58d81537f0 2024-04-20T11:01:46.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.057+0000 7f58ce7fc640 1 -- 172.21.15.27:0/2114792910 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8007260 con 0x7f58d81537f0 2024-04-20T11:01:46.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.058+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d81efb50 con 0x7f58d81537f0 2024-04-20T11:01:46.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.058+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d804fcf0 con 0x7f58d81537f0 2024-04-20T11:01:46.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.059+0000 7f58ce7fc640 1 -- 172.21.15.27:0/2114792910 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58c8007400 con 0x7f58d81537f0 2024-04-20T11:01:46.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.059+0000 7f58ce7fc640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f58bc06c8f0 0x7f58bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:46.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.060+0000 7f58ce7fc640 1 -- 172.21.15.27:0/2114792910 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f58c808b2d0 con 0x7f58d81537f0 2024-04-20T11:01:46.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.060+0000 7f58dd966640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f58bc06c8f0 0x7f58bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:46.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.060+0000 7f58dd966640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f58bc06c8f0 0x7f58bc06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f58d4036000 tx=0x7f58d4008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:46.063 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.064+0000 7f58ce7fc640 1 -- 172.21.15.27:0/2114792910 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f58c80594a0 con 0x7f58d81537f0 2024-04-20T11:01:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.288+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58d80c9050 con 0x7f58d81537f0 2024-04-20T11:01:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.289+0000 7f58ce7fc640 1 -- 172.21.15.27:0/2114792910 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7f58c8058da0 con 0x7f58d81537f0 2024-04-20T11:01:46.289 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:46.289 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:46.289 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f58bc06c8f0 msgr2=0x7f58bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f58bc06c8f0 0x7f58bc06edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f58d4036000 tx=0x7f58d4008030 comp rx=0 tx=0).stop 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d81537f0 msgr2=0x7f58d8153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d81537f0 0x7f58d8153bd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c430 tx=0x7f58c800c8f0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 shutdown_connections 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f58bc06c8f0 0x7f58bc06edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58d8159260 0x7f58d8154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58d81537f0 0x7f58d8153bd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 --2- 172.21.15.27:0/2114792910 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f58d8153410 0x7f58d815b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 >> 172.21.15.27:0/2114792910 conn(0x7f58d80ccbd0 msgr2=0x7f58d80cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:46.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 shutdown_connections 2024-04-20T11:01:46.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:46.291+0000 7f58dfbca640 1 -- 172.21.15.27:0/2114792910 wait complete. 2024-04-20T11:01:46.301 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T11:01:48.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:48.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.423+0000 7f8d0b08c640 1 Processor -- start 2024-04-20T11:01:48.423 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.423+0000 7f8d0b08c640 1 -- start start 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.424+0000 7f8d0b08c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8d040b54f0 0x7f8d040b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.424+0000 7f8d0b08c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040d0c90 0x7f8d040d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.424+0000 7f8d0b08c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d040b5e70 con 0x7f8d04155950 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.424+0000 7f8d0b08c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f8d0415b9c0 con 0x7f8d040b54f0 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.424+0000 7f8d0b08c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0415bca0 con 0x7f8d040d0c90 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d09629640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040d0c90 0x7f8d040d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d03fff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8d040b54f0 0x7f8d040b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d08e28640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f8d04155950 0x7f8d040cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:36614/0 (socket says 172.21.15.27:36614) 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d08e28640 1 -- 172.21.15.27:0/2125757732 learned_addr learned my addr 172.21.15.27:0/2125757732 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d037fe640 1 -- 172.21.15.27:0/2125757732 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743448464 0 0) 0x7f8d040b5e70 con 0x7f8d04155950 2024-04-20T11:01:48.425 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d037fe640 1 -- 172.21.15.27:0/2125757732 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cf4003520 con 0x7f8d04155950 2024-04-20T11:01:48.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d09629640 1 -- 172.21.15.27:0/2125757732 >> v2:172.21.15.137:3300/0 conn(0x7f8d040b54f0 msgr2=0x7f8d040b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:48.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d09629640 1 --2- 172.21.15.27:0/2125757732 >> v2:172.21.15.137:3300/0 conn(0x7f8d040b54f0 0x7f8d040b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:48.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d09629640 1 -- 172.21.15.27:0/2125757732 >> v1:172.21.15.137:6789/0 conn(0x7f8d04155950 legacy=0x7f8d040cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:48.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.425+0000 7f8d09629640 1 -- 172.21.15.27:0/2125757732 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0415bf80 con 0x7f8d040d0c90 2024-04-20T11:01:48.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.426+0000 7f8d09629640 1 --2- 172.21.15.27:0/2125757732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040d0c90 0x7f8d040d10e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009d90 tx=0x7f8cfc02bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93dbaac6a5e55416 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:48.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.426+0000 7f8d037fe640 1 -- 172.21.15.27:0/2125757732 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc039070 con 0x7f8d040d0c90 2024-04-20T11:01:48.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.426+0000 7f8d037fe640 1 -- 172.21.15.27:0/2125757732 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc02cce0 con 0x7f8d040d0c90 2024-04-20T11:01:48.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.427+0000 7f8d037fe640 1 -- 172.21.15.27:0/2125757732 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc034640 con 0x7f8d040d0c90 2024-04-20T11:01:48.427 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.427+0000 7f8d0b08c640 1 -- 172.21.15.27:0/2125757732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040d0c90 msgr2=0x7f8d040d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:48.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.427+0000 7f8d0b08c640 1 --2- 172.21.15.27:0/2125757732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040d0c90 0x7f8d040d10e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009d90 tx=0x7f8cfc02bf30 comp rx=0 tx=0).stop 2024-04-20T11:01:48.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.427+0000 7f8d0b08c640 1 -- 172.21.15.27:0/2125757732 shutdown_connections 2024-04-20T11:01:48.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.427+0000 7f8d0b08c640 1 --2- 172.21.15.27:0/2125757732 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040d0c90 0x7f8d040d10e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:48.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.427+0000 7f8d0b08c640 1 --2- 172.21.15.27:0/2125757732 >> v2:172.21.15.137:3300/0 conn(0x7f8d040b54f0 0x7f8d040b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:48.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.427+0000 7f8d0b08c640 1 -- 172.21.15.27:0/2125757732 >> 172.21.15.27:0/2125757732 conn(0x7f8d040676c0 msgr2=0x7f8d04067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:48.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.427+0000 7f8d0b08c640 1 -- 172.21.15.27:0/2125757732 shutdown_connections 2024-04-20T11:01:48.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.427+0000 7f8d0b08c640 1 -- 172.21.15.27:0/2125757732 wait complete. 2024-04-20T11:01:48.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.429+0000 7f8d0b08c640 1 Processor -- start 2024-04-20T11:01:48.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.429+0000 7f8d0b08c640 1 -- start start 2024-04-20T11:01:48.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.429+0000 7f8d0b08c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040b54f0 0x7f8d041540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:48.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.429+0000 7f8d0b08c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d040d0c90 0x7f8d040c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:48.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.429+0000 7f8d0b08c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d04155950 0x7f8d040ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d0b08c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8d040d1620 con 0x7f8d04155950 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d0b08c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0415c540 con 0x7f8d040d0c90 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d0b08c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0415fb90 con 0x7f8d040b54f0 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d09629640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d04155950 0x7f8d040ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d09629640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d04155950 0x7f8d040ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52084/0 (socket says 172.21.15.27:52084) 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d09629640 1 -- 172.21.15.27:0/4203755126 learned_addr learned my addr 172.21.15.27:0/4203755126 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d08e28640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040b54f0 0x7f8d041540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d03fff640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d040d0c90 0x7f8d040c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d09629640 1 -- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040b54f0 msgr2=0x7f8d041540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d09629640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040b54f0 0x7f8d041540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d09629640 1 -- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d040d0c90 msgr2=0x7f8d040c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d09629640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d040d0c90 0x7f8d040c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d09629640 1 -- 172.21.15.27:0/4203755126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0415b090 con 0x7f8d04155950 2024-04-20T11:01:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.430+0000 7f8d09629640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d04155950 0x7f8d040ca200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc037040 tx=0x7f8cfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.431+0000 7f8d017fa640 1 -- 172.21.15.27:0/4203755126 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc042030 con 0x7f8d04155950 2024-04-20T11:01:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.431+0000 7f8d017fa640 1 -- 172.21.15.27:0/4203755126 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc03a040 con 0x7f8d04155950 2024-04-20T11:01:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.431+0000 7f8d017fa640 1 -- 172.21.15.27:0/4203755126 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc039070 con 0x7f8d04155950 2024-04-20T11:01:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.431+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0415b370 con 0x7f8d04155950 2024-04-20T11:01:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.431+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d040c94c0 con 0x7f8d04155950 2024-04-20T11:01:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.432+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d040b4070 con 0x7f8d04155950 2024-04-20T11:01:48.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.433+0000 7f8d017fa640 1 -- 172.21.15.27:0/4203755126 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cfc045070 con 0x7f8d04155950 2024-04-20T11:01:48.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.434+0000 7f8d017fa640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ce806c8f0 0x7f8ce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:48.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.434+0000 7f8d017fa640 1 -- 172.21.15.27:0/4203755126 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f8cfc0ae140 con 0x7f8d04155950 2024-04-20T11:01:48.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.434+0000 7f8d08e28640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ce806c8f0 0x7f8ce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:48.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.435+0000 7f8d08e28640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ce806c8f0 0x7f8ce806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8005880 tx=0x7f8cf8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.438+0000 7f8d017fa640 1 -- 172.21.15.27:0/4203755126 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8d040b4070 con 0x7f8d04155950 2024-04-20T11:01:48.670 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.671+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d0404fcf0 con 0x7f8d04155950 2024-04-20T11:01:48.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.671+0000 7f8d017fa640 1 -- 172.21.15.27:0/4203755126 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7f8cfc04c070 con 0x7f8d04155950 2024-04-20T11:01:48.671 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:48.671 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:48.672 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ce806c8f0 msgr2=0x7f8ce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ce806c8f0 0x7f8ce806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8005880 tx=0x7f8cf8009040 comp rx=0 tx=0).stop 2024-04-20T11:01:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d04155950 msgr2=0x7f8d040ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d04155950 0x7f8d040ca200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc037040 tx=0x7f8cfc004a40 comp rx=0 tx=0).stop 2024-04-20T11:01:48.673 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 shutdown_connections 2024-04-20T11:01:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8ce806c8f0 0x7f8ce806edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d04155950 0x7f8d040ca200 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8d040d0c90 0x7f8d040c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 --2- 172.21.15.27:0/4203755126 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8d040b54f0 0x7f8d041540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 >> 172.21.15.27:0/4203755126 conn(0x7f8d040676c0 msgr2=0x7f8d0406d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.673+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 shutdown_connections 2024-04-20T11:01:48.674 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:48.674+0000 7f8d0b08c640 1 -- 172.21.15.27:0/4203755126 wait complete. 2024-04-20T11:01:48.685 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T11:01:50.686 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.793+0000 7f5a15ed2640 1 Processor -- start 2024-04-20T11:01:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.794+0000 7f5a15ed2640 1 -- start start 2024-04-20T11:01:50.794 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.794+0000 7f5a15ed2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a10152260 0x7f5a100719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.794+0000 7f5a15ed2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a1015ba80 con 0x7f5a100cd6f0 2024-04-20T11:01:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.794+0000 7f5a15ed2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a1015bd60 con 0x7f5a10152640 2024-04-20T11:01:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.794+0000 7f5a15ed2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5a1015c040 con 0x7f5a10152260 2024-04-20T11:01:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.795+0000 7f5a0f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a10152260 0x7f5a100719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.795+0000 7f5a0f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a10152260 0x7f5a100719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52092/0 (socket says 172.21.15.27:52092) 2024-04-20T11:01:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.795+0000 7f5a0f7fe640 1 -- 172.21.15.27:0/2013914165 learned_addr learned my addr 172.21.15.27:0/2013914165 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.795+0000 7f5a0f7fe640 1 -- 172.21.15.27:0/2013914165 >> v1:172.21.15.137:6789/0 conn(0x7f5a10152640 legacy=0x7f5a100c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.795+0000 7f5a0f7fe640 1 -- 172.21.15.27:0/2013914165 >> v1:172.21.15.27:6789/0 conn(0x7f5a100cd6f0 legacy=0x7f5a10158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:50.795 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.795+0000 7f5a0f7fe640 1 -- 172.21.15.27:0/2013914165 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1015c320 con 0x7f5a10152260 2024-04-20T11:01:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a0f7fe640 1 --2- 172.21.15.27:0/2013914165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a10152260 0x7f5a100719c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009870 tx=0x7f5a0402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e82869f9ca13863 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a0e7fc640 1 -- 172.21.15.27:0/2013914165 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0402f840 con 0x7f5a10152260 2024-04-20T11:01:50.796 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a0e7fc640 1 -- 172.21.15.27:0/2013914165 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0402f9e0 con 0x7f5a10152260 2024-04-20T11:01:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a0e7fc640 1 -- 172.21.15.27:0/2013914165 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0402fdb0 con 0x7f5a10152260 2024-04-20T11:01:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2013914165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a10152260 msgr2=0x7f5a100719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a15ed2640 1 --2- 172.21.15.27:0/2013914165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a10152260 0x7f5a100719c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009870 tx=0x7f5a0402ba20 comp rx=0 tx=0).stop 2024-04-20T11:01:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2013914165 shutdown_connections 2024-04-20T11:01:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a15ed2640 1 --2- 172.21.15.27:0/2013914165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a10152260 0x7f5a100719c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2013914165 >> 172.21.15.27:0/2013914165 conn(0x7f5a100ccbd0 msgr2=0x7f5a100ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2013914165 shutdown_connections 2024-04-20T11:01:50.797 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.796+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2013914165 wait complete. 2024-04-20T11:01:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.798+0000 7f5a15ed2640 1 Processor -- start 2024-04-20T11:01:50.798 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.798+0000 7f5a15ed2640 1 -- start start 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.798+0000 7f5a15ed2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a100cd6f0 0x7f5a101592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.798+0000 7f5a15ed2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a10152260 0x7f5a101f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a15ed2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a10152640 0x7f5a10147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a15ed2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1015c600 con 0x7f5a100cd6f0 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a15ed2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10074260 con 0x7f5a10152260 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a15ed2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1015b720 con 0x7f5a10152640 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a100cd6f0 0x7f5a101592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a100cd6f0 0x7f5a101592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52096/0 (socket says 172.21.15.27:52096) 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0f7fe640 1 -- 172.21.15.27:0/2161991329 learned_addr learned my addr 172.21.15.27:0/2161991329 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0f7fe640 1 -- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a10152640 msgr2=0x7f5a10147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0effd640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a10152260 0x7f5a101f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0ffff640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a10152640 0x7f5a10147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:50.800 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0f7fe640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a10152640 0x7f5a10147a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0f7fe640 1 -- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a10152260 msgr2=0x7f5a101f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0f7fe640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a10152260 0x7f5a101f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0f7fe640 1 -- 172.21.15.27:0/2161991329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a101f1420 con 0x7f5a100cd6f0 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0effd640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a10152260 0x7f5a101f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a0f7fe640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a100cd6f0 0x7f5a101592f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5a0402ba60 tx=0x7f5a04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f59f7fff640 1 -- 172.21.15.27:0/2161991329 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04047070 con 0x7f5a100cd6f0 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f59f7fff640 1 -- 172.21.15.27:0/2161991329 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a04042480 con 0x7f5a100cd6f0 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a10159980 con 0x7f5a100cd6f0 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f59f7fff640 1 -- 172.21.15.27:0/2161991329 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04042870 con 0x7f5a100cd6f0 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.799+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a10159f40 con 0x7f5a100cd6f0 2024-04-20T11:01:50.801 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.800+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a100c4290 con 0x7f5a100cd6f0 2024-04-20T11:01:50.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.801+0000 7f59f7fff640 1 -- 172.21.15.27:0/2161991329 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a0404a070 con 0x7f5a100cd6f0 2024-04-20T11:01:50.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.802+0000 7f59f7fff640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f59fc06c920 0x7f59fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:50.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.802+0000 7f59f7fff640 1 -- 172.21.15.27:0/2161991329 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6169+0+0 (secure 0 0 0) 0x7f5a040ae180 con 0x7f5a100cd6f0 2024-04-20T11:01:50.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.802+0000 7f5a0effd640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f59fc06c920 0x7f59fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:50.802 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.803+0000 7f5a0effd640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f59fc06c920 0x7f59fc06ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5a101465f0 tx=0x7f5a00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:50.806 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:50.807+0000 7f59f7fff640 1 -- 172.21.15.27:0/2161991329 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5a100c4290 con 0x7f5a100cd6f0 2024-04-20T11:01:51.035 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.035+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a1015a180 con 0x7f5a100cd6f0 2024-04-20T11:01:51.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.035+0000 7f59f7fff640 1 -- 172.21.15.27:0/2161991329 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+6938 (secure 0 0 0) 0x7f5a101f1420 con 0x7f5a100cd6f0 2024-04-20T11:01:51.036 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:51.036 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5406,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:06.567284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4278,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.27:6836/4195543922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":4195543922},{"type":"v1","addr":"172.21.15.27:6836","nonce":4195543922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293,4278]},"id":1}]} 2024-04-20T11:01:51.036 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 41 2024-04-20T11:01:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f59fc06c920 msgr2=0x7f59fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f59fc06c920 0x7f59fc06ede0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5a101465f0 tx=0x7f5a00009040 comp rx=0 tx=0).stop 2024-04-20T11:01:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a100cd6f0 msgr2=0x7f5a101592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a100cd6f0 0x7f5a101592f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5a0402ba60 tx=0x7f5a04004730 comp rx=0 tx=0).stop 2024-04-20T11:01:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 shutdown_connections 2024-04-20T11:01:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f59fc06c920 0x7f59fc06ede0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a10152640 0x7f5a10147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a10152260 0x7f5a101f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 --2- 172.21.15.27:0/2161991329 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a100cd6f0 0x7f5a101592f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 >> 172.21.15.27:0/2161991329 conn(0x7f5a100ccbd0 msgr2=0x7f5a100c5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 shutdown_connections 2024-04-20T11:01:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:51.037+0000 7f5a15ed2640 1 -- 172.21.15.27:0/2161991329 wait complete. 2024-04-20T11:01:51.049 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-04-20T11:01:53.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:53.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.165+0000 7f2332a06640 1 Processor -- start 2024-04-20T11:01:53.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.165+0000 7f2332a06640 1 -- start start 2024-04-20T11:01:53.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f2332a06640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f232c155950 0x7f232c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f2332a06640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 0x7f232c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f2332a06640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f232c0b5a30 0x7f232c0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f2332a06640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f232c0b5ea0 con 0x7f232c155950 2024-04-20T11:01:53.167 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f2332a06640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f232c15b9c0 con 0x7f232c0b5a30 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f2332a06640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f232c15bca0 con 0x7f232c0cc330 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f232bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f232c155950 0x7f232c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f232bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f232c155950 0x7f232c0cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52106/0 (socket says 172.21.15.27:52106) 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f232bfff640 1 -- 172.21.15.27:0/1332102023 learned_addr learned my addr 172.21.15.27:0/1332102023 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f2330fa3640 1 --2- 172.21.15.27:0/1332102023 >> v2:172.21.15.137:3300/0 conn(0x7f232c0b5a30 0x7f232c0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f232b7fe640 1 --2- 172.21.15.27:0/1332102023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 0x7f232c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f232bfff640 1 -- 172.21.15.27:0/1332102023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 msgr2=0x7f232c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f232bfff640 1 --2- 172.21.15.27:0/1332102023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 0x7f232c0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f232bfff640 1 -- 172.21.15.27:0/1332102023 >> v2:172.21.15.137:3300/0 conn(0x7f232c0b5a30 msgr2=0x7f232c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f232bfff640 1 --2- 172.21.15.27:0/1332102023 >> v2:172.21.15.137:3300/0 conn(0x7f232c0b5a30 0x7f232c0d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.168 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.166+0000 7f232bfff640 1 -- 172.21.15.27:0/1332102023 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c15bf80 con 0x7f232c155950 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.167+0000 7f232bfff640 1 --2- 172.21.15.27:0/1332102023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 0x7f232c0cbdf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f232c08e4b0 tx=0x7f231c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8130d1ffd394c26a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.167+0000 7f232affd640 1 -- 172.21.15.27:0/1332102023 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c034070 con 0x7f232c155950 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.167+0000 7f232affd640 1 -- 172.21.15.27:0/1332102023 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c030860 con 0x7f232c155950 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.167+0000 7f232affd640 1 -- 172.21.15.27:0/1332102023 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c030c30 con 0x7f232c155950 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.168+0000 7f2332a06640 1 -- 172.21.15.27:0/1332102023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 msgr2=0x7f232c0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.168+0000 7f2332a06640 1 --2- 172.21.15.27:0/1332102023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 0x7f232c0cbdf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f232c08e4b0 tx=0x7f231c02b9a0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.168+0000 7f2332a06640 1 -- 172.21.15.27:0/1332102023 shutdown_connections 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.168+0000 7f2332a06640 1 --2- 172.21.15.27:0/1332102023 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 0x7f232c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.168+0000 7f2332a06640 1 --2- 172.21.15.27:0/1332102023 >> v2:172.21.15.137:3300/0 conn(0x7f232c0b5a30 0x7f232c0d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.168+0000 7f2332a06640 1 --2- 172.21.15.27:0/1332102023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 0x7f232c0cbdf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.170 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.168+0000 7f2332a06640 1 -- 172.21.15.27:0/1332102023 >> 172.21.15.27:0/1332102023 conn(0x7f232c0676c0 msgr2=0x7f232c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:53.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.168+0000 7f2332a06640 1 -- 172.21.15.27:0/1332102023 shutdown_connections 2024-04-20T11:01:53.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.168+0000 7f2332a06640 1 -- 172.21.15.27:0/1332102023 wait complete. 2024-04-20T11:01:53.171 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.169+0000 7f2332a06640 1 Processor -- start 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2332a06640 1 -- start start 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2332a06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f232c0b5a30 0x7f232c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2332a06640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 0x7f232c0c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2332a06640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 0x7f232c0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2332a06640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f232c15fb90 con 0x7f232c155950 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2332a06640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f232c15c540 con 0x7f232c0b5a30 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2332a06640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f232c15fd10 con 0x7f232c0cc330 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2330fa3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 0x7f232c0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2330fa3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 0x7f232c0c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52120/0 (socket says 172.21.15.27:52120) 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f2330fa3640 1 -- 172.21.15.27:0/1458866022 learned_addr learned my addr 172.21.15.27:0/1458866022 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f232bfff640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f232c0b5a30 0x7f232c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.173 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.170+0000 7f232b7fe640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 0x7f232c0c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2330fa3640 1 -- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 msgr2=0x7f232c0c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2330fa3640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 0x7f232c0c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2330fa3640 1 -- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f232c0b5a30 msgr2=0x7f232c15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2330fa3640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f232c0b5a30 0x7f232c15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2330fa3640 1 -- 172.21.15.27:0/1458866022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c0c7480 con 0x7f232c155950 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2330fa3640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 0x7f232c0c64e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f232400eb20 tx=0x7f232400b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2328ff9640 1 -- 172.21.15.27:0/1458866022 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324018020 con 0x7f232c155950 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2328ff9640 1 -- 172.21.15.27:0/1458866022 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232400bc70 con 0x7f232c155950 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2328ff9640 1 -- 172.21.15.27:0/1458866022 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324014640 con 0x7f232c155950 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c1f2530 con 0x7f232c155950 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.171+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c15a0f0 con 0x7f232c155950 2024-04-20T11:01:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.172+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c15f7d0 con 0x7f232c155950 2024-04-20T11:01:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.173+0000 7f2328ff9640 1 -- 172.21.15.27:0/1458866022 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f232401b020 con 0x7f232c155950 2024-04-20T11:01:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.173+0000 7f2328ff9640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f231806c8f0 0x7f231806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.173+0000 7f2328ff9640 1 -- 172.21.15.27:0/1458866022 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f232408c4c0 con 0x7f232c155950 2024-04-20T11:01:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.173+0000 7f232bfff640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f231806c8f0 0x7f231806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.174+0000 7f232bfff640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f231806c8f0 0x7f231806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f231c036000 tx=0x7f231c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.178+0000 7f2328ff9640 1 -- 172.21.15.27:0/1458866022 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f232c15f7d0 con 0x7f232c155950 2024-04-20T11:01:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.427+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f232c04fcf0 con 0x7f232c155950 2024-04-20T11:01:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.428+0000 7f2328ff9640 1 -- 172.21.15.27:0/1458866022 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6109 (secure 0 0 0) 0x7f2324022020 con 0x7f232c155950 2024-04-20T11:01:53.429 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:53.429 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:52.359346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:replay","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4894,5029,5143]},"id":1}]} 2024-04-20T11:01:53.429 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 42 2024-04-20T11:01:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f231806c8f0 msgr2=0x7f231806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f231806c8f0 0x7f231806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f231c036000 tx=0x7f231c03b040 comp rx=0 tx=0).stop 2024-04-20T11:01:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 msgr2=0x7f232c0c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 0x7f232c0c64e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f232400eb20 tx=0x7f232400b0b0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 shutdown_connections 2024-04-20T11:01:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f231806c8f0 0x7f231806edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f232c155950 0x7f232c0c64e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f232c0cc330 0x7f232c0c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 --2- 172.21.15.27:0/1458866022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f232c0b5a30 0x7f232c15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 >> 172.21.15.27:0/1458866022 conn(0x7f232c0676c0 msgr2=0x7f232c0cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 shutdown_connections 2024-04-20T11:01:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.430+0000 7f2332a06640 1 -- 172.21.15.27:0/1458866022 wait complete. 2024-04-20T11:01:53.442 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-04-20T11:01:53.442 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-04-20T11:01:53.442 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:53.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.543+0000 7f83a930d640 1 Processor -- start 2024-04-20T11:01:53.544 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.543+0000 7f83a930d640 1 -- start start 2024-04-20T11:01:53.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.544+0000 7f83a930d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 0x7f83a4072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.544+0000 7f83a930d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f83a406f9f0 0x7f83a4072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.544+0000 7f83a930d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a40c6d40 con 0x7f83a406f610 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.544+0000 7f83a930d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f83a406cbe0 con 0x7f83a406f9f0 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.544+0000 7f83a930d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f83a40c6fe0 con 0x7f83a4153780 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.544+0000 7f83a2ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f83a406f610 0x7f83a4153070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39590/0 (socket says 172.21.15.27:39590) 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.544+0000 7f83a2ffd640 1 -- 172.21.15.27:0/1285583344 learned_addr learned my addr 172.21.15.27:0/1285583344 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.544+0000 7f83a37fe640 1 --2- 172.21.15.27:0/1285583344 >> v2:172.21.15.137:3300/0 conn(0x7f83a406f9f0 0x7f83a4072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.544+0000 7f83a27fc640 1 --2- 172.21.15.27:0/1285583344 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 0x7f83a4072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.545+0000 7f83a1ffb640 1 -- 172.21.15.27:0/1285583344 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1676965438 0 0) 0x7f83a40c6d40 con 0x7f83a406f610 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.545+0000 7f83a1ffb640 1 -- 172.21.15.27:0/1285583344 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f838c003520 con 0x7f83a406f610 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.545+0000 7f83a27fc640 1 -- 172.21.15.27:0/1285583344 >> v2:172.21.15.137:3300/0 conn(0x7f83a406f9f0 msgr2=0x7f83a4072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.545+0000 7f83a27fc640 1 --2- 172.21.15.27:0/1285583344 >> v2:172.21.15.137:3300/0 conn(0x7f83a406f9f0 0x7f83a4072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.545+0000 7f83a27fc640 1 -- 172.21.15.27:0/1285583344 >> v1:172.21.15.27:6789/0 conn(0x7f83a406f610 legacy=0x7f83a4153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.545+0000 7f83a27fc640 1 -- 172.21.15.27:0/1285583344 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a40c75c0 con 0x7f83a4153780 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.545+0000 7f83a37fe640 1 --2- 172.21.15.27:0/1285583344 >> v2:172.21.15.137:3300/0 conn(0x7f83a406f9f0 0x7f83a4072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a27fc640 1 --2- 172.21.15.27:0/1285583344 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 0x7f83a4072200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8398009790 tx=0x7f839802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54241bc04ca4e40d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a1ffb640 1 -- 172.21.15.27:0/1285583344 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398035020 con 0x7f83a4153780 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a1ffb640 1 -- 172.21.15.27:0/1285583344 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8398031890 con 0x7f83a4153780 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a1ffb640 1 -- 172.21.15.27:0/1285583344 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398031c60 con 0x7f83a4153780 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a930d640 1 -- 172.21.15.27:0/1285583344 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 msgr2=0x7f83a4072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a930d640 1 --2- 172.21.15.27:0/1285583344 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 0x7f83a4072200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8398009790 tx=0x7f839802b930 comp rx=0 tx=0).stop 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a930d640 1 -- 172.21.15.27:0/1285583344 shutdown_connections 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a930d640 1 --2- 172.21.15.27:0/1285583344 >> v2:172.21.15.137:3300/0 conn(0x7f83a406f9f0 0x7f83a4072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a930d640 1 --2- 172.21.15.27:0/1285583344 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 0x7f83a4072200 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a930d640 1 -- 172.21.15.27:0/1285583344 >> 172.21.15.27:0/1285583344 conn(0x7f83a406d570 msgr2=0x7f83a406d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a930d640 1 -- 172.21.15.27:0/1285583344 shutdown_connections 2024-04-20T11:01:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.546+0000 7f83a930d640 1 -- 172.21.15.27:0/1285583344 wait complete. 2024-04-20T11:01:53.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.548+0000 7f83a930d640 1 Processor -- start 2024-04-20T11:01:53.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.548+0000 7f83a930d640 1 -- start start 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.548+0000 7f83a930d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a406f610 0x7f83a40c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.548+0000 7f83a930d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a406f9f0 0x7f83a4144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.548+0000 7f83a930d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 0x7f83a41448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.548+0000 7f83a930d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f83a406c590 con 0x7f83a406f9f0 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.548+0000 7f83a930d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f83a415d620 con 0x7f83a406f610 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.548+0000 7f83a930d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f83a406c870 con 0x7f83a4153780 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a27fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a406f9f0 0x7f83a4144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a406f610 0x7f83a40c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a406f610 0x7f83a40c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:33270/0 (socket says 172.21.15.27:33270) 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a2ffd640 1 -- 172.21.15.27:0/3403487297 learned_addr learned my addr 172.21.15.27:0/3403487297 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a37fe640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 0x7f83a41448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a2ffd640 1 -- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 msgr2=0x7f83a41448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a2ffd640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 0x7f83a41448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.555 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a2ffd640 1 -- 172.21.15.27:0/3403487297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a406f9f0 msgr2=0x7f83a4144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a2ffd640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a406f9f0 0x7f83a4144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a2ffd640 1 -- 172.21.15.27:0/3403487297 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a41f23e0 con 0x7f83a406f610 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a2ffd640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a406f610 0x7f83a40c9210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f83940010e0 tx=0x7f839400c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f838b7fe640 1 -- 172.21.15.27:0/3403487297 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394017070 con 0x7f83a406f610 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f838b7fe640 1 -- 172.21.15.27:0/3403487297 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8394005d00 con 0x7f83a406f610 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f838b7fe640 1 -- 172.21.15.27:0/3403487297 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394007310 con 0x7f83a406f610 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a41f26a0 con 0x7f83a406f610 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.549+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a41f2c60 con 0x7f83a406f610 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.550+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a404fcf0 con 0x7f83a406f610 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.551+0000 7f838b7fe640 1 -- 172.21.15.27:0/3403487297 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f839401a070 con 0x7f83a406f610 2024-04-20T11:01:53.556 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.551+0000 7f838b7fe640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f839006c8f0 0x7f839006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:53.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.551+0000 7f838b7fe640 1 -- 172.21.15.27:0/3403487297 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f839408b240 con 0x7f83a406f610 2024-04-20T11:01:53.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.552+0000 7f83a27fc640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f839006c8f0 0x7f839006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:53.557 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.552+0000 7f83a27fc640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f839006c8f0 0x7f839006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8398036000 tx=0x7f8398008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:53.559 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.558+0000 7f838b7fe640 1 -- 172.21.15.27:0/3403487297 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8394058c50 con 0x7f83a406f610 2024-04-20T11:01:53.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.786+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83a415c8e0 con 0x7f83a406f610 2024-04-20T11:01:53.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.787+0000 7f838b7fe640 1 -- 172.21.15.27:0/3403487297 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6109 (secure 0 0 0) 0x7f839405cf50 con 0x7f83a406f610 2024-04-20T11:01:53.788 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:53.788 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:52.359346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:replay","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4894,5029,5143]},"id":1}]} 2024-04-20T11:01:53.788 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 42 2024-04-20T11:01:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f839006c8f0 msgr2=0x7f839006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f839006c8f0 0x7f839006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8398036000 tx=0x7f8398008030 comp rx=0 tx=0).stop 2024-04-20T11:01:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a406f610 msgr2=0x7f83a40c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a406f610 0x7f83a40c9210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f83940010e0 tx=0x7f839400c890 comp rx=0 tx=0).stop 2024-04-20T11:01:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 shutdown_connections 2024-04-20T11:01:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f839006c8f0 0x7f839006edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f83a4153780 0x7f83a41448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83a406f9f0 0x7f83a4144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 --2- 172.21.15.27:0/3403487297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a406f610 0x7f83a40c9210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 >> 172.21.15.27:0/3403487297 conn(0x7f83a406d570 msgr2=0x7f83a406d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 shutdown_connections 2024-04-20T11:01:53.791 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:53.789+0000 7f83a930d640 1 -- 172.21.15.27:0/3403487297 wait complete. 2024-04-20T11:01:53.802 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 42, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 42, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4894": { "addr": "172.21.15.137:6838/2896327439", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6836", "nonce": 2896327439, "type": "v2" }, { "addr": "172.21.15.137:6838", "nonce": 2896327439, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4894, "incarnation": 24, "join_fscid": -1, "name": "b", "rank": 4, "state": "up:active", "state_seq": 17 }, "gid_5029": { "addr": "172.21.15.137:6835/4235742413", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6834", "nonce": 4235742413, "type": "v2" }, { "addr": "172.21.15.137:6835", "nonce": 4235742413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5029, "incarnation": 30, "join_fscid": -1, "name": "f", "rank": 3, "state": "up:active", "state_seq": 22 }, "gid_5143": { "addr": "172.21.15.27:6835/2020174232", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6832", "nonce": 2020174232, "type": "v2" }, { "addr": "172.21.15.27:6835", "nonce": 2020174232, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5143, "incarnation": 36, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 20 }, "gid_5406": { "addr": "172.21.15.27:6837/3526024786", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6834", "nonce": 3526024786, "type": "v2" }, { "addr": "172.21.15.27:6837", "nonce": 3526024786, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5406, "incarnation": 42, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T11:01:52.359346+0000", "qdb_cluster": [ 4293, 4894, 5029, 5143 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 5406, "mds_2": 5143, "mds_3": 5029, "mds_4": 4894 } } } ], "standbys": [] } 2024-04-20T11:01:55.803 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:55.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.923+0000 7f3964f0e640 1 Processor -- start 2024-04-20T11:01:55.924 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.923+0000 7f3964f0e640 1 -- start start 2024-04-20T11:01:55.925 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.924+0000 7f3964f0e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f39600c8d70 0x7f39601601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.924+0000 7f3964f0e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39600cf510 con 0x7f396006d6a0 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.924+0000 7f3964f0e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39600cf7f0 con 0x7f39600ceb40 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.924+0000 7f3964f0e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f39600cc210 con 0x7f39600c8d70 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.924+0000 7f395dd9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f396006d6a0 0x7f39601449d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36534/0 (socket says 172.21.15.27:36534) 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.924+0000 7f395dd9b640 1 -- 172.21.15.27:0/2980105117 learned_addr learned my addr 172.21.15.27:0/2980105117 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.924+0000 7f395e59c640 1 --2- 172.21.15.27:0/2980105117 >> v2:172.21.15.137:3300/0 conn(0x7f39600c8d70 0x7f39601601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.925+0000 7f395d59a640 1 -- 172.21.15.27:0/2980105117 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096677316 0 0) 0x7f39600cf7f0 con 0x7f39600ceb40 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.925+0000 7f395d59a640 1 -- 172.21.15.27:0/2980105117 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3948003520 con 0x7f39600ceb40 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.925+0000 7f395d59a640 1 -- 172.21.15.27:0/2980105117 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404314675 0 0) 0x7f39600cf510 con 0x7f396006d6a0 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.925+0000 7f395e59c640 1 -- 172.21.15.27:0/2980105117 >> v1:172.21.15.137:6789/0 conn(0x7f39600ceb40 legacy=0x7f3960070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.925+0000 7f395e59c640 1 -- 172.21.15.27:0/2980105117 >> v1:172.21.15.27:6789/0 conn(0x7f396006d6a0 legacy=0x7f39601449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:55.926 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.925+0000 7f395e59c640 1 -- 172.21.15.27:0/2980105117 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f396006cbc0 con 0x7f39600c8d70 2024-04-20T11:01:55.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.926+0000 7f395e59c640 1 --2- 172.21.15.27:0/2980105117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39600c8d70 0x7f39601601c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39500097a0 tx=0x7f395002b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17566eef79c0f952 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:55.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.926+0000 7f395d59a640 1 -- 172.21.15.27:0/2980105117 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395002f810 con 0x7f39600c8d70 2024-04-20T11:01:55.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.926+0000 7f395d59a640 1 -- 172.21.15.27:0/2980105117 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395002f9b0 con 0x7f39600c8d70 2024-04-20T11:01:55.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.926+0000 7f395d59a640 1 -- 172.21.15.27:0/2980105117 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395002fd80 con 0x7f39600c8d70 2024-04-20T11:01:55.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.926+0000 7f3964f0e640 1 -- 172.21.15.27:0/2980105117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39600c8d70 msgr2=0x7f39601601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:55.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.926+0000 7f3964f0e640 1 --2- 172.21.15.27:0/2980105117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39600c8d70 0x7f39601601c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39500097a0 tx=0x7f395002b9f0 comp rx=0 tx=0).stop 2024-04-20T11:01:55.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.927+0000 7f3964f0e640 1 -- 172.21.15.27:0/2980105117 shutdown_connections 2024-04-20T11:01:55.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.927+0000 7f3964f0e640 1 --2- 172.21.15.27:0/2980105117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39600c8d70 0x7f39601601c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:55.928 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.927+0000 7f3964f0e640 1 -- 172.21.15.27:0/2980105117 >> 172.21.15.27:0/2980105117 conn(0x7f396006c070 msgr2=0x7f39600676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:55.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.927+0000 7f3964f0e640 1 -- 172.21.15.27:0/2980105117 shutdown_connections 2024-04-20T11:01:55.929 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.927+0000 7f3964f0e640 1 -- 172.21.15.27:0/2980105117 wait complete. 2024-04-20T11:01:55.930 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.928+0000 7f3964f0e640 1 Processor -- start 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.928+0000 7f3964f0e640 1 -- start start 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f3964f0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f396006d6a0 0x7f3960157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f3964f0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39600c8d70 0x7f39601570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f3964f0e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39600ceb40 0x7f3960157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f3964f0e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f396006d180 con 0x7f39600c8d70 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f3964f0e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3960071310 con 0x7f396006d6a0 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f3964f0e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f3960071490 con 0x7f39600ceb40 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39600c8d70 0x7f39601570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39600c8d70 0x7f39601570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52574/0 (socket says 172.21.15.27:52574) 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f396006d6a0 0x7f3960157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395dd9b640 1 -- 172.21.15.27:0/1801816588 learned_addr learned my addr 172.21.15.27:0/1801816588 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395ed9d640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39600ceb40 0x7f3960157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395dd9b640 1 -- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39600ceb40 msgr2=0x7f3960157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395dd9b640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39600ceb40 0x7f3960157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395dd9b640 1 -- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f396006d6a0 msgr2=0x7f3960157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:55.931 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395dd9b640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f396006d6a0 0x7f3960157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:55.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395dd9b640 1 -- 172.21.15.27:0/1801816588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39601f88e0 con 0x7f39600c8d70 2024-04-20T11:01:55.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395e59c640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f396006d6a0 0x7f3960157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:01:55.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.929+0000 7f395dd9b640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39600c8d70 0x7f39601570e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f395400c4a0 tx=0x7f395400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:55.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.930+0000 7f394effd640 1 -- 172.21.15.27:0/1801816588 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954018070 con 0x7f39600c8d70 2024-04-20T11:01:55.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.930+0000 7f394effd640 1 -- 172.21.15.27:0/1801816588 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39540074e0 con 0x7f39600c8d70 2024-04-20T11:01:55.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.930+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39601f8bc0 con 0x7f39600c8d70 2024-04-20T11:01:55.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.930+0000 7f394effd640 1 -- 172.21.15.27:0/1801816588 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3954004a20 con 0x7f39600c8d70 2024-04-20T11:01:55.932 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.930+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601f9180 con 0x7f39600c8d70 2024-04-20T11:01:55.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.931+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39601f93c0 con 0x7f39600c8d70 2024-04-20T11:01:55.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.931+0000 7f394effd640 1 -- 172.21.15.27:0/1801816588 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f395401b070 con 0x7f39600c8d70 2024-04-20T11:01:55.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.931+0000 7f394effd640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f394006c920 0x7f394006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:55.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.932+0000 7f394effd640 1 -- 172.21.15.27:0/1801816588 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f395408b080 con 0x7f39600c8d70 2024-04-20T11:01:55.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.932+0000 7f395e59c640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f394006c920 0x7f394006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:55.933 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.932+0000 7f395e59c640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f394006c920 0x7f394006ede0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3950036000 tx=0x7f3950008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:55.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:55.936+0000 7f394effd640 1 -- 172.21.15.27:0/1801816588 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f39601f93c0 con 0x7f39600c8d70 2024-04-20T11:01:56.160 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.159+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f396004fcf0 con 0x7f39600c8d70 2024-04-20T11:01:56.161 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.160+0000 7f394effd640 1 -- 172.21.15.27:0/1801816588 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+6109 (secure 0 0 0) 0x7f3954058bb0 con 0x7f39600c8d70 2024-04-20T11:01:56.161 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:56.161 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:52.359346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:replay","state_seq":1,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4894,5029,5143]},"id":1}]} 2024-04-20T11:01:56.161 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 42 2024-04-20T11:01:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f394006c920 msgr2=0x7f394006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f394006c920 0x7f394006ede0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3950036000 tx=0x7f3950008030 comp rx=0 tx=0).stop 2024-04-20T11:01:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39600c8d70 msgr2=0x7f39601570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39600c8d70 0x7f39601570e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f395400c4a0 tx=0x7f395400c960 comp rx=0 tx=0).stop 2024-04-20T11:01:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 shutdown_connections 2024-04-20T11:01:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f394006c920 0x7f394006ede0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39600ceb40 0x7f3960157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39600c8d70 0x7f39601570e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:56.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 --2- 172.21.15.27:0/1801816588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f396006d6a0 0x7f3960157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:56.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 >> 172.21.15.27:0/1801816588 conn(0x7f396006c070 msgr2=0x7f39600676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:56.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 shutdown_connections 2024-04-20T11:01:56.164 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:56.162+0000 7f3964f0e640 1 -- 172.21.15.27:0/1801816588 wait complete. 2024-04-20T11:01:56.174 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:01:58.175 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:01:58.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.287+0000 7f805628c640 1 Processor -- start 2024-04-20T11:01:58.289 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.287+0000 7f805628c640 1 -- start start 2024-04-20T11:01:58.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.288+0000 7f805628c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8050155950 0x7f80500cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:58.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.288+0000 7f805628c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f80500cc330 0x7f80500b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:58.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.288+0000 7f805628c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 0x7f80500d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:58.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.288+0000 7f805628c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f80500b5e80 con 0x7f80500cc330 2024-04-20T11:01:58.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.288+0000 7f805628c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f805015b9c0 con 0x7f8050155950 2024-04-20T11:01:58.293 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.288+0000 7f805628c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f805015bca0 con 0x7f80500b5a30 2024-04-20T11:01:58.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.288+0000 7f804f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f80500cc330 0x7f80500b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:58.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.288+0000 7f804f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f80500cc330 0x7f80500b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52584/0 (socket says 172.21.15.27:52584) 2024-04-20T11:01:58.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.288+0000 7f804f7fe640 1 -- 172.21.15.27:0/2122929222 learned_addr learned my addr 172.21.15.27:0/2122929222 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:58.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.289+0000 7f804ffff640 1 --2- 172.21.15.27:0/2122929222 >> v2:172.21.15.137:3300/0 conn(0x7f8050155950 0x7f80500cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:58.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.289+0000 7f8054829640 1 --2- 172.21.15.27:0/2122929222 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 0x7f80500d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:58.294 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.289+0000 7f804f7fe640 1 -- 172.21.15.27:0/2122929222 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 msgr2=0x7f80500d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.289+0000 7f804f7fe640 1 --2- 172.21.15.27:0/2122929222 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 0x7f80500d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.289+0000 7f804f7fe640 1 -- 172.21.15.27:0/2122929222 >> v2:172.21.15.137:3300/0 conn(0x7f8050155950 msgr2=0x7f80500cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.289+0000 7f804f7fe640 1 --2- 172.21.15.27:0/2122929222 >> v2:172.21.15.137:3300/0 conn(0x7f8050155950 0x7f80500cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.289+0000 7f804f7fe640 1 -- 172.21.15.27:0/2122929222 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805015bf80 con 0x7f80500cc330 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.289+0000 7f804f7fe640 1 --2- 172.21.15.27:0/2122929222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80500cc330 0x7f80500b54f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f80500ce7c0 tx=0x7f804402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8a552afd92f697 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f804effd640 1 -- 172.21.15.27:0/2122929222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044034070 con 0x7f80500cc330 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f804effd640 1 -- 172.21.15.27:0/2122929222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8044030860 con 0x7f80500cc330 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f804effd640 1 -- 172.21.15.27:0/2122929222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044030c30 con 0x7f80500cc330 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f805628c640 1 -- 172.21.15.27:0/2122929222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80500cc330 msgr2=0x7f80500b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f805628c640 1 --2- 172.21.15.27:0/2122929222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80500cc330 0x7f80500b54f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f80500ce7c0 tx=0x7f804402b9a0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f805628c640 1 -- 172.21.15.27:0/2122929222 shutdown_connections 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f805628c640 1 --2- 172.21.15.27:0/2122929222 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 0x7f80500d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f805628c640 1 --2- 172.21.15.27:0/2122929222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80500cc330 0x7f80500b54f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f805628c640 1 --2- 172.21.15.27:0/2122929222 >> v2:172.21.15.137:3300/0 conn(0x7f8050155950 0x7f80500cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f805628c640 1 -- 172.21.15.27:0/2122929222 >> 172.21.15.27:0/2122929222 conn(0x7f80500676c0 msgr2=0x7f8050067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:58.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f805628c640 1 -- 172.21.15.27:0/2122929222 shutdown_connections 2024-04-20T11:01:58.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.290+0000 7f805628c640 1 -- 172.21.15.27:0/2122929222 wait complete. 2024-04-20T11:01:58.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.292+0000 7f805628c640 1 Processor -- start 2024-04-20T11:01:58.296 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.292+0000 7f805628c640 1 -- start start 2024-04-20T11:01:58.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.292+0000 7f805628c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 0x7f80501f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:58.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.292+0000 7f805628c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80500cc330 0x7f8050158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:58.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.292+0000 7f805628c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8050155950 0x7f805015a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:58.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.292+0000 7f805628c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f805015fb90 con 0x7f8050155950 2024-04-20T11:01:58.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.292+0000 7f805628c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f805015c540 con 0x7f80500cc330 2024-04-20T11:01:58.304 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.292+0000 7f805628c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f805015fd10 con 0x7f80500b5a30 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f8054829640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8050155950 0x7f805015a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f8054829640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8050155950 0x7f805015a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52592/0 (socket says 172.21.15.27:52592) 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f8054829640 1 -- 172.21.15.27:0/2778282195 learned_addr learned my addr 172.21.15.27:0/2778282195 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f804f7fe640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80500cc330 0x7f8050158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f804ffff640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 0x7f80501f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f8054829640 1 -- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 msgr2=0x7f80501f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f8054829640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 0x7f80501f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f8054829640 1 -- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80500cc330 msgr2=0x7f8050158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f8054829640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80500cc330 0x7f8050158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f8054829640 1 -- 172.21.15.27:0/2778282195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80501f1a50 con 0x7f8050155950 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f804ffff640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 0x7f80501f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f804f7fe640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80500cc330 0x7f8050158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f8054829640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8050155950 0x7f805015a8e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8048007d70 tx=0x7f804800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f804cff9640 1 -- 172.21.15.27:0/2778282195 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048014420 con 0x7f8050155950 2024-04-20T11:01:58.305 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f804cff9640 1 -- 172.21.15.27:0/2778282195 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8048019070 con 0x7f8050155950 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f804cff9640 1 -- 172.21.15.27:0/2778282195 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048014820 con 0x7f8050155950 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80501f1d30 con 0x7f8050155950 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.293+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80501f22f0 con 0x7f8050155950 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.294+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80500b4070 con 0x7f8050155950 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.295+0000 7f804cff9640 1 -- 172.21.15.27:0/2778282195 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80480149c0 con 0x7f8050155950 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.295+0000 7f804cff9640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f803c06c920 0x7f803c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.295+0000 7f804cff9640 1 -- 172.21.15.27:0/2778282195 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f804808c370 con 0x7f8050155950 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.296+0000 7f804ffff640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f803c06c920 0x7f803c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.296+0000 7f804ffff640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f803c06c920 0x7f803c06ede0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8040006fd0 tx=0x7f8040008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:01:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.300+0000 7f804cff9640 1 -- 172.21.15.27:0/2778282195 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f80500b4070 con 0x7f8050155950 2024-04-20T11:01:58.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.527+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f805004fcf0 con 0x7f8050155950 2024-04-20T11:01:58.528 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.528+0000 7f804cff9640 1 -- 172.21.15.27:0/2778282195 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6111 (secure 0 0 0) 0x7f804805a610 con 0x7f8050155950 2024-04-20T11:01:58.529 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:01:58.529 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:57.569716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:resolve","state_seq":13,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293]},"id":1}]} 2024-04-20T11:01:58.529 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 43 2024-04-20T11:01:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f803c06c920 msgr2=0x7f803c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f803c06c920 0x7f803c06ede0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8040006fd0 tx=0x7f8040008040 comp rx=0 tx=0).stop 2024-04-20T11:01:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8050155950 msgr2=0x7f805015a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:01:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8050155950 0x7f805015a8e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8048007d70 tx=0x7f804800e4b0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 shutdown_connections 2024-04-20T11:01:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f803c06c920 0x7f803c06ede0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8050155950 0x7f805015a8e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80500cc330 0x7f8050158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 --2- 172.21.15.27:0/2778282195 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f80500b5a30 0x7f80501f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:01:58.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 >> 172.21.15.27:0/2778282195 conn(0x7f80500676c0 msgr2=0x7f8050152cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:01:58.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 shutdown_connections 2024-04-20T11:01:58.532 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:01:58.530+0000 7f805628c640 1 -- 172.21.15.27:0/2778282195 wait complete. 2024-04-20T11:01:58.542 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:02:00.542 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:00.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.657+0000 7f028b706640 1 Processor -- start 2024-04-20T11:02:00.658 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.657+0000 7f028b706640 1 -- start start 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.658+0000 7f028b706640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0284155950 0x7f02840cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.658+0000 7f028b706640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f02840cc330 0x7f02840b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.658+0000 7f028b706640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02840b5ea0 con 0x7f02840b5a30 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.658+0000 7f028b706640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f028415b9c0 con 0x7f02840cc330 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.658+0000 7f028b706640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f028415bca0 con 0x7f0284155950 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.658+0000 7f0288ca1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f02840cc330 0x7f02840b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.658+0000 7f0288ca1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f02840cc330 0x7f02840b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52602/0 (socket says 172.21.15.27:52602) 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.658+0000 7f0288ca1640 1 -- 172.21.15.27:0/3392923652 learned_addr learned my addr 172.21.15.27:0/3392923652 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.658+0000 7f02894a2640 1 --2- 172.21.15.27:0/3392923652 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0284155950 0x7f02840cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.659+0000 7f0288ca1640 1 -- 172.21.15.27:0/3392923652 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0284155950 msgr2=0x7f02840cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.659+0000 7f0288ca1640 1 --2- 172.21.15.27:0/3392923652 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0284155950 0x7f02840cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.659+0000 7f0288ca1640 1 -- 172.21.15.27:0/3392923652 >> v1:172.21.15.27:6789/0 conn(0x7f02840b5a30 legacy=0x7f02840d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:00.660 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.659+0000 7f0288ca1640 1 -- 172.21.15.27:0/3392923652 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028415bf80 con 0x7f02840cc330 2024-04-20T11:02:00.661 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.659+0000 7f0288ca1640 1 --2- 172.21.15.27:0/3392923652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840cc330 0x7f02840b54f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0274009870 tx=0x7f027402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce338baa7b6202c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f027bfff640 1 -- 172.21.15.27:0/3392923652 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027402f840 con 0x7f02840cc330 2024-04-20T11:02:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f027bfff640 1 -- 172.21.15.27:0/3392923652 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f027402f9e0 con 0x7f02840cc330 2024-04-20T11:02:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f027bfff640 1 -- 172.21.15.27:0/3392923652 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027402fdb0 con 0x7f02840cc330 2024-04-20T11:02:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f028b706640 1 -- 172.21.15.27:0/3392923652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840cc330 msgr2=0x7f02840b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f028b706640 1 --2- 172.21.15.27:0/3392923652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840cc330 0x7f02840b54f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0274009870 tx=0x7f027402ba20 comp rx=0 tx=0).stop 2024-04-20T11:02:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f028b706640 1 -- 172.21.15.27:0/3392923652 shutdown_connections 2024-04-20T11:02:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f028b706640 1 --2- 172.21.15.27:0/3392923652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840cc330 0x7f02840b54f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:00.662 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f028b706640 1 --2- 172.21.15.27:0/3392923652 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f0284155950 0x7f02840cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:00.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f028b706640 1 -- 172.21.15.27:0/3392923652 >> 172.21.15.27:0/3392923652 conn(0x7f02840676c0 msgr2=0x7f0284067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:00.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f028b706640 1 -- 172.21.15.27:0/3392923652 shutdown_connections 2024-04-20T11:02:00.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.660+0000 7f028b706640 1 -- 172.21.15.27:0/3392923652 wait complete. 2024-04-20T11:02:00.664 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f028b706640 1 Processor -- start 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f028b706640 1 -- start start 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f028b706640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840b5a30 0x7f02841f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f028b706640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f02840cc330 0x7f028415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f028b706640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0284155950 0x7f028415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f028b706640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f028415fb90 con 0x7f02840b5a30 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f028b706640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f028415c540 con 0x7f0284155950 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f028b706640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f028415fd10 con 0x7f02840cc330 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f02894a2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840b5a30 0x7f02841f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f02894a2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840b5a30 0x7f02841f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52606/0 (socket says 172.21.15.27:52606) 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f02894a2640 1 -- 172.21.15.27:0/4168158935 learned_addr learned my addr 172.21.15.27:0/4168158935 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.662+0000 7f0288ca1640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f02840cc330 0x7f028415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f0289ca3640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0284155950 0x7f028415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f02894a2640 1 -- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f02840cc330 msgr2=0x7f028415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:00.665 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f02894a2640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f02840cc330 0x7f028415b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:00.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f02894a2640 1 -- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0284155950 msgr2=0x7f028415a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:00.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f02894a2640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0284155950 0x7f028415a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:00.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f02894a2640 1 -- 172.21.15.27:0/4168158935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02840c7b50 con 0x7f02840b5a30 2024-04-20T11:02:00.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f02894a2640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840b5a30 0x7f02841f2630 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f028000e9b0 tx=0x7f028000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:00.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f0279ffb640 1 -- 172.21.15.27:0/4168158935 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280017070 con 0x7f02840b5a30 2024-04-20T11:02:00.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f0279ffb640 1 -- 172.21.15.27:0/4168158935 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028000c9a0 con 0x7f02840b5a30 2024-04-20T11:02:00.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02840c7590 con 0x7f02840b5a30 2024-04-20T11:02:00.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.663+0000 7f0279ffb640 1 -- 172.21.15.27:0/4168158935 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028000cd90 con 0x7f02840b5a30 2024-04-20T11:02:00.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.664+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028415a0d0 con 0x7f02840b5a30 2024-04-20T11:02:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.665+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028404fcf0 con 0x7f02840b5a30 2024-04-20T11:02:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.665+0000 7f0279ffb640 1 -- 172.21.15.27:0/4168158935 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f028001a070 con 0x7f02840b5a30 2024-04-20T11:02:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.666+0000 7f0279ffb640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f026806c8f0 0x7f026806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.666+0000 7f0279ffb640 1 -- 172.21.15.27:0/4168158935 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f028008bd60 con 0x7f02840b5a30 2024-04-20T11:02:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.666+0000 7f0288ca1640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f026806c8f0 0x7f026806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:00.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.666+0000 7f0288ca1640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f026806c8f0 0x7f026806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0274036000 tx=0x7f0274008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:00.671 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.670+0000 7f0279ffb640 1 -- 172.21.15.27:0/4168158935 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0280059890 con 0x7f02840b5a30 2024-04-20T11:02:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.894+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02840cd0a0 con 0x7f02840b5a30 2024-04-20T11:02:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.895+0000 7f0279ffb640 1 -- 172.21.15.27:0/4168158935 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+6111 (secure 0 0 0) 0x7f028005db90 con 0x7f02840b5a30 2024-04-20T11:02:00.896 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:00.896 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:01:57.569716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:resolve","state_seq":13,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293]},"id":1}]} 2024-04-20T11:02:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 43 2024-04-20T11:02:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f026806c8f0 msgr2=0x7f026806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f026806c8f0 0x7f026806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0274036000 tx=0x7f0274008030 comp rx=0 tx=0).stop 2024-04-20T11:02:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840b5a30 msgr2=0x7f02841f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840b5a30 0x7f02841f2630 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f028000e9b0 tx=0x7f028000ee70 comp rx=0 tx=0).stop 2024-04-20T11:02:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 shutdown_connections 2024-04-20T11:02:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f026806c8f0 0x7f026806edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0284155950 0x7f028415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:00.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f02840cc330 0x7f028415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:00.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 --2- 172.21.15.27:0/4168158935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02840b5a30 0x7f02841f2630 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:00.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 >> 172.21.15.27:0/4168158935 conn(0x7f02840676c0 msgr2=0x7f02840cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:00.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 shutdown_connections 2024-04-20T11:02:00.899 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:00.897+0000 7f028b706640 1 -- 172.21.15.27:0/4168158935 wait complete. 2024-04-20T11:02:00.910 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:02:02.911 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:03.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.022+0000 7fd913d7e640 1 Processor -- start 2024-04-20T11:02:03.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.022+0000 7fd913d7e640 1 -- start start 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.023+0000 7fd913d7e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c155950 0x7fd90c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.023+0000 7fd913d7e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90c15b9c0 con 0x7fd90c0b5c00 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.023+0000 7fd913d7e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90c15bca0 con 0x7fd90c0cc330 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.023+0000 7fd913d7e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c15bf80 con 0x7fd90c155950 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd91231b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd90c0b5c00 0x7fd90c0d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36548/0 (socket says 172.21.15.27:36548) 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd91231b640 1 -- 172.21.15.27:0/2393551848 learned_addr learned my addr 172.21.15.27:0/2393551848 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd911b1a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c155950 0x7fd90c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd911319640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fd90c0cc330 0x7fd90c0b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:53488/0 (socket says 172.21.15.27:53488) 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd910b18640 1 -- 172.21.15.27:0/2393551848 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 410121289 0 0) 0x7fd90c15b9c0 con 0x7fd90c0b5c00 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd910b18640 1 -- 172.21.15.27:0/2393551848 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8f8003520 con 0x7fd90c0b5c00 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd910b18640 1 -- 172.21.15.27:0/2393551848 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218979086 0 0) 0x7fd90c15bca0 con 0x7fd90c0cc330 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd910b18640 1 -- 172.21.15.27:0/2393551848 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd90c15b9c0 con 0x7fd90c0cc330 2024-04-20T11:02:03.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd911b1a640 1 -- 172.21.15.27:0/2393551848 >> v1:172.21.15.137:6789/0 conn(0x7fd90c0cc330 legacy=0x7fd90c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:03.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd911b1a640 1 -- 172.21.15.27:0/2393551848 >> v1:172.21.15.27:6789/0 conn(0x7fd90c0b5c00 legacy=0x7fd90c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:03.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.024+0000 7fd911b1a640 1 -- 172.21.15.27:0/2393551848 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c15c260 con 0x7fd90c155950 2024-04-20T11:02:03.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.025+0000 7fd911b1a640 1 --2- 172.21.15.27:0/2393551848 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c155950 0x7fd90c0cbdf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9080097d0 tx=0x7fd90802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26543055a91263d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:03.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.025+0000 7fd910b18640 1 -- 172.21.15.27:0/2393551848 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908034070 con 0x7fd90c155950 2024-04-20T11:02:03.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.025+0000 7fd910b18640 1 -- 172.21.15.27:0/2393551848 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd908030880 con 0x7fd90c155950 2024-04-20T11:02:03.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.025+0000 7fd910b18640 1 -- 172.21.15.27:0/2393551848 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908030c30 con 0x7fd90c155950 2024-04-20T11:02:03.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.026+0000 7fd913d7e640 1 -- 172.21.15.27:0/2393551848 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c155950 msgr2=0x7fd90c0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:03.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.026+0000 7fd913d7e640 1 --2- 172.21.15.27:0/2393551848 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c155950 0x7fd90c0cbdf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9080097d0 tx=0x7fd90802b9c0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.026+0000 7fd913d7e640 1 -- 172.21.15.27:0/2393551848 shutdown_connections 2024-04-20T11:02:03.027 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.026+0000 7fd913d7e640 1 --2- 172.21.15.27:0/2393551848 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c155950 0x7fd90c0cbdf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.026+0000 7fd913d7e640 1 -- 172.21.15.27:0/2393551848 >> 172.21.15.27:0/2393551848 conn(0x7fd90c0676c0 msgr2=0x7fd90c067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:03.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.026+0000 7fd913d7e640 1 -- 172.21.15.27:0/2393551848 shutdown_connections 2024-04-20T11:02:03.028 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.026+0000 7fd913d7e640 1 -- 172.21.15.27:0/2393551848 wait complete. 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.027+0000 7fd913d7e640 1 Processor -- start 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.027+0000 7fd913d7e640 1 -- start start 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd913d7e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c0b5c00 0x7fd90c0cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd913d7e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd90c0cc330 0x7fd90c1f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd913d7e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c155950 0x7fd90c1f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd913d7e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c15fd10 con 0x7fd90c155950 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd913d7e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c15fb90 con 0x7fd90c0cc330 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd913d7e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c155d30 con 0x7fd90c0b5c00 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911b1a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c0b5c00 0x7fd90c0cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd91231b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c155950 0x7fd90c1f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911b1a640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c0b5c00 0x7fd90c0cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:34704/0 (socket says 172.21.15.27:34704) 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911319640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd90c0cc330 0x7fd90c1f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:03.030 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911b1a640 1 -- 172.21.15.27:0/3693849456 learned_addr learned my addr 172.21.15.27:0/3693849456 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911b1a640 1 -- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd90c0cc330 msgr2=0x7fd90c1f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911b1a640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd90c0cc330 0x7fd90c1f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911b1a640 1 -- 172.21.15.27:0/3693849456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c155950 msgr2=0x7fd90c1f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911b1a640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c155950 0x7fd90c1f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911b1a640 1 -- 172.21.15.27:0/3693849456 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c1f2150 con 0x7fd90c0b5c00 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.028+0000 7fd911b1a640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c0b5c00 0x7fd90c0cd980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9080098f0 tx=0x7fd9080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.029+0000 7fd9027fc640 1 -- 172.21.15.27:0/3693849456 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908048070 con 0x7fd90c0b5c00 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.029+0000 7fd9027fc640 1 -- 172.21.15.27:0/3693849456 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd908030680 con 0x7fd90c0b5c00 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.029+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c1f23f0 con 0x7fd90c0b5c00 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.029+0000 7fd9027fc640 1 -- 172.21.15.27:0/3693849456 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908034040 con 0x7fd90c0b5c00 2024-04-20T11:02:03.031 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.029+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c15cdd0 con 0x7fd90c0b5c00 2024-04-20T11:02:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.030+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90c04fcf0 con 0x7fd90c0b5c00 2024-04-20T11:02:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.030+0000 7fd9027fc640 1 -- 172.21.15.27:0/3693849456 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd908037070 con 0x7fd90c0b5c00 2024-04-20T11:02:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.031+0000 7fd9027fc640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd8f006c920 0x7fd8f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.031+0000 7fd9027fc640 1 -- 172.21.15.27:0/3693849456 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fd9080ad460 con 0x7fd90c0b5c00 2024-04-20T11:02:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.031+0000 7fd911319640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd8f006c920 0x7fd8f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:03.032 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.031+0000 7fd911319640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd8f006c920 0x7fd8f006ede0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd90c151630 tx=0x7fd8fc007cf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.036+0000 7fd9027fc640 1 -- 172.21.15.27:0/3693849456 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd90807aef0 con 0x7fd90c0b5c00 2024-04-20T11:02:03.262 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.261+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd90c0cc930 con 0x7fd90c0b5c00 2024-04-20T11:02:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.263+0000 7fd9027fc640 1 -- 172.21.15.27:0/3693849456 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+6113 (secure 0 0 0) 0x7fd90807f1f0 con 0x7fd90c0b5c00 2024-04-20T11:02:03.263 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:03.264 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:02.901833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4894,5029,5143]},"id":1}]} 2024-04-20T11:02:03.264 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 44 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd8f006c920 msgr2=0x7fd8f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd8f006c920 0x7fd8f006ede0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd90c151630 tx=0x7fd8fc007cf0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c0b5c00 msgr2=0x7fd90c0cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c0b5c00 0x7fd90c0cd980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9080098f0 tx=0x7fd9080046c0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 shutdown_connections 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fd8f006c920 0x7fd8f006ede0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c155950 0x7fd90c1f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd90c0cc330 0x7fd90c1f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 --2- 172.21.15.27:0/3693849456 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd90c0b5c00 0x7fd90c0cd980 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 >> 172.21.15.27:0/3693849456 conn(0x7fd90c0676c0 msgr2=0x7fd90c15f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:03.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 shutdown_connections 2024-04-20T11:02:03.267 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:03.265+0000 7fd913d7e640 1 -- 172.21.15.27:0/3693849456 wait complete. 2024-04-20T11:02:03.277 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:02:05.278 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:05.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.387+0000 7f8fcab04640 1 Processor -- start 2024-04-20T11:02:05.388 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.387+0000 7f8fcab04640 1 -- start start 2024-04-20T11:02:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.388+0000 7f8fcab04640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 0x7f8fc40b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.388+0000 7f8fcab04640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc40b5e70 con 0x7f8fc4155950 2024-04-20T11:02:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.388+0000 7f8fcab04640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc415b9c0 con 0x7f8fc40d0c90 2024-04-20T11:02:05.391 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.388+0000 7f8fcab04640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc415bca0 con 0x7f8fc40b54f0 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.388+0000 7f8fc88a0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8fc4155950 0x7f8fc40cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42422/0 (socket says 172.21.15.27:42422) 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.388+0000 7f8fc88a0640 1 -- 172.21.15.27:0/1104834041 learned_addr learned my addr 172.21.15.27:0/1104834041 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc3fff640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 0x7f8fc40b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502258695 0 0) 0x7f8fc40b5e70 con 0x7f8fc4155950 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fac003520 con 0x7f8fc4155950 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026917342 0 0) 0x7f8fc415b9c0 con 0x7f8fc40d0c90 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fc40b5e70 con 0x7f8fc40d0c90 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1714586460 0 0) 0x7f8fac003520 con 0x7f8fc4155950 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 msgr2=0x7f8fc40b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 0x7f8fc40b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 >> v1:172.21.15.137:6789/0 conn(0x7f8fc40d0c90 legacy=0x7f8fc40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc415bf80 con 0x7f8fc4155950 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7f8fb4001670 con 0x7f8fc4155950 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc3fff640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 0x7f8fc40b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:02:05.392 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 >> v1:172.21.15.27:6789/0 conn(0x7f8fc4155950 legacy=0x7f8fc40cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 0x7f8fac003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fac004800 0x7f8fac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 0x7f8fac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc415bca0 con 0x7f8fac004800 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac00ba70 con 0x7f8fac003b70 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8fac00bc40 con 0x7f8fc40b54f0 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc88a0640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 0x7f8fac003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc3fff640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fac004800 0x7f8fac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.389+0000 7f8fc90a1640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 0x7f8fac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.390+0000 7f8fc88a0640 1 -- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 msgr2=0x7f8fac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.390+0000 7f8fc88a0640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 0x7f8fac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.390+0000 7f8fc88a0640 1 -- 172.21.15.27:0/1104834041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fac004800 msgr2=0x7f8fac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.390+0000 7f8fc88a0640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fac004800 0x7f8fac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:05.393 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.390+0000 7f8fc88a0640 1 -- 172.21.15.27:0/1104834041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac003520 con 0x7f8fac003b70 2024-04-20T11:02:05.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.390+0000 7f8fc90a1640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc40b54f0 0x7f8fac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:02:05.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.390+0000 7f8fc3fff640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fac004800 0x7f8fac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:02:05.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.390+0000 7f8fc88a0640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 0x7f8fac003f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007f90 tx=0x7f8fac005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d742412720a51e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:05.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4007be0 con 0x7f8fac003b70 2024-04-20T11:02:05.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb4030070 con 0x7f8fac003b70 2024-04-20T11:02:05.394 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fc37fe640 1 -- 172.21.15.27:0/1104834041 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4005d30 con 0x7f8fac003b70 2024-04-20T11:02:05.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fcab04640 1 -- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 msgr2=0x7f8fac003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:05.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fcab04640 1 --2- 172.21.15.27:0/1104834041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 0x7f8fac003f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007f90 tx=0x7f8fac005830 comp rx=0 tx=0).stop 2024-04-20T11:02:05.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fc90a1640 1 -- 172.21.15.27:0/1104834041 reap_dead start 2024-04-20T11:02:05.397 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fcab04640 1 -- 172.21.15.27:0/1104834041 shutdown_connections 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fcab04640 1 -- 172.21.15.27:0/1104834041 >> 172.21.15.27:0/1104834041 conn(0x7f8fc40676c0 msgr2=0x7f8fc4067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fcab04640 1 -- 172.21.15.27:0/1104834041 shutdown_connections 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.391+0000 7f8fcab04640 1 -- 172.21.15.27:0/1104834041 wait complete. 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.393+0000 7f8fcab04640 1 Processor -- start 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.393+0000 7f8fcab04640 1 -- start start 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.393+0000 7f8fcab04640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 0x7f8fc40c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.393+0000 7f8fcab04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fc4159720 0x7f8fc40c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fcab04640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc414ec10 0x7f8fc4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fcab04640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc415fb90 con 0x7f8fc4159720 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fcab04640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc415c540 con 0x7f8fac003b70 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fcab04640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc415fd10 con 0x7f8fc414ec10 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fc4159720 0x7f8fc40c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fc4159720 0x7f8fc40c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37360/0 (socket says 172.21.15.27:37360) 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc3fff640 1 -- 172.21.15.27:0/547671874 learned_addr learned my addr 172.21.15.27:0/547671874 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:05.398 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc88a0640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 0x7f8fc40c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc90a1640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc414ec10 0x7f8fc4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc3fff640 1 -- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc414ec10 msgr2=0x7f8fc4159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc3fff640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc414ec10 0x7f8fc4159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc3fff640 1 -- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 msgr2=0x7f8fc40c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc3fff640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 0x7f8fc40c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc3fff640 1 -- 172.21.15.27:0/547671874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc41f5320 con 0x7f8fc4159720 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc90a1640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc414ec10 0x7f8fc4159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc3fff640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fc4159720 0x7f8fc40c6e10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8011cd0 tx=0x7f8fb800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc17fa640 1 -- 172.21.15.27:0/547671874 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800fd20 con 0x7f8fc4159720 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc17fa640 1 -- 172.21.15.27:0/547671874 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb8014440 con 0x7f8fc4159720 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc41f5600 con 0x7f8fc4159720 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fc17fa640 1 -- 172.21.15.27:0/547671874 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80147f0 con 0x7f8fc4159720 2024-04-20T11:02:05.399 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.394+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc40c60d0 con 0x7f8fc4159720 2024-04-20T11:02:05.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.395+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc404fcf0 con 0x7f8fc4159720 2024-04-20T11:02:05.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.396+0000 7f8fc17fa640 1 -- 172.21.15.27:0/547671874 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fb8012040 con 0x7f8fc4159720 2024-04-20T11:02:05.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.396+0000 7f8fc17fa640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8fb006c8f0 0x7f8fb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:05.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.396+0000 7f8fc17fa640 1 -- 172.21.15.27:0/547671874 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f8fb808c610 con 0x7f8fc4159720 2024-04-20T11:02:05.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.397+0000 7f8fc88a0640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8fb006c8f0 0x7f8fb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:05.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.397+0000 7f8fc88a0640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8fb006c8f0 0x7f8fb006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4034000 tx=0x7f8fb4009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:05.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.401+0000 7f8fc17fa640 1 -- 172.21.15.27:0/547671874 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8fb8019030 con 0x7f8fc4159720 2024-04-20T11:02:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.645+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fc4159d00 con 0x7f8fc4159720 2024-04-20T11:02:05.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.645+0000 7f8fc17fa640 1 -- 172.21.15.27:0/547671874 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+6113 (secure 0 0 0) 0x7f8fb805a820 con 0x7f8fc4159720 2024-04-20T11:02:05.647 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:05.647 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:02.901833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4894,5029,5143]},"id":1}]} 2024-04-20T11:02:05.647 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 44 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8fb006c8f0 msgr2=0x7f8fb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8fb006c8f0 0x7f8fb006edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4034000 tx=0x7f8fb4009030 comp rx=0 tx=0).stop 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fc4159720 msgr2=0x7f8fc40c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fc4159720 0x7f8fc40c6e10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8011cd0 tx=0x7f8fb800f610 comp rx=0 tx=0).stop 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 shutdown_connections 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f8fb006c8f0 0x7f8fb006edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8fc414ec10 0x7f8fc4159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fc4159720 0x7f8fc40c6e10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 --2- 172.21.15.27:0/547671874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac003b70 0x7f8fc40c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 >> 172.21.15.27:0/547671874 conn(0x7f8fc40676c0 msgr2=0x7f8fc40cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:05.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 shutdown_connections 2024-04-20T11:02:05.650 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:05.648+0000 7f8fcab04640 1 -- 172.21.15.27:0/547671874 wait complete. 2024-04-20T11:02:05.660 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 44, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-04-20T10:55:42.150319+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 44, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2, 3, 4 ], "info": { "gid_4293": { "addr": "172.21.15.137:6839/2929802235", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6837", "nonce": 2929802235, "type": "v2" }, { "addr": "172.21.15.137:6839", "nonce": 2929802235, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4293, "incarnation": 4, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4894": { "addr": "172.21.15.137:6838/2896327439", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6836", "nonce": 2896327439, "type": "v2" }, { "addr": "172.21.15.137:6838", "nonce": 2896327439, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4894, "incarnation": 24, "join_fscid": -1, "name": "b", "rank": 4, "state": "up:active", "state_seq": 17 }, "gid_5029": { "addr": "172.21.15.137:6835/4235742413", "addrs": { "addrvec": [ { "addr": "172.21.15.137:6834", "nonce": 4235742413, "type": "v2" }, { "addr": "172.21.15.137:6835", "nonce": 4235742413, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5029, "incarnation": 30, "join_fscid": -1, "name": "f", "rank": 3, "state": "up:active", "state_seq": 22 }, "gid_5143": { "addr": "172.21.15.27:6835/2020174232", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6832", "nonce": 2020174232, "type": "v2" }, { "addr": "172.21.15.27:6835", "nonce": 2020174232, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5143, "incarnation": 36, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 20 }, "gid_5406": { "addr": "172.21.15.27:6837/3526024786", "addrs": { "addrvec": [ { "addr": "172.21.15.27:6834", "nonce": 3526024786, "type": "v2" }, { "addr": "172.21.15.27:6837", "nonce": 3526024786, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_12": "quiesce subvolumes", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5406, "incarnation": 42, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:reconnect", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 5, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-04-20T11:02:02.901833+0000", "qdb_cluster": [ 4293, 4894, 5029, 5143 ], "qdb_leader": 4293, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "snap_rstat": true, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4293, "mds_1": 5406, "mds_2": 5143, "mds_3": 5029, "mds_4": 4894 } } } ], "standbys": [] } 2024-04-20T11:02:07.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:07.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.774+0000 7f4ebe1a4640 1 Processor -- start 2024-04-20T11:02:07.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.774+0000 7f4ebe1a4640 1 -- start start 2024-04-20T11:02:07.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4ebe1a4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4eb81556f0 0x7f4eb815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:07.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4ebe1a4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4eb8070ca0 0x7f4eb815f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:07.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4ebe1a4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 0x7f4eb80cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:07.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4ebe1a4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb806ead0 con 0x7f4eb81556f0 2024-04-20T11:02:07.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4ebe1a4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb8071110 con 0x7f4eb8070ca0 2024-04-20T11:02:07.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4ebe1a4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb815fac0 con 0x7f4eb80714e0 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4eb77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4eb81556f0 0x7f4eb815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4eb77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4eb81556f0 0x7f4eb815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37368/0 (socket says 172.21.15.27:37368) 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4eb77fe640 1 -- 172.21.15.27:0/3864818597 learned_addr learned my addr 172.21.15.27:0/3864818597 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4eb6ffd640 1 --2- 172.21.15.27:0/3864818597 >> v2:172.21.15.137:3300/0 conn(0x7f4eb8070ca0 0x7f4eb815f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.775+0000 7f4eb7fff640 1 --2- 172.21.15.27:0/3864818597 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 0x7f4eb80cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.776+0000 7f4eb77fe640 1 -- 172.21.15.27:0/3864818597 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 msgr2=0x7f4eb80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.776+0000 7f4eb77fe640 1 --2- 172.21.15.27:0/3864818597 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 0x7f4eb80cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.776+0000 7f4eb77fe640 1 -- 172.21.15.27:0/3864818597 >> v2:172.21.15.137:3300/0 conn(0x7f4eb8070ca0 msgr2=0x7f4eb815f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.776+0000 7f4eb77fe640 1 --2- 172.21.15.27:0/3864818597 >> v2:172.21.15.137:3300/0 conn(0x7f4eb8070ca0 0x7f4eb815f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.776+0000 7f4eb77fe640 1 -- 172.21.15.27:0/3864818597 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb80c9e60 con 0x7f4eb81556f0 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.776+0000 7f4eb7fff640 1 --2- 172.21.15.27:0/3864818597 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 0x7f4eb80cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:07.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.776+0000 7f4eb77fe640 1 --2- 172.21.15.27:0/3864818597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 0x7f4eb815b580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4eb806d460 tx=0x7f4ea802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5776e3c81eee9020 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:07.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4eb67fc640 1 -- 172.21.15.27:0/3864818597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea802e070 con 0x7f4eb81556f0 2024-04-20T11:02:07.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4eb67fc640 1 -- 172.21.15.27:0/3864818597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea8037070 con 0x7f4eb81556f0 2024-04-20T11:02:07.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4eb67fc640 1 -- 172.21.15.27:0/3864818597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8031a00 con 0x7f4eb81556f0 2024-04-20T11:02:07.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/3864818597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 msgr2=0x7f4eb815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:07.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/3864818597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 0x7f4eb815b580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4eb806d460 tx=0x7f4ea802b880 comp rx=0 tx=0).stop 2024-04-20T11:02:07.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/3864818597 shutdown_connections 2024-04-20T11:02:07.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/3864818597 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 0x7f4eb80cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:07.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/3864818597 >> v2:172.21.15.137:3300/0 conn(0x7f4eb8070ca0 0x7f4eb815f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:07.783 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/3864818597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 0x7f4eb815b580 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/3864818597 >> 172.21.15.27:0/3864818597 conn(0x7f4eb8067ae0 msgr2=0x7f4eb8067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/3864818597 shutdown_connections 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.777+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/3864818597 wait complete. 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.778+0000 7f4ebe1a4640 1 Processor -- start 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.779+0000 7f4ebe1a4640 1 -- start start 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.779+0000 7f4ebe1a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8070ca0 0x7f4eb81503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.779+0000 7f4ebe1a4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 0x7f4eb81f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.779+0000 7f4ebe1a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 0x7f4eb81f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.779+0000 7f4ebe1a4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80c9b80 con 0x7f4eb81556f0 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.779+0000 7f4ebe1a4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80c98a0 con 0x7f4eb8070ca0 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.779+0000 7f4ebe1a4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb806e480 con 0x7f4eb80714e0 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.779+0000 7f4eb7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 0x7f4eb81f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.779+0000 7f4eb7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 0x7f4eb81f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37380/0 (socket says 172.21.15.27:37380) 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4eb7fff640 1 -- 172.21.15.27:0/4176265819 learned_addr learned my addr 172.21.15.27:0/4176265819 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4eb6ffd640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 0x7f4eb81f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4eb77fe640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8070ca0 0x7f4eb81503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4eb7fff640 1 -- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 msgr2=0x7f4eb81f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4eb7fff640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 0x7f4eb81f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4eb7fff640 1 -- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8070ca0 msgr2=0x7f4eb81503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4eb7fff640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8070ca0 0x7f4eb81503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4eb7fff640 1 -- 172.21.15.27:0/4176265819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81516d0 con 0x7f4eb81556f0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4eb7fff640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 0x7f4eb81f4d30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0007df0 tx=0x7f4eb000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4e9ffff640 1 -- 172.21.15.27:0/4176265819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb001a070 con 0x7f4eb81556f0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4e9ffff640 1 -- 172.21.15.27:0/4176265819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb0003990 con 0x7f4eb81556f0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb8151990 con 0x7f4eb81556f0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4e9ffff640 1 -- 172.21.15.27:0/4176265819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0003d80 con 0x7f4eb81556f0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.780+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb8151f50 con 0x7f4eb81556f0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.781+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb80688e0 con 0x7f4eb81556f0 2024-04-20T11:02:07.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.782+0000 7f4e9ffff640 1 -- 172.21.15.27:0/4176265819 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4eb001d070 con 0x7f4eb81556f0 2024-04-20T11:02:07.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.782+0000 7f4e9ffff640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:07.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.782+0000 7f4e9ffff640 1 -- 172.21.15.27:0/4176265819 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f4eb008bf30 con 0x7f4eb81556f0 2024-04-20T11:02:07.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.782+0000 7f4eb77fe640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:07.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.783+0000 7f4eb77fe640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ea406c8f0 0x7f4ea406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8036000 tx=0x7f4ea803d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:07.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:07.787+0000 7f4e9ffff640 1 -- 172.21.15.27:0/4176265819 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4eb0010070 con 0x7f4eb81556f0 2024-04-20T11:02:08.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.013+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4eb804fcf0 con 0x7f4eb81556f0 2024-04-20T11:02:08.014 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.013+0000 7f4e9ffff640 1 -- 172.21.15.27:0/4176265819 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+6110 (secure 0 0 0) 0x7f4eb005a1d0 con 0x7f4eb81556f0 2024-04-20T11:02:08.014 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:08.015 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:07.909358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293]},"id":1}]} 2024-04-20T11:02:08.015 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 45 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ea406c8f0 msgr2=0x7f4ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ea406c8f0 0x7f4ea406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8036000 tx=0x7f4ea803d040 comp rx=0 tx=0).stop 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 msgr2=0x7f4eb81f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 0x7f4eb81f4d30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0007df0 tx=0x7f4eb000d4a0 comp rx=0 tx=0).stop 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 shutdown_connections 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4ea406c8f0 0x7f4ea406edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4eb81556f0 0x7f4eb81f4d30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4eb80714e0 0x7f4eb81f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 --2- 172.21.15.27:0/4176265819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8070ca0 0x7f4eb81503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 >> 172.21.15.27:0/4176265819 conn(0x7f4eb8067ae0 msgr2=0x7f4eb8067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:08.017 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.015+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 shutdown_connections 2024-04-20T11:02:08.018 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:08.016+0000 7f4ebe1a4640 1 -- 172.21.15.27:0/4176265819 wait complete. 2024-04-20T11:02:08.027 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:02:10.029 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.139+0000 7fa46c38f640 1 Processor -- start 2024-04-20T11:02:10.139 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.139+0000 7fa46c38f640 1 -- start start 2024-04-20T11:02:10.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46c38f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa4640b54f0 0x7fa4640b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:10.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46c38f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640d0c90 0x7fa4640d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:10.140 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46c38f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4640b5e70 con 0x7fa464155950 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46c38f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa46415b9c0 con 0x7fa4640b54f0 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46c38f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fa46415bca0 con 0x7fa4640d0c90 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46992a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa4640b54f0 0x7fa4640b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46992a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa4640b54f0 0x7fa4640b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37384/0 (socket says 172.21.15.27:37384) 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46992a640 1 -- 172.21.15.27:0/3452180374 learned_addr learned my addr 172.21.15.27:0/3452180374 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46a92c640 1 --2- 172.21.15.27:0/3452180374 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640d0c90 0x7fa4640d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46992a640 1 -- 172.21.15.27:0/3452180374 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640d0c90 msgr2=0x7fa4640d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46992a640 1 --2- 172.21.15.27:0/3452180374 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640d0c90 0x7fa4640d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46992a640 1 -- 172.21.15.27:0/3452180374 >> v1:172.21.15.137:6789/0 conn(0x7fa464155950 legacy=0x7fa4640cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46992a640 1 -- 172.21.15.27:0/3452180374 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46415bf80 con 0x7fa4640b54f0 2024-04-20T11:02:10.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.140+0000 7fa46a92c640 1 --2- 172.21.15.27:0/3452180374 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640d0c90 0x7fa4640d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.141+0000 7fa46992a640 1 --2- 172.21.15.27:0/3452180374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4640b54f0 0x7fa4640b5930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa454009870 tx=0x7fa45402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a27fe178697e2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.141+0000 7fa469129640 1 -- 172.21.15.27:0/3452180374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45402f840 con 0x7fa4640b54f0 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.141+0000 7fa469129640 1 -- 172.21.15.27:0/3452180374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa45402f9e0 con 0x7fa4640b54f0 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.141+0000 7fa469129640 1 -- 172.21.15.27:0/3452180374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45402fdb0 con 0x7fa4640b54f0 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.142+0000 7fa46c38f640 1 -- 172.21.15.27:0/3452180374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4640b54f0 msgr2=0x7fa4640b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.142+0000 7fa46c38f640 1 --2- 172.21.15.27:0/3452180374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4640b54f0 0x7fa4640b5930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa454009870 tx=0x7fa45402ba20 comp rx=0 tx=0).stop 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.142+0000 7fa46c38f640 1 -- 172.21.15.27:0/3452180374 shutdown_connections 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.142+0000 7fa46c38f640 1 --2- 172.21.15.27:0/3452180374 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640d0c90 0x7fa4640d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.142+0000 7fa46c38f640 1 --2- 172.21.15.27:0/3452180374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa4640b54f0 0x7fa4640b5930 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.142+0000 7fa46c38f640 1 -- 172.21.15.27:0/3452180374 >> 172.21.15.27:0/3452180374 conn(0x7fa4640676c0 msgr2=0x7fa464067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.142+0000 7fa46c38f640 1 -- 172.21.15.27:0/3452180374 shutdown_connections 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.142+0000 7fa46c38f640 1 -- 172.21.15.27:0/3452180374 wait complete. 2024-04-20T11:02:10.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.143+0000 7fa46c38f640 1 Processor -- start 2024-04-20T11:02:10.144 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.143+0000 7fa46c38f640 1 -- start start 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46c38f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640b54f0 0x7fa464154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46c38f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4640d0c90 0x7fa46415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46c38f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa464155950 0x7fa46415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46c38f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4640d1620 con 0x7fa464155950 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46c38f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa46415c540 con 0x7fa4640d0c90 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46c38f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fa46415fb90 con 0x7fa4640b54f0 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a92c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa464155950 0x7fa46415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a92c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa464155950 0x7fa46415e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37388/0 (socket says 172.21.15.27:37388) 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a92c640 1 -- 172.21.15.27:0/3231585669 learned_addr learned my addr 172.21.15.27:0/3231585669 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46992a640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4640d0c90 0x7fa46415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a12b640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640b54f0 0x7fa464154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:10.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a92c640 1 -- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640b54f0 msgr2=0x7fa464154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a92c640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640b54f0 0x7fa464154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a92c640 1 -- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4640d0c90 msgr2=0x7fa46415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a92c640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4640d0c90 0x7fa46415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a92c640 1 -- 172.21.15.27:0/3231585669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4641f22d0 con 0x7fa464155950 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.144+0000 7fa46a12b640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640b54f0 0x7fa464154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.145+0000 7fa46a92c640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa464155950 0x7fa46415e140 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa460007de0 tx=0x7fa46000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.145+0000 7fa45affd640 1 -- 172.21.15.27:0/3231585669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46001a070 con 0x7fa464155950 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.145+0000 7fa45affd640 1 -- 172.21.15.27:0/3231585669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa460003990 con 0x7fa464155950 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.145+0000 7fa45affd640 1 -- 172.21.15.27:0/3231585669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460003d80 con 0x7fa464155950 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.145+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4641f1750 con 0x7fa464155950 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.145+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4641f1d10 con 0x7fa464155950 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.146+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4640b4070 con 0x7fa464155950 2024-04-20T11:02:10.147 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.146+0000 7fa45affd640 1 -- 172.21.15.27:0/3231585669 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa46001d070 con 0x7fa464155950 2024-04-20T11:02:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.147+0000 7fa45affd640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa44806c8f0 0x7fa44806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.147+0000 7fa45affd640 1 -- 172.21.15.27:0/3231585669 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fa46008c040 con 0x7fa464155950 2024-04-20T11:02:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.147+0000 7fa46a12b640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa44806c8f0 0x7fa44806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:10.148 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.147+0000 7fa46a12b640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa44806c8f0 0x7fa44806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa45c001700 tx=0x7fa45c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.151+0000 7fa45affd640 1 -- 172.21.15.27:0/3231585669 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa4640b4070 con 0x7fa464155950 2024-04-20T11:02:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.374+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa46404fcf0 con 0x7fa464155950 2024-04-20T11:02:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.374+0000 7fa45affd640 1 -- 172.21.15.27:0/3231585669 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+6110 (secure 0 0 0) 0x7fa460010070 con 0x7fa464155950 2024-04-20T11:02:10.374 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:10.374 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:07.909358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293]},"id":1}]} 2024-04-20T11:02:10.374 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 45 2024-04-20T11:02:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa44806c8f0 msgr2=0x7fa44806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa44806c8f0 0x7fa44806edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa45c001700 tx=0x7fa45c009040 comp rx=0 tx=0).stop 2024-04-20T11:02:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa464155950 msgr2=0x7fa46415e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa464155950 0x7fa46415e140 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa460007de0 tx=0x7fa46000d4a0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 shutdown_connections 2024-04-20T11:02:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa44806c8f0 0x7fa44806edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa464155950 0x7fa46415e140 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4640d0c90 0x7fa46415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 --2- 172.21.15.27:0/3231585669 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa4640b54f0 0x7fa464154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:10.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 >> 172.21.15.27:0/3231585669 conn(0x7fa4640676c0 msgr2=0x7fa4640cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:10.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 shutdown_connections 2024-04-20T11:02:10.377 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:10.376+0000 7fa46c38f640 1 -- 172.21.15.27:0/3231585669 wait complete. 2024-04-20T11:02:10.387 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:02:12.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:12.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.513+0000 7fa5f22a8640 1 Processor -- start 2024-04-20T11:02:12.513 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.513+0000 7fa5f22a8640 1 -- start start 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.514+0000 7fa5f22a8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa5ec155950 0x7fa5ec0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.514+0000 7fa5f22a8640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fa5ec0cc330 0x7fa5ec0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.514+0000 7fa5f22a8640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ec15b9c0 con 0x7fa5ec0b5a30 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.514+0000 7fa5f22a8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fa5ec15bca0 con 0x7fa5ec155950 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.514+0000 7fa5f22a8640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fa5ec15bf80 con 0x7fa5ec0cc330 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.514+0000 7fa5ebfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa5ec155950 0x7fa5ec0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.514+0000 7fa5ebfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa5ec155950 0x7fa5ec0cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37404/0 (socket says 172.21.15.27:37404) 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.514+0000 7fa5ebfff640 1 -- 172.21.15.27:0/1427603025 learned_addr learned my addr 172.21.15.27:0/1427603025 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.515+0000 7fa5eb7fe640 1 --2- 172.21.15.27:0/1427603025 >> v2:172.21.15.137:3300/0 conn(0x7fa5ec0cc330 0x7fa5ec0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.515+0000 7fa5ebfff640 1 -- 172.21.15.27:0/1427603025 >> v2:172.21.15.137:3300/0 conn(0x7fa5ec0cc330 msgr2=0x7fa5ec0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:12.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.515+0000 7fa5ebfff640 1 --2- 172.21.15.27:0/1427603025 >> v2:172.21.15.137:3300/0 conn(0x7fa5ec0cc330 0x7fa5ec0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.515+0000 7fa5ebfff640 1 -- 172.21.15.27:0/1427603025 >> v1:172.21.15.137:6789/0 conn(0x7fa5ec0b5a30 legacy=0x7fa5ec0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:12.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.515+0000 7fa5ebfff640 1 -- 172.21.15.27:0/1427603025 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec15c260 con 0x7fa5ec155950 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5ebfff640 1 --2- 172.21.15.27:0/1427603025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 0x7fa5ec0cbdf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0b44e0 tx=0x7fa5dc02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fb632546684ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5eaffd640 1 -- 172.21.15.27:0/1427603025 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc02f850 con 0x7fa5ec155950 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5eaffd640 1 -- 172.21.15.27:0/1427603025 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5dc02f9f0 con 0x7fa5ec155950 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5eaffd640 1 -- 172.21.15.27:0/1427603025 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc02fdc0 con 0x7fa5ec155950 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5f22a8640 1 -- 172.21.15.27:0/1427603025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 msgr2=0x7fa5ec0cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5f22a8640 1 --2- 172.21.15.27:0/1427603025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 0x7fa5ec0cbdf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0b44e0 tx=0x7fa5dc02ba30 comp rx=0 tx=0).stop 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5f22a8640 1 -- 172.21.15.27:0/1427603025 shutdown_connections 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5f22a8640 1 --2- 172.21.15.27:0/1427603025 >> v2:172.21.15.137:3300/0 conn(0x7fa5ec0cc330 0x7fa5ec0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5f22a8640 1 --2- 172.21.15.27:0/1427603025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 0x7fa5ec0cbdf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5f22a8640 1 -- 172.21.15.27:0/1427603025 >> 172.21.15.27:0/1427603025 conn(0x7fa5ec0676c0 msgr2=0x7fa5ec067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:12.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5f22a8640 1 -- 172.21.15.27:0/1427603025 shutdown_connections 2024-04-20T11:02:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.516+0000 7fa5f22a8640 1 -- 172.21.15.27:0/1427603025 wait complete. 2024-04-20T11:02:12.518 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.517+0000 7fa5f22a8640 1 Processor -- start 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.518+0000 7fa5f22a8640 1 -- start start 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.518+0000 7fa5f22a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ec0b5a30 0x7fa5ec06d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.518+0000 7fa5f22a8640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5ec0cc330 0x7fa5ec1f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.518+0000 7fa5f22a8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 0x7fa5ec1f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.518+0000 7fa5f22a8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec15fb90 con 0x7fa5ec155950 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.518+0000 7fa5f22a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec0b5e40 con 0x7fa5ec0b5a30 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.518+0000 7fa5f22a8640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ec15fd10 con 0x7fa5ec0cc330 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f0845640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 0x7fa5ec1f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f0845640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 0x7fa5ec1f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37412/0 (socket says 172.21.15.27:37412) 2024-04-20T11:02:12.520 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f0845640 1 -- 172.21.15.27:0/869544282 learned_addr learned my addr 172.21.15.27:0/869544282 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5ebfff640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ec0b5a30 0x7fa5ec06d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5eb7fe640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5ec0cc330 0x7fa5ec1f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f0845640 1 -- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5ec0cc330 msgr2=0x7fa5ec1f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f0845640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5ec0cc330 0x7fa5ec1f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f0845640 1 -- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ec0b5a30 msgr2=0x7fa5ec06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f0845640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ec0b5a30 0x7fa5ec06d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f0845640 1 -- 172.21.15.27:0/869544282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec1f5830 con 0x7fa5ec155950 2024-04-20T11:02:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5eb7fe640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5ec0cc330 0x7fa5ec1f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:02:12.521 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f0845640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 0x7fa5ec1f21d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400c760 tx=0x7fa5e400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5e8ff9640 1 -- 172.21.15.27:0/869544282 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4003b70 con 0x7fa5ec155950 2024-04-20T11:02:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5e8ff9640 1 -- 172.21.15.27:0/869544282 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e4003d30 con 0x7fa5ec155950 2024-04-20T11:02:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ec15caa0 con 0x7fa5ec155950 2024-04-20T11:02:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5e8ff9640 1 -- 172.21.15.27:0/869544282 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400aa90 con 0x7fa5ec155950 2024-04-20T11:02:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.519+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ec15d060 con 0x7fa5ec155950 2024-04-20T11:02:12.522 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.520+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ec15f9b0 con 0x7fa5ec155950 2024-04-20T11:02:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.521+0000 7fa5e8ff9640 1 -- 172.21.15.27:0/869544282 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5e400ac30 con 0x7fa5ec155950 2024-04-20T11:02:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.521+0000 7fa5e8ff9640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa5d806c8f0 0x7fa5d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.521+0000 7fa5e8ff9640 1 -- 172.21.15.27:0/869544282 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fa5e408bbe0 con 0x7fa5ec155950 2024-04-20T11:02:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.521+0000 7fa5ebfff640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa5d806c8f0 0x7fa5d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.522+0000 7fa5ebfff640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa5d806c8f0 0x7fa5d806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc036000 tx=0x7fa5dc008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.526+0000 7fa5e8ff9640 1 -- 172.21.15.27:0/869544282 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa5ec15f9b0 con 0x7fa5ec155950 2024-04-20T11:02:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.748+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5ec04fcf0 con 0x7fa5ec155950 2024-04-20T11:02:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.749+0000 7fa5e8ff9640 1 -- 172.21.15.27:0/869544282 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+6110 (secure 0 0 0) 0x7fa5e4059df0 con 0x7fa5ec155950 2024-04-20T11:02:12.749 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:12.749 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:07.909358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5029,4894,4293]},"id":1}]} 2024-04-20T11:02:12.750 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 45 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa5d806c8f0 msgr2=0x7fa5d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa5d806c8f0 0x7fa5d806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc036000 tx=0x7fa5dc008030 comp rx=0 tx=0).stop 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 msgr2=0x7fa5ec1f21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 0x7fa5ec1f21d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400c760 tx=0x7fa5e400cc20 comp rx=0 tx=0).stop 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 shutdown_connections 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fa5d806c8f0 0x7fa5d806edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5ec155950 0x7fa5ec1f21d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fa5ec0cc330 0x7fa5ec1f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 --2- 172.21.15.27:0/869544282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5ec0b5a30 0x7fa5ec06d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 >> 172.21.15.27:0/869544282 conn(0x7fa5ec0676c0 msgr2=0x7fa5ec0cfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 shutdown_connections 2024-04-20T11:02:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:12.751+0000 7fa5f22a8640 1 -- 172.21.15.27:0/869544282 wait complete. 2024-04-20T11:02:12.762 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-04-20T11:02:12.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:02:12.834 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:02:14.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.881+0000 7f78bcb84640 1 Processor -- start 2024-04-20T11:02:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.881+0000 7f78bcb84640 1 -- start start 2024-04-20T11:02:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78bcb84640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78b81556f0 0x7f78b815b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78bcb84640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f78b815f580 0x7f78b80cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78bcb84640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78b80c9e60 con 0x7f78b8070ca0 2024-04-20T11:02:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78bcb84640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f78b806ead0 con 0x7f78b81556f0 2024-04-20T11:02:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78bcb84640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f78b8071110 con 0x7f78b815f580 2024-04-20T11:02:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78b659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78b81556f0 0x7f78b815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78b659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78b81556f0 0x7f78b815b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35334/0 (socket says 172.21.15.27:35334) 2024-04-20T11:02:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78b659c640 1 -- 172.21.15.27:0/3094197608 learned_addr learned my addr 172.21.15.27:0/3094197608 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:14.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78b6d9d640 1 --2- 172.21.15.27:0/3094197608 >> v2:172.21.15.137:3300/0 conn(0x7f78b815f580 0x7f78b80cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78b659c640 1 -- 172.21.15.27:0/3094197608 >> v2:172.21.15.137:3300/0 conn(0x7f78b815f580 msgr2=0x7f78b80cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78b659c640 1 --2- 172.21.15.27:0/3094197608 >> v2:172.21.15.137:3300/0 conn(0x7f78b815f580 0x7f78b80cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78b659c640 1 -- 172.21.15.27:0/3094197608 >> v1:172.21.15.27:6789/0 conn(0x7f78b8070ca0 legacy=0x7f78b80714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.882+0000 7f78b659c640 1 -- 172.21.15.27:0/3094197608 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b806cce0 con 0x7f78b81556f0 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.883+0000 7f78b659c640 1 --2- 172.21.15.27:0/3094197608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b81556f0 0x7f78b815b580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f78b80b3c10 tx=0x7f78a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa914faec9fd6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.883+0000 7f78b5d9b640 1 -- 172.21.15.27:0/3094197608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8035020 con 0x7f78b81556f0 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.883+0000 7f78b5d9b640 1 -- 172.21.15.27:0/3094197608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a8031870 con 0x7f78b81556f0 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.883+0000 7f78b5d9b640 1 -- 172.21.15.27:0/3094197608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8031c40 con 0x7f78b81556f0 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.884+0000 7f78bcb84640 1 -- 172.21.15.27:0/3094197608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b81556f0 msgr2=0x7f78b815b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.884+0000 7f78bcb84640 1 --2- 172.21.15.27:0/3094197608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b81556f0 0x7f78b815b580 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f78b80b3c10 tx=0x7f78a802b910 comp rx=0 tx=0).stop 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.884+0000 7f78bcb84640 1 -- 172.21.15.27:0/3094197608 shutdown_connections 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.884+0000 7f78bcb84640 1 --2- 172.21.15.27:0/3094197608 >> v2:172.21.15.137:3300/0 conn(0x7f78b815f580 0x7f78b80cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:14.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.884+0000 7f78bcb84640 1 --2- 172.21.15.27:0/3094197608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b81556f0 0x7f78b815b580 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:14.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.884+0000 7f78bcb84640 1 -- 172.21.15.27:0/3094197608 >> 172.21.15.27:0/3094197608 conn(0x7f78b8067ae0 msgr2=0x7f78b8067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:14.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.884+0000 7f78bcb84640 1 -- 172.21.15.27:0/3094197608 shutdown_connections 2024-04-20T11:02:14.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.884+0000 7f78bcb84640 1 -- 172.21.15.27:0/3094197608 wait complete. 2024-04-20T11:02:14.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.885+0000 7f78bcb84640 1 Processor -- start 2024-04-20T11:02:14.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.885+0000 7f78bcb84640 1 -- start start 2024-04-20T11:02:14.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78bcb84640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b8070ca0 0x7f78b8158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:14.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78bcb84640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f78b81556f0 0x7f78b814ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:14.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78bcb84640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78b815f580 0x7f78b80cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:14.889 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78bcb84640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f78b806e480 con 0x7f78b8070ca0 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78bcb84640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f78b80c9b80 con 0x7f78b815f580 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78bcb84640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f78b806e760 con 0x7f78b81556f0 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78b659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b8070ca0 0x7f78b8158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78b659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b8070ca0 0x7f78b8158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35336/0 (socket says 172.21.15.27:35336) 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78b659c640 1 -- 172.21.15.27:0/560762012 learned_addr learned my addr 172.21.15.27:0/560762012 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78b6d9d640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78b815f580 0x7f78b80cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78affff640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f78b81556f0 0x7f78b814ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78b659c640 1 -- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f78b81556f0 msgr2=0x7f78b814ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78b659c640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f78b81556f0 0x7f78b814ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78b659c640 1 -- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78b815f580 msgr2=0x7f78b80cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:14.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78b659c640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78b815f580 0x7f78b80cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.886+0000 7f78b659c640 1 -- 172.21.15.27:0/560762012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b81e8440 con 0x7f78b8070ca0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.887+0000 7f78b659c640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b8070ca0 0x7f78b8158440 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f78a802b950 tx=0x7f78a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.887+0000 7f78aeffd640 1 -- 172.21.15.27:0/560762012 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8035040 con 0x7f78b8070ca0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.887+0000 7f78aeffd640 1 -- 172.21.15.27:0/560762012 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a8030d00 con 0x7f78b8070ca0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.887+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b81e8720 con 0x7f78b8070ca0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.887+0000 7f78aeffd640 1 -- 172.21.15.27:0/560762012 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a80456a0 con 0x7f78b8070ca0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.887+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b814f010 con 0x7f78b8070ca0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.888+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b80688e0 con 0x7f78b8070ca0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.888+0000 7f78aeffd640 1 -- 172.21.15.27:0/560762012 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78a8038020 con 0x7f78b8070ca0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.889+0000 7f78aeffd640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f789806c8f0 0x7f789806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.889+0000 7f78aeffd640 1 -- 172.21.15.27:0/560762012 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f78a80ade30 con 0x7f78b8070ca0 2024-04-20T11:02:14.891 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.889+0000 7f78affff640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f789806c8f0 0x7f789806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:14.892 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.889+0000 7f78affff640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f789806c8f0 0x7f789806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78b81f57c0 tx=0x7f78a0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:14.893 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:14.893+0000 7f78aeffd640 1 -- 172.21.15.27:0/560762012 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f78a807ba80 con 0x7f78b8070ca0 2024-04-20T11:02:15.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.118+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78b804fcf0 con 0x7f78b8070ca0 2024-04-20T11:02:15.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.118+0000 7f78aeffd640 1 -- 172.21.15.27:0/560762012 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+6115 (secure 0 0 0) 0x7f78a807fd80 con 0x7f78b8070ca0 2024-04-20T11:02:15.118 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:15.119 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:12.937424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4894,5029,5406,5143]},"id":1}]} 2024-04-20T11:02:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f789806c8f0 msgr2=0x7f789806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f789806c8f0 0x7f789806edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f78b81f57c0 tx=0x7f78a0009040 comp rx=0 tx=0).stop 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b8070ca0 msgr2=0x7f78b8158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b8070ca0 0x7f78b8158440 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f78a802b950 tx=0x7f78a80046c0 comp rx=0 tx=0).stop 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 shutdown_connections 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f789806c8f0 0x7f789806edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78b815f580 0x7f78b80cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f78b81556f0 0x7f78b814ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 --2- 172.21.15.27:0/560762012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78b8070ca0 0x7f78b8158440 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 >> 172.21.15.27:0/560762012 conn(0x7f78b8067ae0 msgr2=0x7f78b806f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 shutdown_connections 2024-04-20T11:02:15.122 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:15.120+0000 7f78bcb84640 1 -- 172.21.15.27:0/560762012 wait complete. 2024-04-20T11:02:15.131 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=1, replacing gid=4278 2024-04-20T11:02:15.131 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.a 2024-04-20T11:02:15.677 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-20T11:02:15.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-20T11:02:15.965 DEBUG:teuthology.parallel:result is None 2024-04-20T11:02:15.966 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-20T11:02:15.995 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-20T11:02:15.996 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2024-04-20T11:02:16.010 INFO:tasks.mds_thrash:joining mds_thrasher 2024-04-20T11:02:16.344 INFO:tasks.ceph.mds.d.smithi137.stderr:2024-04-20T11:02:16.345+0000 7f8f0d139640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-20T11:02:22.132 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-04-20T11:02:22.132 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T11:02:22.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T11:02:22.135 INFO:tasks.ceph.mds.a:Started 2024-04-20T11:02:22.135 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:22.222 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2024-04-20T11:02:22.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.249+0000 7f4dba1f4640 1 Processor -- start 2024-04-20T11:02:22.249 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.249+0000 7f4dba1f4640 1 -- start start 2024-04-20T11:02:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4dba1f4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db4155950 0x7f4db40cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4dba1f4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4db40cc330 0x7f4db40b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4dba1f4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f4db40b5a30 0x7f4db40d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4dba1f4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f4db40b5ea0 con 0x7f4db40cc330 2024-04-20T11:02:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4dba1f4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f4db415b9c0 con 0x7f4db40b5a30 2024-04-20T11:02:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4dba1f4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f4db415bca0 con 0x7f4db4155950 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4db40cc330 0x7f4db40b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4db40cc330 0x7f4db40b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35358/0 (socket says 172.21.15.27:35358) 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db2ffd640 1 -- 172.21.15.27:0/849894085 learned_addr learned my addr 172.21.15.27:0/849894085 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db3fff640 1 --2- 172.21.15.27:0/849894085 >> v2:172.21.15.137:3300/0 conn(0x7f4db40b5a30 0x7f4db40d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db37fe640 1 --2- 172.21.15.27:0/849894085 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db4155950 0x7f4db40cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db2ffd640 1 -- 172.21.15.27:0/849894085 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db4155950 msgr2=0x7f4db40cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db2ffd640 1 --2- 172.21.15.27:0/849894085 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db4155950 0x7f4db40cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db2ffd640 1 -- 172.21.15.27:0/849894085 >> v2:172.21.15.137:3300/0 conn(0x7f4db40b5a30 msgr2=0x7f4db40d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db2ffd640 1 --2- 172.21.15.27:0/849894085 >> v2:172.21.15.137:3300/0 conn(0x7f4db40b5a30 0x7f4db40d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.250+0000 7f4db2ffd640 1 -- 172.21.15.27:0/849894085 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db415bf80 con 0x7f4db40cc330 2024-04-20T11:02:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.251+0000 7f4db2ffd640 1 --2- 172.21.15.27:0/849894085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 0x7f4db40b54f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4db408e4b0 tx=0x7f4da802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e201d104427f311e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.251+0000 7f4db27fc640 1 -- 172.21.15.27:0/849894085 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8034070 con 0x7f4db40cc330 2024-04-20T11:02:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.251+0000 7f4db27fc640 1 -- 172.21.15.27:0/849894085 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da8030860 con 0x7f4db40cc330 2024-04-20T11:02:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.251+0000 7f4db27fc640 1 -- 172.21.15.27:0/849894085 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8030c30 con 0x7f4db40cc330 2024-04-20T11:02:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.252+0000 7f4dba1f4640 1 -- 172.21.15.27:0/849894085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 msgr2=0x7f4db40b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.252+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/849894085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 0x7f4db40b54f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4db408e4b0 tx=0x7f4da802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.252+0000 7f4dba1f4640 1 -- 172.21.15.27:0/849894085 shutdown_connections 2024-04-20T11:02:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.252+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/849894085 >> v2:172.21.15.137:3300/0 conn(0x7f4db40b5a30 0x7f4db40d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.252+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/849894085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 0x7f4db40b54f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.252+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/849894085 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db4155950 0x7f4db40cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.252+0000 7f4dba1f4640 1 -- 172.21.15.27:0/849894085 >> 172.21.15.27:0/849894085 conn(0x7f4db40676c0 msgr2=0x7f4db4067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.252+0000 7f4dba1f4640 1 -- 172.21.15.27:0/849894085 shutdown_connections 2024-04-20T11:02:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.252+0000 7f4dba1f4640 1 -- 172.21.15.27:0/849894085 wait complete. 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.253+0000 7f4dba1f4640 1 Processor -- start 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.253+0000 7f4dba1f4640 1 -- start start 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.254+0000 7f4dba1f4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40b5a30 0x7f4db40c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.254+0000 7f4dba1f4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 0x7f4db41526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.254+0000 7f4dba1f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db4155950 0x7f4db4152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.254+0000 7f4dba1f4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4db415fb90 con 0x7f4db40cc330 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.254+0000 7f4dba1f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4db415c540 con 0x7f4db4155950 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.254+0000 7f4dba1f4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f4db415fd10 con 0x7f4db40b5a30 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.254+0000 7f4db2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 0x7f4db41526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.254+0000 7f4db2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 0x7f4db41526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35360/0 (socket says 172.21.15.27:35360) 2024-04-20T11:02:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.254+0000 7f4db2ffd640 1 -- 172.21.15.27:0/225934110 learned_addr learned my addr 172.21.15.27:0/225934110 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4db2ffd640 1 -- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40b5a30 msgr2=0x7f4db40c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4db37fe640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40b5a30 0x7f4db40c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4db3fff640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db4155950 0x7f4db4152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4db2ffd640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40b5a30 0x7f4db40c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4db2ffd640 1 -- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db4155950 msgr2=0x7f4db4152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4db2ffd640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db4155950 0x7f4db4152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4db2ffd640 1 -- 172.21.15.27:0/225934110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db4077100 con 0x7f4db40cc330 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4db2ffd640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 0x7f4db41526b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4da8000c00 tx=0x7f4da80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4d9bfff640 1 -- 172.21.15.27:0/225934110 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8034040 con 0x7f4db40cc330 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4d9bfff640 1 -- 172.21.15.27:0/225934110 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da8039d90 con 0x7f4db40cc330 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4d9bfff640 1 -- 172.21.15.27:0/225934110 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8043610 con 0x7f4db40cc330 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db40c8000 con 0x7f4db40cc330 2024-04-20T11:02:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.255+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db40cac60 con 0x7f4db40cc330 2024-04-20T11:02:22.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.256+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db404fcf0 con 0x7f4db40cc330 2024-04-20T11:02:22.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.256+0000 7f4d9bfff640 1 -- 172.21.15.27:0/225934110 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4da8037070 con 0x7f4db40cc330 2024-04-20T11:02:22.257 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.257+0000 7f4d9bfff640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4da006c920 0x7f4da006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:22.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.257+0000 7f4d9bfff640 1 -- 172.21.15.27:0/225934110 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f4da80adee0 con 0x7f4db40cc330 2024-04-20T11:02:22.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.257+0000 7f4db37fe640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4da006c920 0x7f4da006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:22.258 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.258+0000 7f4db37fe640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4da006c920 0x7f4da006ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006fd0 tx=0x7f4da4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:22.261 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.262+0000 7f4d9bfff640 1 -- 172.21.15.27:0/225934110 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4db4077100 con 0x7f4db40cc330 2024-04-20T11:02:22.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.492+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4db40b6ca0 con 0x7f4db40cc330 2024-04-20T11:02:22.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.493+0000 7f4d9bfff640 1 -- 172.21.15.27:0/225934110 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+6115 (secure 0 0 0) 0x7f4da803e070 con 0x7f4db40cc330 2024-04-20T11:02:22.492 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:22.493 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:12.937424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4894,5029,5406,5143]},"id":1}]} 2024-04-20T11:02:22.493 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4da006c920 msgr2=0x7f4da006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4da006c920 0x7f4da006ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006fd0 tx=0x7f4da4008040 comp rx=0 tx=0).stop 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 msgr2=0x7f4db41526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 0x7f4db41526b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4da8000c00 tx=0x7f4da80046c0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 shutdown_connections 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4da006c920 0x7f4da006ede0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4db4155950 0x7f4db4152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4db40cc330 0x7f4db41526b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 --2- 172.21.15.27:0/225934110 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4db40b5a30 0x7f4db40c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 >> 172.21.15.27:0/225934110 conn(0x7f4db40676c0 msgr2=0x7f4db4067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 shutdown_connections 2024-04-20T11:02:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:22.495+0000 7f4dba1f4640 1 -- 172.21.15.27:0/225934110 wait complete. 2024-04-20T11:02:22.506 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-04-20T11:02:24.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:24.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.621+0000 7f800fb9d640 1 Processor -- start 2024-04-20T11:02:24.621 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.622+0000 7f800fb9d640 1 -- start start 2024-04-20T11:02:24.622 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.622+0000 7f800fb9d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f800815f230 0x7f800815c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.622+0000 7f800fb9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f80080ccbd0 0x7f8008155410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.622+0000 7f800fb9d640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800815f9a0 con 0x7f800815ee50 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.622+0000 7f800fb9d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f80080cb870 con 0x7f80080ccbd0 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.622+0000 7f800fb9d640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f80080cbb30 con 0x7f800815f230 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.622+0000 7f800e13a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f80080ccbd0 0x7f8008155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.622+0000 7f800e13a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f80080ccbd0 0x7f8008155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38566/0 (socket says 172.21.15.27:38566) 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.622+0000 7f800e13a640 1 -- 172.21.15.27:0/3322042198 learned_addr learned my addr 172.21.15.27:0/3322042198 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.623+0000 7f800d138640 1 --2- 172.21.15.27:0/3322042198 >> v2:172.21.15.137:3300/0 conn(0x7f800815f230 0x7f800815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.623+0000 7f800e13a640 1 -- 172.21.15.27:0/3322042198 >> v2:172.21.15.137:3300/0 conn(0x7f800815f230 msgr2=0x7f800815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.623+0000 7f800e13a640 1 --2- 172.21.15.27:0/3322042198 >> v2:172.21.15.137:3300/0 conn(0x7f800815f230 0x7f800815c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.623+0000 7f800e13a640 1 -- 172.21.15.27:0/3322042198 >> v1:172.21.15.137:6789/0 conn(0x7f800815ee50 legacy=0x7f800806cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.623+0000 7f800e13a640 1 -- 172.21.15.27:0/3322042198 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80080cbe10 con 0x7f80080ccbd0 2024-04-20T11:02:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.623+0000 7f800d138640 1 --2- 172.21.15.27:0/3322042198 >> v2:172.21.15.137:3300/0 conn(0x7f800815f230 0x7f800815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800e13a640 1 --2- 172.21.15.27:0/3322042198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80080ccbd0 0x7f8008155410 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f80081592a0 tx=0x7f800002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b36b8769e36f770 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800c937640 1 -- 172.21.15.27:0/3322042198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800002cb20 con 0x7f80080ccbd0 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800c937640 1 -- 172.21.15.27:0/3322042198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800002ccc0 con 0x7f80080ccbd0 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800c937640 1 -- 172.21.15.27:0/3322042198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000034610 con 0x7f80080ccbd0 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800fb9d640 1 -- 172.21.15.27:0/3322042198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80080ccbd0 msgr2=0x7f8008155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800fb9d640 1 --2- 172.21.15.27:0/3322042198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80080ccbd0 0x7f8008155410 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f80081592a0 tx=0x7f800002bf80 comp rx=0 tx=0).stop 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800fb9d640 1 -- 172.21.15.27:0/3322042198 shutdown_connections 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800fb9d640 1 --2- 172.21.15.27:0/3322042198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80080ccbd0 0x7f8008155410 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800fb9d640 1 --2- 172.21.15.27:0/3322042198 >> v2:172.21.15.137:3300/0 conn(0x7f800815f230 0x7f800815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800fb9d640 1 -- 172.21.15.27:0/3322042198 >> 172.21.15.27:0/3322042198 conn(0x7f800806e010 msgr2=0x7f80080d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800fb9d640 1 -- 172.21.15.27:0/3322042198 shutdown_connections 2024-04-20T11:02:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.624+0000 7f800fb9d640 1 -- 172.21.15.27:0/3322042198 wait complete. 2024-04-20T11:02:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800fb9d640 1 Processor -- start 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800fb9d640 1 -- start start 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800fb9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80080ccbd0 0x7f8008151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800fb9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f800815ee50 0x7f800814e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800fb9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800815f230 0x7f800814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800fb9d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f80080cc6b0 con 0x7f800815f230 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800fb9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f80080cc3d0 con 0x7f80080ccbd0 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800fb9d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f800815cae0 con 0x7f800815ee50 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800e13a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800815f230 0x7f800814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800e13a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800815f230 0x7f800814ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38570/0 (socket says 172.21.15.27:38570) 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.626+0000 7f800e13a640 1 -- 172.21.15.27:0/2787482096 learned_addr learned my addr 172.21.15.27:0/2787482096 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800d939640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80080ccbd0 0x7f8008151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800d138640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f800815ee50 0x7f800814e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800e13a640 1 -- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f800815ee50 msgr2=0x7f800814e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800e13a640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f800815ee50 0x7f800814e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800e13a640 1 -- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80080ccbd0 msgr2=0x7f8008151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800e13a640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80080ccbd0 0x7f8008151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800e13a640 1 -- 172.21.15.27:0/2787482096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8008157b70 con 0x7f800815f230 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800d138640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f800815ee50 0x7f800814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800d939640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80080ccbd0 0x7f8008151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800e13a640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800815f230 0x7f800814ed40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8000009f40 tx=0x7f8000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f7ffe7fc640 1 -- 172.21.15.27:0/2787482096 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000043020 con 0x7f800815f230 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f7ffe7fc640 1 -- 172.21.15.27:0/2787482096 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800003e450 con 0x7f800815f230 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8008157de0 con 0x7f800815f230 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f7ffe7fc640 1 -- 172.21.15.27:0/2787482096 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800003e820 con 0x7f800815f230 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.627+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8008158c10 con 0x7f800815f230 2024-04-20T11:02:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.628+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80080688e0 con 0x7f800815f230 2024-04-20T11:02:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.629+0000 7f7ffe7fc640 1 -- 172.21.15.27:0/2787482096 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8000046020 con 0x7f800815f230 2024-04-20T11:02:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.629+0000 7f7ffe7fc640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7fec06c8f0 0x7f7fec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.629+0000 7f7ffe7fc640 1 -- 172.21.15.27:0/2787482096 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f80000ae740 con 0x7f800815f230 2024-04-20T11:02:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.629+0000 7f800d939640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7fec06c8f0 0x7f7fec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:24.630 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.630+0000 7f800d939640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7fec06c8f0 0x7f7fec06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8004006fd0 tx=0x7f8004008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.634+0000 7f7ffe7fc640 1 -- 172.21.15.27:0/2787482096 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f800004d030 con 0x7f800815f230 2024-04-20T11:02:24.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.858+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80081f0fc0 con 0x7f800815f230 2024-04-20T11:02:24.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.858+0000 7f7ffe7fc640 1 -- 172.21.15.27:0/2787482096 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+6939 (secure 0 0 0) 0x7f80081f0fc0 con 0x7f800815f230 2024-04-20T11:02:24.858 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:24.858 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6836/1455214274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1455214274},{"type":"v1","addr":"172.21.15.27:6836","nonce":1455214274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:12.937424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5406,5029,4894,4293]},"id":1}]} 2024-04-20T11:02:24.859 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 47 2024-04-20T11:02:24.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.860+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7fec06c8f0 msgr2=0x7f7fec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.860+0000 7f800fb9d640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7fec06c8f0 0x7f7fec06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8004006fd0 tx=0x7f8004008040 comp rx=0 tx=0).stop 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.860+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800815f230 msgr2=0x7f800814ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.860+0000 7f800fb9d640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800815f230 0x7f800814ed40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8000009f40 tx=0x7f8000004a40 comp rx=0 tx=0).stop 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.861+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 shutdown_connections 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.861+0000 7f800fb9d640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f7fec06c8f0 0x7f7fec06edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.861+0000 7f800fb9d640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800815f230 0x7f800814ed40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.861+0000 7f800fb9d640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f800815ee50 0x7f800814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.861+0000 7f800fb9d640 1 --2- 172.21.15.27:0/2787482096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80080ccbd0 0x7f8008151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.861+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 >> 172.21.15.27:0/2787482096 conn(0x7f800806e010 msgr2=0x7f8008152ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.861+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 shutdown_connections 2024-04-20T11:02:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:24.861+0000 7f800fb9d640 1 -- 172.21.15.27:0/2787482096 wait complete. 2024-04-20T11:02:24.871 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-04-20T11:02:24.871 INFO:tasks.mds_thrash.fs.[cephfs]:stat['max_mds'] = 0 2024-04-20T11:02:24.872 INFO:tasks.mds_thrash.fs.[cephfs]:stat['deactivate'] = 0 2024-04-20T11:02:24.872 INFO:tasks.mds_thrash.fs.[cephfs]:stat['kill'] = 5 2024-04-20T11:02:24.872 INFO:tasks.mds_thrash:done joining 2024-04-20T11:02:24.872 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-20T11:02:24.882 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-20T11:02:24.882 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T11:02:24.882 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:02:24.898 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-04-20T11:02:24.898 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T11:02:24.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:02:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.095+0000 7f5b46b23640 1 Processor -- start 2024-04-20T11:02:25.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.095+0000 7f5b46b23640 1 -- start start 2024-04-20T11:02:25.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.095+0000 7f5b46b23640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5b401556f0 0x7f5b4006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.095+0000 7f5b46b23640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5b4015f710 0x7f5b4015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.095+0000 7f5b46b23640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b4015ec70 con 0x7f5b40071570 2024-04-20T11:02:25.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.095+0000 7f5b46b23640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5b4006ead0 con 0x7f5b4015f710 2024-04-20T11:02:25.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.095+0000 7f5b46b23640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f5b4006e510 con 0x7f5b401556f0 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.096+0000 7f5b3ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5b4015f710 0x7f5b4015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.096+0000 7f5b3ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5b4015f710 0x7f5b4015c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38574/0 (socket says 172.21.15.27:38574) 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.096+0000 7f5b3ffff640 1 -- 172.21.15.27:0/1536911952 learned_addr learned my addr 172.21.15.27:0/1536911952 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.096+0000 7f5b448bf640 1 --2- 172.21.15.27:0/1536911952 >> v2:172.21.15.137:3300/0 conn(0x7f5b401556f0 0x7f5b4006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.096+0000 7f5b3ffff640 1 -- 172.21.15.27:0/1536911952 >> v2:172.21.15.137:3300/0 conn(0x7f5b401556f0 msgr2=0x7f5b4006ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.096+0000 7f5b3ffff640 1 --2- 172.21.15.27:0/1536911952 >> v2:172.21.15.137:3300/0 conn(0x7f5b401556f0 0x7f5b4006ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.096+0000 7f5b3ffff640 1 -- 172.21.15.27:0/1536911952 >> v1:172.21.15.137:6789/0 conn(0x7f5b40071570 legacy=0x7f5b400cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.096+0000 7f5b3ffff640 1 -- 172.21.15.27:0/1536911952 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4006e7f0 con 0x7f5b4015f710 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b3ffff640 1 --2- 172.21.15.27:0/1536911952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 0x7f5b4015c9d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5b400c94c0 tx=0x7f5b2c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37124af2c7a4bf08 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b3f7fe640 1 -- 172.21.15.27:0/1536911952 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c02f850 con 0x7f5b4015f710 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b3f7fe640 1 -- 172.21.15.27:0/1536911952 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2c02f9f0 con 0x7f5b4015f710 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b3f7fe640 1 -- 172.21.15.27:0/1536911952 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c02fdc0 con 0x7f5b4015f710 2024-04-20T11:02:25.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b46b23640 1 -- 172.21.15.27:0/1536911952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 msgr2=0x7f5b4015c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b46b23640 1 --2- 172.21.15.27:0/1536911952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 0x7f5b4015c9d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5b400c94c0 tx=0x7f5b2c02ba30 comp rx=0 tx=0).stop 2024-04-20T11:02:25.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b46b23640 1 -- 172.21.15.27:0/1536911952 shutdown_connections 2024-04-20T11:02:25.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b46b23640 1 --2- 172.21.15.27:0/1536911952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 0x7f5b4015c9d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b46b23640 1 --2- 172.21.15.27:0/1536911952 >> v2:172.21.15.137:3300/0 conn(0x7f5b401556f0 0x7f5b4006ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b46b23640 1 -- 172.21.15.27:0/1536911952 >> 172.21.15.27:0/1536911952 conn(0x7f5b40067ae0 msgr2=0x7f5b40067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:25.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b46b23640 1 -- 172.21.15.27:0/1536911952 shutdown_connections 2024-04-20T11:02:25.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.097+0000 7f5b46b23640 1 -- 172.21.15.27:0/1536911952 wait complete. 2024-04-20T11:02:25.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.098+0000 7f5b46b23640 1 Processor -- start 2024-04-20T11:02:25.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.099+0000 7f5b46b23640 1 -- start start 2024-04-20T11:02:25.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.099+0000 7f5b46b23640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b40071570 0x7f5b40150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.099+0000 7f5b46b23640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b401556f0 0x7f5b4015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.099+0000 7f5b46b23640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 0x7f5b4015d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.099+0000 7f5b46b23640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b400719c0 con 0x7f5b4015f710 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.099+0000 7f5b46b23640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4015f1f0 con 0x7f5b401556f0 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.099+0000 7f5b46b23640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5b400c9800 con 0x7f5b40071570 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b450c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 0x7f5b4015d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b450c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 0x7f5b4015d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38582/0 (socket says 172.21.15.27:38582) 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b450c0640 1 -- 172.21.15.27:0/210201995 learned_addr learned my addr 172.21.15.27:0/210201995 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b448bf640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b40071570 0x7f5b40150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b3ffff640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b401556f0 0x7f5b4015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b450c0640 1 -- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b40071570 msgr2=0x7f5b40150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b450c0640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b40071570 0x7f5b40150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b450c0640 1 -- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b401556f0 msgr2=0x7f5b4015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b450c0640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b401556f0 0x7f5b4015d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b450c0640 1 -- 172.21.15.27:0/210201995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b40158b80 con 0x7f5b4015f710 2024-04-20T11:02:25.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b450c0640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 0x7f5b4015d8e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800c7d0 tx=0x7f5b3800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:25.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b3d7fa640 1 -- 172.21.15.27:0/210201995 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38003b70 con 0x7f5b4015f710 2024-04-20T11:02:25.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b3d7fa640 1 -- 172.21.15.27:0/210201995 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b38003d30 con 0x7f5b4015f710 2024-04-20T11:02:25.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b401e5bf0 con 0x7f5b4015f710 2024-04-20T11:02:25.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b3d7fa640 1 -- 172.21.15.27:0/210201995 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3800aa90 con 0x7f5b4015f710 2024-04-20T11:02:25.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.100+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b401e61b0 con 0x7f5b4015f710 2024-04-20T11:02:25.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.101+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b400688e0 con 0x7f5b4015f710 2024-04-20T11:02:25.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.102+0000 7f5b3d7fa640 1 -- 172.21.15.27:0/210201995 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b3800ac30 con 0x7f5b4015f710 2024-04-20T11:02:25.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.102+0000 7f5b3d7fa640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b2406c920 0x7f5b2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.103+0000 7f5b3d7fa640 1 -- 172.21.15.27:0/210201995 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f5b3808bb50 con 0x7f5b4015f710 2024-04-20T11:02:25.105 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.103+0000 7f5b448bf640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b2406c920 0x7f5b2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.103+0000 7f5b448bf640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b2406c920 0x7f5b2406ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5b34006fd0 tx=0x7f5b34008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:25.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.107+0000 7f5b3d7fa640 1 -- 172.21.15.27:0/210201995 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5b38059710 con 0x7f5b4015f710 2024-04-20T11:02:25.291 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.291+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b40070730 con 0x7f5b4015f710 2024-04-20T11:02:25.292 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.292+0000 7f5b3d7fa640 1 -- 172.21.15.27:0/210201995 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+17470 (secure 0 0 0) 0x7f5b40070730 con 0x7f5b4015f710 2024-04-20T11:02:25.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:25.293 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T11:01:52.359149+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6836/4195543922":"2024-04-21T11:01:52.359070+0000","172.21.15.27:6833/4195543922":"2024-04-21T11:01:52.359070+0000","172.21.15.27:6837/1680990583":"2024-04-21T11:00:42.348974+0000","172.21.15.27:6834/1680990583":"2024-04-21T11:00:42.348974+0000","172.21.15.27:6832/859496751":"2024-04-21T10:59:37.339287+0000","172.21.15.137:6835/1164978945":"2024-04-21T10:58:22.329607+0000","172.21.15.137:6834/1164978945":"2024-04-21T10:58:22.329607+0000","172.21.15.27:6835/859496751":"2024-04-21T10:59:37.339287+0000","172.21.15.137:6838/1990627502":"2024-04-21T10:57:27.321673+0000","172.21.15.137:6836/1990627502":"2024-04-21T10:57:27.321673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.294+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b2406c920 msgr2=0x7f5b2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.294+0000 7f5b46b23640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b2406c920 0x7f5b2406ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5b34006fd0 tx=0x7f5b34008040 comp rx=0 tx=0).stop 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.294+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 msgr2=0x7f5b4015d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.294+0000 7f5b46b23640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 0x7f5b4015d8e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800c7d0 tx=0x7f5b3800cc90 comp rx=0 tx=0).stop 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.294+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 shutdown_connections 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.294+0000 7f5b46b23640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5b2406c920 0x7f5b2406ede0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.295+0000 7f5b46b23640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b4015f710 0x7f5b4015d8e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.295+0000 7f5b46b23640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b401556f0 0x7f5b4015d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.295+0000 7f5b46b23640 1 --2- 172.21.15.27:0/210201995 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5b40071570 0x7f5b40150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.295+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 >> 172.21.15.27:0/210201995 conn(0x7f5b40067ae0 msgr2=0x7f5b400c53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.295+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 shutdown_connections 2024-04-20T11:02:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.295+0000 7f5b46b23640 1 -- 172.21.15.27:0/210201995 wait complete. 2024-04-20T11:02:25.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:55:29.717620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:55:39.832461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:55:41.103975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T11:02:25.306 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T11:02:25.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.410+0000 7f8120c70640 1 Processor -- start 2024-04-20T11:02:25.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.410+0000 7f8120c70640 1 -- start start 2024-04-20T11:02:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.411+0000 7f8120c70640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f811c155a20 0x7f811c0c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.411+0000 7f8120c70640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f811c15ec40 0x7f811c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.411+0000 7f8120c70640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f811c15f210 con 0x7f811c0c62e0 2024-04-20T11:02:25.412 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.411+0000 7f8120c70640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f811c155e00 con 0x7f811c15ec40 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.411+0000 7f8120c70640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f811c15ba30 con 0x7f811c155a20 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.411+0000 7f811ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f811c15ec40 0x7f811c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.411+0000 7f811ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f811c15ec40 0x7f811c0cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38592/0 (socket says 172.21.15.27:38592) 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.411+0000 7f811ad9d640 1 -- 172.21.15.27:0/3822111369 learned_addr learned my addr 172.21.15.27:0/3822111369 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.412+0000 7f811a59c640 1 --2- 172.21.15.27:0/3822111369 >> v2:172.21.15.137:3300/0 conn(0x7f811c155a20 0x7f811c0c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.412+0000 7f811ad9d640 1 -- 172.21.15.27:0/3822111369 >> v2:172.21.15.137:3300/0 conn(0x7f811c155a20 msgr2=0x7f811c0c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.412+0000 7f811ad9d640 1 --2- 172.21.15.27:0/3822111369 >> v2:172.21.15.137:3300/0 conn(0x7f811c155a20 0x7f811c0c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.412+0000 7f811ad9d640 1 -- 172.21.15.27:0/3822111369 >> v1:172.21.15.137:6789/0 conn(0x7f811c0c62e0 legacy=0x7f811c15bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.412+0000 7f811ad9d640 1 -- 172.21.15.27:0/3822111369 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811c15f4f0 con 0x7f811c15ec40 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.412+0000 7f811ad9d640 1 --2- 172.21.15.27:0/3822111369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 0x7f811c0cbe00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f811c0cbb20 tx=0x7f810c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156fc1fd6293236b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f811959a640 1 -- 172.21.15.27:0/3822111369 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c02cbb0 con 0x7f811c15ec40 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f811959a640 1 -- 172.21.15.27:0/3822111369 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810c02cd50 con 0x7f811c15ec40 2024-04-20T11:02:25.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f811959a640 1 -- 172.21.15.27:0/3822111369 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c034610 con 0x7f811c15ec40 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f8120c70640 1 -- 172.21.15.27:0/3822111369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 msgr2=0x7f811c0cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f8120c70640 1 --2- 172.21.15.27:0/3822111369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 0x7f811c0cbe00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f811c0cbb20 tx=0x7f810c02c010 comp rx=0 tx=0).stop 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f8120c70640 1 -- 172.21.15.27:0/3822111369 shutdown_connections 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f8120c70640 1 --2- 172.21.15.27:0/3822111369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 0x7f811c0cbe00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f8120c70640 1 --2- 172.21.15.27:0/3822111369 >> v2:172.21.15.137:3300/0 conn(0x7f811c155a20 0x7f811c0c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f8120c70640 1 -- 172.21.15.27:0/3822111369 >> 172.21.15.27:0/3822111369 conn(0x7f811c0678a0 msgr2=0x7f811c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f8120c70640 1 -- 172.21.15.27:0/3822111369 shutdown_connections 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.413+0000 7f8120c70640 1 -- 172.21.15.27:0/3822111369 wait complete. 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f8120c70640 1 Processor -- start 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f8120c70640 1 -- start start 2024-04-20T11:02:25.416 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f8120c70640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f811c0c62e0 0x7f811c1e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f8120c70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f811c155a20 0x7f811c15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f8120c70640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 0x7f811c14a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f8120c70640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f811c15c5d0 con 0x7f811c15ec40 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f8120c70640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f811c15fab0 con 0x7f811c155a20 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f8120c70640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f811c0c67c0 con 0x7f811c0c62e0 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f811ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 0x7f811c14a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f811ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 0x7f811c14a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38604/0 (socket says 172.21.15.27:38604) 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.415+0000 7f811ad9d640 1 -- 172.21.15.27:0/608456219 learned_addr learned my addr 172.21.15.27:0/608456219 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f8119d9b640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f811c155a20 0x7f811c15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f811a59c640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f811c0c62e0 0x7f811c1e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f811ad9d640 1 -- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f811c0c62e0 msgr2=0x7f811c1e51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f811ad9d640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f811c0c62e0 0x7f811c1e51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.417 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f811ad9d640 1 -- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f811c155a20 msgr2=0x7f811c15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f811ad9d640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f811c155a20 0x7f811c15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f811ad9d640 1 -- 172.21.15.27:0/608456219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811c1f17b0 con 0x7f811c15ec40 2024-04-20T11:02:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f811ad9d640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 0x7f811c14a6e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f810c009f40 tx=0x7f810c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f810affd640 1 -- 172.21.15.27:0/608456219 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c043020 con 0x7f811c15ec40 2024-04-20T11:02:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f810affd640 1 -- 172.21.15.27:0/608456219 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810c03e450 con 0x7f811c15ec40 2024-04-20T11:02:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811c1e60d0 con 0x7f811c15ec40 2024-04-20T11:02:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f810affd640 1 -- 172.21.15.27:0/608456219 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c009510 con 0x7f811c15ec40 2024-04-20T11:02:25.418 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.416+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f811c1e6690 con 0x7f811c15ec40 2024-04-20T11:02:25.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.417+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f811c06e690 con 0x7f811c15ec40 2024-04-20T11:02:25.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.418+0000 7f810affd640 1 -- 172.21.15.27:0/608456219 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f810c046020 con 0x7f811c15ec40 2024-04-20T11:02:25.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.418+0000 7f810affd640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f80fc06c920 0x7f80fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.419 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.418+0000 7f810affd640 1 -- 172.21.15.27:0/608456219 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f810c0ae3b0 con 0x7f811c15ec40 2024-04-20T11:02:25.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.418+0000 7f811a59c640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f80fc06c920 0x7f80fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.420 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.419+0000 7f811a59c640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f80fc06c920 0x7f80fc06ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8114009b40 tx=0x7f8114009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:25.422 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.422+0000 7f810affd640 1 -- 172.21.15.27:0/608456219 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f811c06e690 con 0x7f811c15ec40 2024-04-20T11:02:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.605+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f811c067c80 con 0x7f811c15ec40 2024-04-20T11:02:25.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.606+0000 7f810affd640 1 -- 172.21.15.27:0/608456219 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f811c067c80 con 0x7f811c15ec40 2024-04-20T11:02:25.606 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f80fc06c920 msgr2=0x7f80fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f80fc06c920 0x7f80fc06ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8114009b40 tx=0x7f8114009190 comp rx=0 tx=0).stop 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 msgr2=0x7f811c14a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 0x7f811c14a6e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f810c009f40 tx=0x7f810c004a40 comp rx=0 tx=0).stop 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 shutdown_connections 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f80fc06c920 0x7f80fc06ede0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f811c15ec40 0x7f811c14a6e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f811c155a20 0x7f811c15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 --2- 172.21.15.27:0/608456219 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f811c0c62e0 0x7f811c1e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 >> 172.21.15.27:0/608456219 conn(0x7f811c0678a0 msgr2=0x7f811c06df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 shutdown_connections 2024-04-20T11:02:25.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.608+0000 7f8120c70640 1 -- 172.21.15.27:0/608456219 wait complete. 2024-04-20T11:02:25.619 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T11:02:25.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.722+0000 7f4c5a2e4640 1 Processor -- start 2024-04-20T11:02:25.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.722+0000 7f4c5a2e4640 1 -- start start 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.723+0000 7f4c5a2e4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c540c9f50 0x7f4c5415ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.723+0000 7f4c5a2e4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5415b3e0 con 0x7f4c541553f0 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.723+0000 7f4c5a2e4640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5415b680 con 0x7f4c54071f60 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.723+0000 7f4c5a2e4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5415e9d0 con 0x7f4c540c9f50 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.723+0000 7f4c53fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4c541553f0 0x7f4c5406fb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58594/0 (socket says 172.21.15.27:58594) 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.723+0000 7f4c53fff640 1 -- 172.21.15.27:0/3585055311 learned_addr learned my addr 172.21.15.27:0/3585055311 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.723+0000 7f4c537fe640 1 --2- 172.21.15.27:0/3585055311 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c540c9f50 0x7f4c5415ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.724+0000 7f4c52ffd640 1 -- 172.21.15.27:0/3585055311 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 543973596 0 0) 0x7f4c5415b3e0 con 0x7f4c541553f0 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.724+0000 7f4c52ffd640 1 -- 172.21.15.27:0/3585055311 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c3c003520 con 0x7f4c541553f0 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.724+0000 7f4c52ffd640 1 -- 172.21.15.27:0/3585055311 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2812470224 0 0) 0x7f4c5415b680 con 0x7f4c54071f60 2024-04-20T11:02:25.724 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.724+0000 7f4c52ffd640 1 -- 172.21.15.27:0/3585055311 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c5415b3e0 con 0x7f4c54071f60 2024-04-20T11:02:25.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.724+0000 7f4c52ffd640 1 -- 172.21.15.27:0/3585055311 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1928283338 0 0) 0x7f4c3c003520 con 0x7f4c541553f0 2024-04-20T11:02:25.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.724+0000 7f4c537fe640 1 -- 172.21.15.27:0/3585055311 >> v1:172.21.15.137:6789/0 conn(0x7f4c54071f60 legacy=0x7f4c54165d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.724+0000 7f4c537fe640 1 -- 172.21.15.27:0/3585055311 >> v1:172.21.15.27:6789/0 conn(0x7f4c541553f0 legacy=0x7f4c5406fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.725 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.724+0000 7f4c537fe640 1 -- 172.21.15.27:0/3585055311 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5415b960 con 0x7f4c540c9f50 2024-04-20T11:02:25.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.725+0000 7f4c537fe640 1 --2- 172.21.15.27:0/3585055311 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c540c9f50 0x7f4c5415ee60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c480097d0 tx=0x7f4c4802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31572b2f5e4c8024 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:25.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.725+0000 7f4c52ffd640 1 -- 172.21.15.27:0/3585055311 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44001690 con 0x7f4c540c9f50 2024-04-20T11:02:25.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.725+0000 7f4c52ffd640 1 -- 172.21.15.27:0/3585055311 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c48037070 con 0x7f4c540c9f50 2024-04-20T11:02:25.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.725+0000 7f4c52ffd640 1 -- 172.21.15.27:0/3585055311 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48031780 con 0x7f4c540c9f50 2024-04-20T11:02:25.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.725+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/3585055311 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c540c9f50 msgr2=0x7f4c5415ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.725+0000 7f4c5a2e4640 1 --2- 172.21.15.27:0/3585055311 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c540c9f50 0x7f4c5415ee60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4c480097d0 tx=0x7f4c4802b8a0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.725+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/3585055311 shutdown_connections 2024-04-20T11:02:25.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.725+0000 7f4c5a2e4640 1 --2- 172.21.15.27:0/3585055311 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c540c9f50 0x7f4c5415ee60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.726 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.725+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/3585055311 >> 172.21.15.27:0/3585055311 conn(0x7f4c540ca610 msgr2=0x7f4c540caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:25.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.726+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/3585055311 shutdown_connections 2024-04-20T11:02:25.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.726+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/3585055311 wait complete. 2024-04-20T11:02:25.727 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.727+0000 7f4c5a2e4640 1 Processor -- start 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.727+0000 7f4c5a2e4640 1 -- start start 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.727+0000 7f4c5a2e4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c54071f60 0x7f4c540cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.727+0000 7f4c5a2e4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c541553f0 0x7f4c540cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.727+0000 7f4c5a2e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c540cc8b0 0x7f4c540cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.727+0000 7f4c5a2e4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5415c200 con 0x7f4c540cc8b0 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c5a2e4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5415bf20 con 0x7f4c54071f60 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c5a2e4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f4c541557d0 con 0x7f4c541553f0 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c58881640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c540cc8b0 0x7f4c540cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c58881640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c540cc8b0 0x7f4c540cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38608/0 (socket says 172.21.15.27:38608) 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c58881640 1 -- 172.21.15.27:0/714572523 learned_addr learned my addr 172.21.15.27:0/714572523 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:25.729 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c537fe640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c541553f0 0x7f4c540cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c53fff640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c54071f60 0x7f4c540cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c58881640 1 -- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c541553f0 msgr2=0x7f4c540cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c58881640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c541553f0 0x7f4c540cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c58881640 1 -- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c54071f60 msgr2=0x7f4c540cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c58881640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c54071f60 0x7f4c540cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c58881640 1 -- 172.21.15.27:0/714572523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c54155e20 con 0x7f4c540cc8b0 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c58881640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c540cc8b0 0x7f4c540cdcc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00c7c0 tx=0x7f4c4c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c50ff9640 1 -- 172.21.15.27:0/714572523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c003b70 con 0x7f4c540cc8b0 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c50ff9640 1 -- 172.21.15.27:0/714572523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4c003d10 con 0x7f4c540cc8b0 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c541560c0 con 0x7f4c540cc8b0 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c50ff9640 1 -- 172.21.15.27:0/714572523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00aa50 con 0x7f4c540cc8b0 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.728+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c54157100 con 0x7f4c540cc8b0 2024-04-20T11:02:25.730 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.729+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c540ccd70 con 0x7f4c540cc8b0 2024-04-20T11:02:25.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.730+0000 7f4c50ff9640 1 -- 172.21.15.27:0/714572523 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c4c00abf0 con 0x7f4c540cc8b0 2024-04-20T11:02:25.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.730+0000 7f4c50ff9640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4c4006c920 0x7f4c4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:25.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.730+0000 7f4c50ff9640 1 -- 172.21.15.27:0/714572523 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f4c4c08bb50 con 0x7f4c540cc8b0 2024-04-20T11:02:25.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.730+0000 7f4c53fff640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4c4006c920 0x7f4c4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:25.731 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.731+0000 7f4c53fff640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4c4006c920 0x7f4c4006ede0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c44001200 tx=0x7f4c44001040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:25.734 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.735+0000 7f4c50ff9640 1 -- 172.21.15.27:0/714572523 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4c4c059710 con 0x7f4c540cc8b0 2024-04-20T11:02:25.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.919+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4c5404fcf0 con 0x7f4c540cc8b0 2024-04-20T11:02:25.919 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.919+0000 7f4c50ff9640 1 -- 172.21.15.27:0/714572523 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f4c4c05da10 con 0x7f4c540cc8b0 2024-04-20T11:02:25.919 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2024-04-20T11:02:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4c4006c920 msgr2=0x7f4c4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4c4006c920 0x7f4c4006ede0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c44001200 tx=0x7f4c44001040 comp rx=0 tx=0).stop 2024-04-20T11:02:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c540cc8b0 msgr2=0x7f4c540cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c540cc8b0 0x7f4c540cdcc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00c7c0 tx=0x7f4c4c00cc80 comp rx=0 tx=0).stop 2024-04-20T11:02:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 shutdown_connections 2024-04-20T11:02:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f4c4006c920 0x7f4c4006ede0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c540cc8b0 0x7f4c540cdcc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f4c541553f0 0x7f4c540cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 --2- 172.21.15.27:0/714572523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c54071f60 0x7f4c540cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 >> 172.21.15.27:0/714572523 conn(0x7f4c540ca610 msgr2=0x7f4c540ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 shutdown_connections 2024-04-20T11:02:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:25.921+0000 7f4c5a2e4640 1 -- 172.21.15.27:0/714572523 wait complete. 2024-04-20T11:02:25.934 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T11:02:26.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.039+0000 7fe68c443640 1 Processor -- start 2024-04-20T11:02:26.039 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.039+0000 7fe68c443640 1 -- start start 2024-04-20T11:02:26.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.040+0000 7fe68c443640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe68415ec40 0x7fe6840c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.040+0000 7fe68c443640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68415f210 con 0x7fe6840c62e0 2024-04-20T11:02:26.041 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.040+0000 7fe68c443640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68415f4f0 con 0x7fe684155a20 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.040+0000 7fe68c443640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe684155e00 con 0x7fe68415ec40 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.040+0000 7fe6899de640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe6840c62e0 0x7fe6840cbe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58608/0 (socket says 172.21.15.27:58608) 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.040+0000 7fe6899de640 1 -- 172.21.15.27:0/2563699740 learned_addr learned my addr 172.21.15.27:0/2563699740 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe68a9e0640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe68415ec40 0x7fe6840c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 450788022 0 0) 0x7fe68415f210 con 0x7fe6840c62e0 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe670003520 con 0x7fe6840c62e0 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1466395224 0 0) 0x7fe68415f4f0 con 0x7fe684155a20 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe68415f210 con 0x7fe684155a20 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1752182328 0 0) 0x7fe670003520 con 0x7fe6840c62e0 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe68415ec40 msgr2=0x7fe6840c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe68415ec40 0x7fe6840c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 >> v1:172.21.15.137:6789/0 conn(0x7fe684155a20 legacy=0x7fe68415bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68415f7d0 con 0x7fe6840c62e0 2024-04-20T11:02:26.042 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7fe680001670 con 0x7fe6840c62e0 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 >> v1:172.21.15.27:6789/0 conn(0x7fe6840c62e0 legacy=0x7fe6840cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe68415ec40 0x7fe670003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6700040d0 0x7fe670004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe670004d40 0x7fe6700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe684155e00 con 0x7fe6700040d0 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700142b0 con 0x7fe670004d40 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe6700144b0 con 0x7fe68415ec40 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe6899de640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6700040d0 0x7fe670004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe68a1df640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe68415ec40 0x7fe670003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.041+0000 7fe68a9e0640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe670004d40 0x7fe6700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.042+0000 7fe6899de640 1 -- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe68415ec40 msgr2=0x7fe670003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.042+0000 7fe6899de640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe68415ec40 0x7fe670003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.042+0000 7fe6899de640 1 -- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe670004d40 msgr2=0x7fe6700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.042+0000 7fe6899de640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe670004d40 0x7fe6700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.043 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.042+0000 7fe6899de640 1 -- 172.21.15.27:0/2563699740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe670003520 con 0x7fe6700040d0 2024-04-20T11:02:26.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.042+0000 7fe6899de640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6700040d0 0x7fe670004800 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe680007fd0 tx=0x7fe670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d384c0b4158c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:26.046 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680007ca0 con 0x7fe6700040d0 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe680030070 con 0x7fe6700040d0 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe6891dd640 1 -- 172.21.15.27:0/2563699740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680005d70 con 0x7fe6700040d0 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe68c443640 1 -- 172.21.15.27:0/2563699740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6700040d0 msgr2=0x7fe670004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe68c443640 1 --2- 172.21.15.27:0/2563699740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6700040d0 0x7fe670004800 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe680007fd0 tx=0x7fe670005830 comp rx=0 tx=0).stop 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe68a9e0640 1 -- 172.21.15.27:0/2563699740 reap_dead start 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe68c443640 1 -- 172.21.15.27:0/2563699740 shutdown_connections 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe68c443640 1 -- 172.21.15.27:0/2563699740 >> 172.21.15.27:0/2563699740 conn(0x7fe6840678a0 msgr2=0x7fe684067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe68c443640 1 -- 172.21.15.27:0/2563699740 shutdown_connections 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.043+0000 7fe68c443640 1 -- 172.21.15.27:0/2563699740 wait complete. 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.044+0000 7fe68c443640 1 Processor -- start 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.044+0000 7fe68c443640 1 -- start start 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68c443640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6700040d0 0x7fe6841ed520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68c443640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe684150590 0x7fe6841eda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68c443640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6841494c0 0x7fe684149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.047 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68c443640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe68415c5d0 con 0x7fe6841494c0 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68c443640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe68415ba30 con 0x7fe6700040d0 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68c443640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe6840c67c0 con 0x7fe684150590 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68a9e0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6841494c0 0x7fe684149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68a9e0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6841494c0 0x7fe684149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38634/0 (socket says 172.21.15.27:38634) 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68a9e0640 1 -- 172.21.15.27:0/1594215358 learned_addr learned my addr 172.21.15.27:0/1594215358 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe6899de640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe684150590 0x7fe6841eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.045+0000 7fe68a1df640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6700040d0 0x7fe6841ed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe68a9e0640 1 -- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe684150590 msgr2=0x7fe6841eda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe68a9e0640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe684150590 0x7fe6841eda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe68a9e0640 1 -- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6700040d0 msgr2=0x7fe6841ed520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe68a9e0640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6700040d0 0x7fe6841ed520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe68a9e0640 1 -- 172.21.15.27:0/1594215358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6840d1620 con 0x7fe6841494c0 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe6899de640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe684150590 0x7fe6841eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe68a1df640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6700040d0 0x7fe6841ed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:26.048 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe68a9e0640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6841494c0 0x7fe684149940 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe6840ce890 tx=0x7fe67c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:26.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe67affd640 1 -- 172.21.15.27:0/1594215358 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00fde0 con 0x7fe6841494c0 2024-04-20T11:02:26.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe67affd640 1 -- 172.21.15.27:0/1594215358 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe67c01d070 con 0x7fe6841494c0 2024-04-20T11:02:26.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe67affd640 1 -- 172.21.15.27:0/1594215358 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c015670 con 0x7fe6841494c0 2024-04-20T11:02:26.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68414dc60 con 0x7fe6841494c0 2024-04-20T11:02:26.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.046+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68414df40 con 0x7fe6841494c0 2024-04-20T11:02:26.049 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.047+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68404fcf0 con 0x7fe6841494c0 2024-04-20T11:02:26.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.048+0000 7fe67affd640 1 -- 172.21.15.27:0/1594215358 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe67c015810 con 0x7fe6841494c0 2024-04-20T11:02:26.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.048+0000 7fe67affd640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe66806c8f0 0x7fe66806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.048+0000 7fe67affd640 1 -- 172.21.15.27:0/1594215358 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fe67c08c6b0 con 0x7fe6841494c0 2024-04-20T11:02:26.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.048+0000 7fe68a1df640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe66806c8f0 0x7fe66806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.049+0000 7fe68a1df640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe66806c8f0 0x7fe66806edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6740035f0 tx=0x7fe674009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:26.052 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.052+0000 7fe67affd640 1 -- 172.21.15.27:0/1594215358 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe67c010040 con 0x7fe6841494c0 2024-04-20T11:02:26.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.237+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe684155220 con 0x7fe6841494c0 2024-04-20T11:02:26.237 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.237+0000 7fe67affd640 1 -- 172.21.15.27:0/1594215358 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fe67c05a830 con 0x7fe6841494c0 2024-04-20T11:02:26.237 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2024-04-20T11:02:26.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe66806c8f0 msgr2=0x7fe66806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.239 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe66806c8f0 0x7fe66806edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6740035f0 tx=0x7fe674009040 comp rx=0 tx=0).stop 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6841494c0 msgr2=0x7fe684149940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6841494c0 0x7fe684149940 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe6840ce890 tx=0x7fe67c00f580 comp rx=0 tx=0).stop 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 shutdown_connections 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe66806c8f0 0x7fe66806edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6841494c0 0x7fe684149940 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe684150590 0x7fe6841eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 --2- 172.21.15.27:0/1594215358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6700040d0 0x7fe6841ed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 >> 172.21.15.27:0/1594215358 conn(0x7fe6840678a0 msgr2=0x7fe68414f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 shutdown_connections 2024-04-20T11:02:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.239+0000 7fe68c443640 1 -- 172.21.15.27:0/1594215358 wait complete. 2024-04-20T11:02:26.252 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T11:02:26.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.353+0000 7fc8b56ba640 1 Processor -- start 2024-04-20T11:02:26.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.354+0000 7fc8b56ba640 1 -- start start 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.354+0000 7fc8b56ba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc8b0155760 0x7fc8b006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.354+0000 7fc8b56ba640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b015b670 0x7fc8b015bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.354+0000 7fc8b56ba640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b015c020 con 0x7fc8b00701f0 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.354+0000 7fc8b56ba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b00719a0 con 0x7fc8b0155760 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.354+0000 7fc8b56ba640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b0070d70 con 0x7fc8b015b670 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.354+0000 7fc8aeffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc8b0155760 0x7fc8b006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.354+0000 7fc8ae7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc8b00701f0 0x7fc8b00cf220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58622/0 (socket says 172.21.15.27:58622) 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.354+0000 7fc8ae7fc640 1 -- 172.21.15.27:0/3406246863 learned_addr learned my addr 172.21.15.27:0/3406246863 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.355+0000 7fc8af7fe640 1 --2- 172.21.15.27:0/3406246863 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b015b670 0x7fc8b015bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.355+0000 7fc8aeffd640 1 -- 172.21.15.27:0/3406246863 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b015b670 msgr2=0x7fc8b015bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.355+0000 7fc8aeffd640 1 --2- 172.21.15.27:0/3406246863 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b015b670 0x7fc8b015bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.355+0000 7fc8aeffd640 1 -- 172.21.15.27:0/3406246863 >> v1:172.21.15.27:6789/0 conn(0x7fc8b00701f0 legacy=0x7fc8b00cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.355+0000 7fc8aeffd640 1 -- 172.21.15.27:0/3406246863 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b0070ad0 con 0x7fc8b0155760 2024-04-20T11:02:26.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.355+0000 7fc8adffb640 1 -- 172.21.15.27:0/3406246863 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 464496379 0 0) 0x7fc8b015c020 con 0x7fc8b00701f0 2024-04-20T11:02:26.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.355+0000 7fc8af7fe640 1 --2- 172.21.15.27:0/3406246863 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b015b670 0x7fc8b015bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:02:26.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8aeffd640 1 --2- 172.21.15.27:0/3406246863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b0155760 0x7fc8b006ca40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00097a0 tx=0x7fc8a002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5855a7fdddbaeaa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:26.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8adffb640 1 -- 172.21.15.27:0/3406246863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0035020 con 0x7fc8b0155760 2024-04-20T11:02:26.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8adffb640 1 -- 172.21.15.27:0/3406246863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a0031810 con 0x7fc8b0155760 2024-04-20T11:02:26.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8adffb640 1 -- 172.21.15.27:0/3406246863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0031be0 con 0x7fc8b0155760 2024-04-20T11:02:26.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8b56ba640 1 -- 172.21.15.27:0/3406246863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b0155760 msgr2=0x7fc8b006ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8b56ba640 1 --2- 172.21.15.27:0/3406246863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b0155760 0x7fc8b006ca40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00097a0 tx=0x7fc8a002b8b0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.357 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8b56ba640 1 -- 172.21.15.27:0/3406246863 shutdown_connections 2024-04-20T11:02:26.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8b56ba640 1 --2- 172.21.15.27:0/3406246863 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b015b670 0x7fc8b015bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8b56ba640 1 --2- 172.21.15.27:0/3406246863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b0155760 0x7fc8b006ca40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8b56ba640 1 -- 172.21.15.27:0/3406246863 >> 172.21.15.27:0/3406246863 conn(0x7fc8b0067ae0 msgr2=0x7fc8b0067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:26.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8b56ba640 1 -- 172.21.15.27:0/3406246863 shutdown_connections 2024-04-20T11:02:26.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.356+0000 7fc8b56ba640 1 -- 172.21.15.27:0/3406246863 wait complete. 2024-04-20T11:02:26.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8b56ba640 1 Processor -- start 2024-04-20T11:02:26.358 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8b56ba640 1 -- start start 2024-04-20T11:02:26.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8b56ba640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b0155760 0x7fc8b014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8b56ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b015b670 0x7fc8b00c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8b56ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b014c480 0x7fc8b0148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8b56ba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b006e480 con 0x7fc8b014c480 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8b56ba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b006e7f0 con 0x7fc8b015b670 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8b56ba640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b0070ff0 con 0x7fc8b0155760 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b014c480 0x7fc8b0148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b014c480 0x7fc8b0148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38640/0 (socket says 172.21.15.27:38640) 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.358+0000 7fc8af7fe640 1 -- 172.21.15.27:0/1893491222 learned_addr learned my addr 172.21.15.27:0/1893491222 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8ae7fc640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b015b670 0x7fc8b00c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8aeffd640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b0155760 0x7fc8b014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8af7fe640 1 -- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b0155760 msgr2=0x7fc8b014b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8af7fe640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b0155760 0x7fc8b014b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8af7fe640 1 -- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b015b670 msgr2=0x7fc8b00c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8af7fe640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b015b670 0x7fc8b00c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8af7fe640 1 -- 172.21.15.27:0/1893491222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b00c9e30 con 0x7fc8b014c480 2024-04-20T11:02:26.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8af7fe640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b014c480 0x7fc8b0148800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800cd50 tx=0x7fc8a800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8977fe640 1 -- 172.21.15.27:0/1893491222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800f680 con 0x7fc8b014c480 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8977fe640 1 -- 172.21.15.27:0/1893491222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a800f840 con 0x7fc8b014c480 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b014a1b0 con 0x7fc8b014c480 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8977fe640 1 -- 172.21.15.27:0/1893491222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800fc30 con 0x7fc8b014c480 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.359+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b014aa80 con 0x7fc8b014c480 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.360+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b00688e0 con 0x7fc8b014c480 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.361+0000 7fc8977fe640 1 -- 172.21.15.27:0/1893491222 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8a80213f0 con 0x7fc8b014c480 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.361+0000 7fc8977fe640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc89c06c8f0 0x7fc89c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.361+0000 7fc8977fe640 1 -- 172.21.15.27:0/1893491222 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fc8a808ca90 con 0x7fc8b014c480 2024-04-20T11:02:26.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.361+0000 7fc8aeffd640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc89c06c8f0 0x7fc89c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:26.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.362+0000 7fc8aeffd640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc89c06c8f0 0x7fc89c06edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0036000 tx=0x7fc8a0008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:26.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.365+0000 7fc8977fe640 1 -- 172.21.15.27:0/1893491222 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc8a8014030 con 0x7fc8b014c480 2024-04-20T11:02:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.561+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc8b0154530 con 0x7fc8b014c480 2024-04-20T11:02:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.561+0000 7fc8977fe640 1 -- 172.21.15.27:0/1893491222 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v27) ==== 81+0+612 (secure 0 0 0) 0x7fc8b0154530 con 0x7fc8b014c480 2024-04-20T11:02:26.561 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6836/4195543922 2024-04-21T11:01:52.359070+0000 2024-04-20T11:02:26.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6833/4195543922 2024-04-21T11:01:52.359070+0000 2024-04-20T11:02:26.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6837/1680990583 2024-04-21T11:00:42.348974+0000 2024-04-20T11:02:26.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6834/1680990583 2024-04-21T11:00:42.348974+0000 2024-04-20T11:02:26.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6832/859496751 2024-04-21T10:59:37.339287+0000 2024-04-20T11:02:26.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.137:6835/1164978945 2024-04-21T10:58:22.329607+0000 2024-04-20T11:02:26.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.137:6834/1164978945 2024-04-21T10:58:22.329607+0000 2024-04-20T11:02:26.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27:6835/859496751 2024-04-21T10:59:37.339287+0000 2024-04-20T11:02:26.562 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.137:6838/1990627502 2024-04-21T10:57:27.321673+0000 2024-04-20T11:02:26.563 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.137:6836/1990627502 2024-04-21T10:57:27.321673+0000 2024-04-20T11:02:26.564 INFO:teuthology.orchestra.run.smithi027.stderr:listed 10 entries 2024-04-20T11:02:26.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc89c06c8f0 msgr2=0x7fc89c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc89c06c8f0 0x7fc89c06edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0036000 tx=0x7fc8a0008030 comp rx=0 tx=0).stop 2024-04-20T11:02:26.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b014c480 msgr2=0x7fc8b0148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:26.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b014c480 0x7fc8b0148800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800cd50 tx=0x7fc8a800a650 comp rx=0 tx=0).stop 2024-04-20T11:02:26.564 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 shutdown_connections 2024-04-20T11:02:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc89c06c8f0 0x7fc89c06edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8b014c480 0x7fc8b0148800 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8b015b670 0x7fc8b00c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 --2- 172.21.15.27:0/1893491222 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc8b0155760 0x7fc8b014b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 >> 172.21.15.27:0/1893491222 conn(0x7fc8b0067ae0 msgr2=0x7fc8b0153e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 shutdown_connections 2024-04-20T11:02:26.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.563+0000 7fc8b56ba640 1 -- 172.21.15.27:0/1893491222 wait complete. 2024-04-20T11:02:26.574 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-20T11:02:26.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:02:26.574 DEBUG:teuthology.orchestra.run.smithi027:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-20T11:02:26.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T11:02:26.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:02:26.620 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T11:02:26.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:02:26.658 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2024-04-20T11:02:26.714 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T11:02:26.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:02:26.715 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T11:02:26.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T11:02:26.782 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2024-04-20T11:02:26.881 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-20T11:02:26.894 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T11:02:26.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:02:26.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.998+0000 7f6c47515640 1 Processor -- start 2024-04-20T11:02:26.997 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.998+0000 7f6c47515640 1 -- start start 2024-04-20T11:02:26.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c47515640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c40072240 0x7f6c400726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:26.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c47515640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c4015c730 con 0x7f6c400cd740 2024-04-20T11:02:26.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c47515640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c40072f40 con 0x7f6c400cdb20 2024-04-20T11:02:26.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c47515640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4015c2a0 con 0x7f6c40072240 2024-04-20T11:02:26.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c452b1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6c400cd740 0x7f6c4015cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58636/0 (socket says 172.21.15.27:58636) 2024-04-20T11:02:26.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c452b1640 1 -- 172.21.15.27:0/3666823143 learned_addr learned my addr 172.21.15.27:0/3666823143 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:26.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c45ab2640 1 --2- 172.21.15.27:0/3666823143 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c40072240 0x7f6c400726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c33fff640 1 -- 172.21.15.27:0/3666823143 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1077811 0 0) 0x7f6c4015c730 con 0x7f6c400cd740 2024-04-20T11:02:27.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c33fff640 1 -- 172.21.15.27:0/3666823143 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c2c003520 con 0x7f6c400cd740 2024-04-20T11:02:27.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c33fff640 1 -- 172.21.15.27:0/3666823143 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1648960799 0 0) 0x7f6c40072f40 con 0x7f6c400cdb20 2024-04-20T11:02:27.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c33fff640 1 -- 172.21.15.27:0/3666823143 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c4015c730 con 0x7f6c400cdb20 2024-04-20T11:02:27.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c33fff640 1 -- 172.21.15.27:0/3666823143 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 650950820 0 0) 0x7f6c2c003520 con 0x7f6c400cd740 2024-04-20T11:02:27.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c45ab2640 1 -- 172.21.15.27:0/3666823143 >> v1:172.21.15.137:6789/0 conn(0x7f6c400cdb20 legacy=0x7f6c40155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c45ab2640 1 -- 172.21.15.27:0/3666823143 >> v1:172.21.15.27:6789/0 conn(0x7f6c400cd740 legacy=0x7f6c4015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.000 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:26.999+0000 7f6c45ab2640 1 -- 172.21.15.27:0/3666823143 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4015b970 con 0x7f6c40072240 2024-04-20T11:02:27.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.000+0000 7f6c45ab2640 1 --2- 172.21.15.27:0/3666823143 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c40072240 0x7f6c400726b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009dc0 tx=0x7f6c3802be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ade537c08ff003e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:27.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.000+0000 7f6c33fff640 1 -- 172.21.15.27:0/3666823143 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38039070 con 0x7f6c40072240 2024-04-20T11:02:27.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.000+0000 7f6c33fff640 1 -- 172.21.15.27:0/3666823143 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3802c9a0 con 0x7f6c40072240 2024-04-20T11:02:27.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.000+0000 7f6c33fff640 1 -- 172.21.15.27:0/3666823143 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c001670 con 0x7f6c40072240 2024-04-20T11:02:27.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.001+0000 7f6c47515640 1 -- 172.21.15.27:0/3666823143 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c40072240 msgr2=0x7f6c400726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.001 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.001+0000 7f6c47515640 1 --2- 172.21.15.27:0/3666823143 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c40072240 0x7f6c400726b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009dc0 tx=0x7f6c3802be70 comp rx=0 tx=0).stop 2024-04-20T11:02:27.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.001+0000 7f6c47515640 1 -- 172.21.15.27:0/3666823143 shutdown_connections 2024-04-20T11:02:27.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.001+0000 7f6c47515640 1 --2- 172.21.15.27:0/3666823143 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c40072240 0x7f6c400726b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.001+0000 7f6c47515640 1 -- 172.21.15.27:0/3666823143 >> 172.21.15.27:0/3666823143 conn(0x7f6c4006d5b0 msgr2=0x7f6c4006d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:27.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.001+0000 7f6c47515640 1 -- 172.21.15.27:0/3666823143 shutdown_connections 2024-04-20T11:02:27.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.001+0000 7f6c47515640 1 -- 172.21.15.27:0/3666823143 wait complete. 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.002+0000 7f6c47515640 1 Processor -- start 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.002+0000 7f6c47515640 1 -- start start 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c47515640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c40072240 0x7f6c400c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c47515640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c400cd740 0x7f6c4015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c47515640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c400cdb20 0x7f6c401f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c47515640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4015d360 con 0x7f6c40072240 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c47515640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4015bf30 con 0x7f6c400cd740 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c47515640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6c40072bf0 con 0x7f6c400cdb20 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c452b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c40072240 0x7f6c400c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c452b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c40072240 0x7f6c400c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38646/0 (socket says 172.21.15.27:38646) 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c452b1640 1 -- 172.21.15.27:0/2382594767 learned_addr learned my addr 172.21.15.27:0/2382594767 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c45ab2640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c400cdb20 0x7f6c401f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c44ab0640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c400cd740 0x7f6c4015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c452b1640 1 -- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c400cdb20 msgr2=0x7f6c401f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c452b1640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c400cdb20 0x7f6c401f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c452b1640 1 -- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c400cd740 msgr2=0x7f6c4015a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c452b1640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c400cd740 0x7f6c4015a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c452b1640 1 -- 172.21.15.27:0/2382594767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c40150d90 con 0x7f6c40072240 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.003+0000 7f6c452b1640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c40072240 0x7f6c400c51b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c780 tx=0x7f6c3c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.004+0000 7f6c31ffb640 1 -- 172.21.15.27:0/2382594767 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c005ad0 con 0x7f6c40072240 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.004+0000 7f6c31ffb640 1 -- 172.21.15.27:0/2382594767 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3c005c70 con 0x7f6c40072240 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.004+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c401f0e90 con 0x7f6c40072240 2024-04-20T11:02:27.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.004+0000 7f6c31ffb640 1 -- 172.21.15.27:0/2382594767 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0072b0 con 0x7f6c40072240 2024-04-20T11:02:27.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.004+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c401e8810 con 0x7f6c40072240 2024-04-20T11:02:27.006 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.005+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4004fcf0 con 0x7f6c40072240 2024-04-20T11:02:27.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.005+0000 7f6c31ffb640 1 -- 172.21.15.27:0/2382594767 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c3c007450 con 0x7f6c40072240 2024-04-20T11:02:27.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.006+0000 7f6c31ffb640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6c2406c8f0 0x7f6c2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.006+0000 7f6c31ffb640 1 -- 172.21.15.27:0/2382594767 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f6c3c08bb90 con 0x7f6c40072240 2024-04-20T11:02:27.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.006+0000 7f6c44ab0640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6c2406c8f0 0x7f6c2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.007 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.007+0000 7f6c44ab0640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6c2406c8f0 0x7f6c2406edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c40150fb0 tx=0x7f6c34009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:27.009 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.010+0000 7f6c31ffb640 1 -- 172.21.15.27:0/2382594767 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c3c005e20 con 0x7f6c40072240 2024-04-20T11:02:27.190 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.191+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c40155090 con 0x7f6c2406c8f0 2024-04-20T11:02:27.197 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.198+0000 7f6c31ffb640 1 -- 172.21.15.27:0/2382594767 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253823 (secure 0 0 0) 0x7f6c40155090 con 0x7f6c2406c8f0 2024-04-20T11:02:27.200 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:27.202 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6c2406c8f0 msgr2=0x7f6c2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6c2406c8f0 0x7f6c2406edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c40150fb0 tx=0x7f6c34009040 comp rx=0 tx=0).stop 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c40072240 msgr2=0x7f6c400c51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c40072240 0x7f6c400c51b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c780 tx=0x7f6c3c00cc40 comp rx=0 tx=0).stop 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 shutdown_connections 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f6c2406c8f0 0x7f6c2406edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c400cdb20 0x7f6c401f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c400cd740 0x7f6c4015a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 --2- 172.21.15.27:0/2382594767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c40072240 0x7f6c400c51b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 >> 172.21.15.27:0/2382594767 conn(0x7f6c4006d5b0 msgr2=0x7f6c401553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 shutdown_connections 2024-04-20T11:02:27.204 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.201+0000 7f6c47515640 1 -- 172.21.15.27:0/2382594767 wait complete. 2024-04-20T11:02:27.212 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-04-20T11:02:25.792397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":152723202,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":1845,"num_read_kb":3292270,"num_write":30944,"num_write_kb":4353505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34478,"ondisk_log_size":34478,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":659848,"kb_used_data":300808,"kb_used_omap":12,"kb_used_meta":358963,"kb_avail":754314872,"statfs":{"total":773094113280,"available":772418428928,"internally_reserved":0,"allocated":308027392,"data_stored":306366156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":367578673},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3210049,"num_objects":-403,"num_object_clones":0,"num_object_copies":-806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-403,"num_whiteouts":0,"num_read":74,"num_read_kb":117,"num_write":480,"num_write_kb":1281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002609"},"pg_stats":[{"pgid":"3.3f","version":"27'417","reported_seq":460,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.434387+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T11:02:12.434387+0000","last_peered":"2024-04-20T11:02:12.434387+0000","last_clean":"2024-04-20T11:02:12.434387+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T11:02:12.434387+0000","last_undegraded":"2024-04-20T11:02:12.434387+0000","last_fullsized":"2024-04-20T11:02:12.434387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":31614,"num_write":375,"num_write_kb":59119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"27'425","reported_seq":470,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.442935+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T11:02:16.442935+0000","last_peered":"2024-04-20T11:02:16.442935+0000","last_clean":"2024-04-20T11:02:16.442935+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T11:02:16.442935+0000","last_undegraded":"2024-04-20T11:02:16.442935+0000","last_fullsized":"2024-04-20T11:02:16.442935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":40656,"num_write":384,"num_write_kb":54681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"27'399","reported_seq":447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.459090+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T11:02:12.459090+0000","last_peered":"2024-04-20T11:02:12.459090+0000","last_clean":"2024-04-20T11:02:12.459090+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T11:02:12.459090+0000","last_undegraded":"2024-04-20T11:02:12.459090+0000","last_fullsized":"2024-04-20T11:02:12.459090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":42823,"num_write":366,"num_write_kb":47539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'384","reported_seq":426,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.471314+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T11:02:16.471314+0000","last_peered":"2024-04-20T11:02:16.471314+0000","last_clean":"2024-04-20T11:02:16.471314+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T11:02:16.471314+0000","last_undegraded":"2024-04-20T11:02:16.471314+0000","last_fullsized":"2024-04-20T11:02:16.471314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":22108,"num_write":353,"num_write_kb":40048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'426","reported_seq":472,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.445404+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T11:02:16.445404+0000","last_peered":"2024-04-20T11:02:16.445404+0000","last_clean":"2024-04-20T11:02:16.445404+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T11:02:16.445404+0000","last_undegraded":"2024-04-20T11:02:16.445404+0000","last_fullsized":"2024-04-20T11:02:16.445404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":63832,"num_write":387,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'365","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.453897+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T11:02:11.453897+0000","last_peered":"2024-04-20T11:02:11.453897+0000","last_clean":"2024-04-20T11:02:11.453897+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T11:02:11.453897+0000","last_undegraded":"2024-04-20T11:02:11.453897+0000","last_fullsized":"2024-04-20T11:02:11.453897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":22263,"num_write":332,"num_write_kb":22043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'431","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.452173+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T11:02:12.452173+0000","last_peered":"2024-04-20T11:02:12.452173+0000","last_clean":"2024-04-20T11:02:12.452173+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T11:02:12.452173+0000","last_undegraded":"2024-04-20T11:02:12.452173+0000","last_fullsized":"2024-04-20T11:02:12.452173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":74379,"num_write":399,"num_write_kb":92171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'360","reported_seq":400,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.478262+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T11:02:16.478262+0000","last_peered":"2024-04-20T11:02:16.478262+0000","last_clean":"2024-04-20T11:02:16.478262+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T11:02:16.478262+0000","last_undegraded":"2024-04-20T11:02:16.478262+0000","last_fullsized":"2024-04-20T11:02:16.478262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":32797,"num_write":314,"num_write_kb":38008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"27'422","reported_seq":468,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.442111+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T11:02:16.442111+0000","last_peered":"2024-04-20T11:02:16.442111+0000","last_clean":"2024-04-20T11:02:16.442111+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T11:02:16.442111+0000","last_undegraded":"2024-04-20T11:02:16.442111+0000","last_fullsized":"2024-04-20T11:02:16.442111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":52787,"num_write":396,"num_write_kb":59707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"27'409","reported_seq":465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.456181+0000","last_change":"2024-04-20T10:56:22.913401+0000","last_active":"2024-04-20T11:02:16.456181+0000","last_peered":"2024-04-20T11:02:16.456181+0000","last_clean":"2024-04-20T11:02:16.456181+0000","last_became_active":"2024-04-20T10:56:22.913113+0000","last_became_peered":"2024-04-20T10:56:22.913113+0000","last_unstale":"2024-04-20T11:02:16.456181+0000","last_undegraded":"2024-04-20T11:02:16.456181+0000","last_fullsized":"2024-04-20T11:02:16.456181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:17.248882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":77299,"num_write":373,"num_write_kb":63557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"27'381","reported_seq":425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.456654+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T11:02:16.456654+0000","last_peered":"2024-04-20T11:02:16.456654+0000","last_clean":"2024-04-20T11:02:16.456654+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T11:02:16.456654+0000","last_undegraded":"2024-04-20T11:02:16.456654+0000","last_fullsized":"2024-04-20T11:02:16.456654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":50462,"num_write":358,"num_write_kb":46678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'387","reported_seq":434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.452830+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T11:02:16.452830+0000","last_peered":"2024-04-20T11:02:16.452830+0000","last_clean":"2024-04-20T11:02:16.452830+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T11:02:16.452830+0000","last_undegraded":"2024-04-20T11:02:16.452830+0000","last_fullsized":"2024-04-20T11:02:16.452830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":64677,"num_write":357,"num_write_kb":71584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"27'405","reported_seq":454,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.452369+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T11:02:12.452369+0000","last_peered":"2024-04-20T11:02:12.452369+0000","last_clean":"2024-04-20T11:02:12.452369+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T11:02:12.452369+0000","last_undegraded":"2024-04-20T11:02:12.452369+0000","last_fullsized":"2024-04-20T11:02:12.452369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":59589,"num_write":367,"num_write_kb":71789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"27'414","reported_seq":451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.450507+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T11:02:16.450507+0000","last_peered":"2024-04-20T11:02:16.450507+0000","last_clean":"2024-04-20T11:02:16.450507+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T11:02:16.450507+0000","last_undegraded":"2024-04-20T11:02:16.450507+0000","last_fullsized":"2024-04-20T11:02:16.450507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":30435,"num_write":362,"num_write_kb":42694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"27'412","reported_seq":453,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.441267+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T11:02:11.441267+0000","last_peered":"2024-04-20T11:02:11.441267+0000","last_clean":"2024-04-20T11:02:11.441267+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T11:02:11.441267+0000","last_undegraded":"2024-04-20T11:02:11.441267+0000","last_fullsized":"2024-04-20T11:02:11.441267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":33945,"num_write":377,"num_write_kb":75986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"27'442","reported_seq":495,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.475976+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T11:02:16.475976+0000","last_peered":"2024-04-20T11:02:16.475976+0000","last_clean":"2024-04-20T11:02:16.475976+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T11:02:16.475976+0000","last_undegraded":"2024-04-20T11:02:16.475976+0000","last_fullsized":"2024-04-20T11:02:16.475976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":54533,"num_write":414,"num_write_kb":39250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"27'419","reported_seq":469,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.472619+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T11:02:16.472619+0000","last_peered":"2024-04-20T11:02:16.472619+0000","last_clean":"2024-04-20T11:02:16.472619+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T11:02:16.472619+0000","last_undegraded":"2024-04-20T11:02:16.472619+0000","last_fullsized":"2024-04-20T11:02:16.472619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":68755,"num_write":396,"num_write_kb":79570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"27'419","reported_seq":466,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.483508+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T11:02:16.483508+0000","last_peered":"2024-04-20T11:02:16.483508+0000","last_clean":"2024-04-20T11:02:16.483508+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T11:02:16.483508+0000","last_undegraded":"2024-04-20T11:02:16.483508+0000","last_fullsized":"2024-04-20T11:02:16.483508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53186,"num_write":383,"num_write_kb":88017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"27'432","reported_seq":484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.453682+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T11:02:16.453682+0000","last_peered":"2024-04-20T11:02:16.453682+0000","last_clean":"2024-04-20T11:02:16.453682+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T11:02:16.453682+0000","last_undegraded":"2024-04-20T11:02:16.453682+0000","last_fullsized":"2024-04-20T11:02:16.453682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":86853,"num_write":391,"num_write_kb":112571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"27'447","reported_seq":488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.489722+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T11:02:11.489722+0000","last_peered":"2024-04-20T11:02:11.489722+0000","last_clean":"2024-04-20T11:02:11.489722+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T11:02:11.489722+0000","last_undegraded":"2024-04-20T11:02:11.489722+0000","last_fullsized":"2024-04-20T11:02:11.489722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36592,"num_write":417,"num_write_kb":59154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"27'412","reported_seq":461,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.451071+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T11:02:16.451071+0000","last_peered":"2024-04-20T11:02:16.451071+0000","last_clean":"2024-04-20T11:02:16.451071+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T11:02:16.451071+0000","last_undegraded":"2024-04-20T11:02:16.451071+0000","last_fullsized":"2024-04-20T11:02:16.451071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":55674,"num_write":374,"num_write_kb":59056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"27'435","reported_seq":480,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.431501+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T11:02:12.431501+0000","last_peered":"2024-04-20T11:02:12.431501+0000","last_clean":"2024-04-20T11:02:12.431501+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T11:02:12.431501+0000","last_undegraded":"2024-04-20T11:02:12.431501+0000","last_fullsized":"2024-04-20T11:02:12.431501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":66341,"num_write":402,"num_write_kb":96389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'401","reported_seq":437,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.447612+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T11:02:16.447612+0000","last_peered":"2024-04-20T11:02:16.447612+0000","last_clean":"2024-04-20T11:02:16.447612+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T11:02:16.447612+0000","last_undegraded":"2024-04-20T11:02:16.447612+0000","last_fullsized":"2024-04-20T11:02:16.447612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":25926,"num_write":360,"num_write_kb":50488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'466","reported_seq":519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.484754+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T11:02:16.484754+0000","last_peered":"2024-04-20T11:02:16.484754+0000","last_clean":"2024-04-20T11:02:16.484754+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T11:02:16.484754+0000","last_undegraded":"2024-04-20T11:02:16.484754+0000","last_fullsized":"2024-04-20T11:02:16.484754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":61071,"num_write":433,"num_write_kb":92498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"27'403","reported_seq":438,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.462861+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T11:02:16.462861+0000","last_peered":"2024-04-20T11:02:16.462861+0000","last_clean":"2024-04-20T11:02:16.462861+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T11:02:16.462861+0000","last_undegraded":"2024-04-20T11:02:16.462861+0000","last_fullsized":"2024-04-20T11:02:16.462861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":21468,"num_write":380,"num_write_kb":33760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"27'445","reported_seq":477,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.458342+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T11:02:16.458342+0000","last_peered":"2024-04-20T11:02:16.458342+0000","last_clean":"2024-04-20T11:02:16.458342+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T11:02:16.458342+0000","last_undegraded":"2024-04-20T11:02:16.458342+0000","last_fullsized":"2024-04-20T11:02:16.458342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8561,"num_write":404,"num_write_kb":47121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'426","reported_seq":481,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.768817+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T11:02:14.768817+0000","last_peered":"2024-04-20T11:02:14.768817+0000","last_clean":"2024-04-20T11:02:14.768817+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T11:02:14.768817+0000","last_undegraded":"2024-04-20T11:02:14.768817+0000","last_fullsized":"2024-04-20T11:02:14.768817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":82311,"num_write":397,"num_write_kb":100756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"27'402","reported_seq":451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.486267+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T11:02:16.486267+0000","last_peered":"2024-04-20T11:02:16.486267+0000","last_clean":"2024-04-20T11:02:16.486267+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T11:02:16.486267+0000","last_undegraded":"2024-04-20T11:02:16.486267+0000","last_fullsized":"2024-04-20T11:02:16.486267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":40362,"num_write":368,"num_write_kb":83934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'430","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.477185+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T11:02:11.477185+0000","last_peered":"2024-04-20T11:02:11.477185+0000","last_clean":"2024-04-20T11:02:11.477185+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T11:02:11.477185+0000","last_undegraded":"2024-04-20T11:02:11.477185+0000","last_fullsized":"2024-04-20T11:02:11.477185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":51907,"num_write":395,"num_write_kb":76367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"27'471","reported_seq":523,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.452623+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T11:02:16.452623+0000","last_peered":"2024-04-20T11:02:16.452623+0000","last_clean":"2024-04-20T11:02:16.452623+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T11:02:16.452623+0000","last_undegraded":"2024-04-20T11:02:16.452623+0000","last_fullsized":"2024-04-20T11:02:16.452623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":45044,"num_write":427,"num_write_kb":43197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"27'413","reported_seq":450,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.458689+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T11:02:16.458689+0000","last_peered":"2024-04-20T11:02:16.458689+0000","last_clean":"2024-04-20T11:02:16.458689+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T11:02:16.458689+0000","last_undegraded":"2024-04-20T11:02:16.458689+0000","last_fullsized":"2024-04-20T11:02:16.458689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":30452,"num_write":380,"num_write_kb":32013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"27'445","reported_seq":481,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.447290+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T11:02:16.447290+0000","last_peered":"2024-04-20T11:02:16.447290+0000","last_clean":"2024-04-20T11:02:16.447290+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T11:02:16.447290+0000","last_undegraded":"2024-04-20T11:02:16.447290+0000","last_fullsized":"2024-04-20T11:02:16.447290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":23674,"num_write":409,"num_write_kb":56716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"27'390","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.453596+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T11:02:16.453596+0000","last_peered":"2024-04-20T11:02:16.453596+0000","last_clean":"2024-04-20T11:02:16.453596+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T11:02:16.453596+0000","last_undegraded":"2024-04-20T11:02:16.453596+0000","last_fullsized":"2024-04-20T11:02:16.453596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36784,"num_write":356,"num_write_kb":38078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'160","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.426958+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T11:02:07.426958+0000","last_peered":"2024-04-20T11:02:07.426958+0000","last_clean":"2024-04-20T11:02:07.426958+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T11:02:07.426958+0000","last_undegraded":"2024-04-20T11:02:07.426958+0000","last_fullsized":"2024-04-20T11:02:07.426958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195298,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":111,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'395","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.447080+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T11:02:16.447080+0000","last_peered":"2024-04-20T11:02:16.447080+0000","last_clean":"2024-04-20T11:02:16.447080+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T11:02:16.447080+0000","last_undegraded":"2024-04-20T11:02:16.447080+0000","last_fullsized":"2024-04-20T11:02:16.447080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":26712,"num_write":367,"num_write_kb":67448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"27'322","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.494755+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T11:02:11.494755+0000","last_peered":"2024-04-20T11:02:11.494755+0000","last_clean":"2024-04-20T11:02:11.494755+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T11:02:11.494755+0000","last_undegraded":"2024-04-20T11:02:11.494755+0000","last_fullsized":"2024-04-20T11:02:11.494755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":260,"num_write_kb":12413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"27'95","reported_seq":129,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:05.455443+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T11:02:05.455443+0000","last_peered":"2024-04-20T11:02:05.455443+0000","last_clean":"2024-04-20T11:02:05.455443+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T11:02:05.455443+0000","last_undegraded":"2024-04-20T11:02:05.455443+0000","last_fullsized":"2024-04-20T11:02:05.455443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":20,"num_write":51,"num_write_kb":4136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'441","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.428113+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T11:02:12.428113+0000","last_peered":"2024-04-20T11:02:12.428113+0000","last_clean":"2024-04-20T11:02:12.428113+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T11:02:12.428113+0000","last_undegraded":"2024-04-20T11:02:12.428113+0000","last_fullsized":"2024-04-20T11:02:12.428113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":16783,"num_write":403,"num_write_kb":42765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"27'797","reported_seq":828,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.485457+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T11:02:16.485457+0000","last_peered":"2024-04-20T11:02:16.485457+0000","last_clean":"2024-04-20T11:02:16.485457+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T11:02:16.485457+0000","last_undegraded":"2024-04-20T11:02:16.485457+0000","last_fullsized":"2024-04-20T11:02:16.485457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":724,"num_write_kb":8631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"27'447","reported_seq":510,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.473487+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T11:02:16.473487+0000","last_peered":"2024-04-20T11:02:16.473487+0000","last_clean":"2024-04-20T11:02:16.473487+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T11:02:16.473487+0000","last_undegraded":"2024-04-20T11:02:16.473487+0000","last_fullsized":"2024-04-20T11:02:16.473487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":131398,"num_write":411,"num_write_kb":92477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"27'73","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.943331+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T11:02:17.943331+0000","last_peered":"2024-04-20T11:02:17.943331+0000","last_clean":"2024-04-20T11:02:17.943331+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T11:02:17.943331+0000","last_undegraded":"2024-04-20T11:02:17.943331+0000","last_fullsized":"2024-04-20T11:02:17.943331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":22,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"27'445","reported_seq":498,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.441375+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T11:02:12.441375+0000","last_peered":"2024-04-20T11:02:12.441375+0000","last_clean":"2024-04-20T11:02:12.441375+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T11:02:12.441375+0000","last_undegraded":"2024-04-20T11:02:12.441375+0000","last_fullsized":"2024-04-20T11:02:12.441375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":75132,"num_write":413,"num_write_kb":100522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'95","reported_seq":131,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.944720+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T11:02:17.944720+0000","last_peered":"2024-04-20T11:02:17.944720+0000","last_clean":"2024-04-20T11:02:17.944720+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T11:02:17.944720+0000","last_undegraded":"2024-04-20T11:02:17.944720+0000","last_fullsized":"2024-04-20T11:02:17.944720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":51,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"27'441","reported_seq":523,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.445313+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T11:02:16.445313+0000","last_peered":"2024-04-20T11:02:16.445313+0000","last_clean":"2024-04-20T11:02:16.445313+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T11:02:16.445313+0000","last_undegraded":"2024-04-20T11:02:16.445313+0000","last_fullsized":"2024-04-20T11:02:16.445313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":176347,"num_write":418,"num_write_kb":171699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"27'268","reported_seq":300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.162494+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T11:02:17.162494+0000","last_peered":"2024-04-20T11:02:17.162494+0000","last_clean":"2024-04-20T11:02:17.162494+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T11:02:17.162494+0000","last_undegraded":"2024-04-20T11:02:17.162494+0000","last_fullsized":"2024-04-20T11:02:17.162494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":225,"num_write_kb":4349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'397","reported_seq":444,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.438660+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T11:02:16.438660+0000","last_peered":"2024-04-20T11:02:16.438660+0000","last_clean":"2024-04-20T11:02:16.438660+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T11:02:16.438660+0000","last_undegraded":"2024-04-20T11:02:16.438660+0000","last_fullsized":"2024-04-20T11:02:16.438660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53530,"num_write":359,"num_write_kb":71690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"27'395","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.945316+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T11:02:17.945316+0000","last_peered":"2024-04-20T11:02:17.945316+0000","last_clean":"2024-04-20T11:02:17.945316+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T11:02:17.945316+0000","last_undegraded":"2024-04-20T11:02:17.945316+0000","last_fullsized":"2024-04-20T11:02:17.945316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":364,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"27'423","reported_seq":474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.471695+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T11:02:16.471695+0000","last_peered":"2024-04-20T11:02:16.471695+0000","last_clean":"2024-04-20T11:02:16.471695+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T11:02:16.471695+0000","last_undegraded":"2024-04-20T11:02:16.471695+0000","last_fullsized":"2024-04-20T11:02:16.471695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":72220,"num_write":383,"num_write_kb":72034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"27'357","reported_seq":396,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.946815+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T11:02:17.946815+0000","last_peered":"2024-04-20T11:02:17.946815+0000","last_clean":"2024-04-20T11:02:17.946815+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T11:02:17.946815+0000","last_undegraded":"2024-04-20T11:02:17.946815+0000","last_fullsized":"2024-04-20T11:02:17.946815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":299,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"27'397","reported_seq":434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.483200+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T11:02:16.483200+0000","last_peered":"2024-04-20T11:02:16.483200+0000","last_clean":"2024-04-20T11:02:16.483200+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T11:02:16.483200+0000","last_undegraded":"2024-04-20T11:02:16.483200+0000","last_fullsized":"2024-04-20T11:02:16.483200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":32812,"num_write":361,"num_write_kb":30179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"27'73","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.943894+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T11:02:17.943894+0000","last_peered":"2024-04-20T11:02:17.943894+0000","last_clean":"2024-04-20T11:02:17.943894+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T11:02:17.943894+0000","last_undegraded":"2024-04-20T11:02:17.943894+0000","last_fullsized":"2024-04-20T11:02:17.943894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":17,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'403","reported_seq":446,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.441404+0000","last_change":"2024-04-20T10:56:22.904413+0000","last_active":"2024-04-20T11:02:16.441404+0000","last_peered":"2024-04-20T11:02:16.441404+0000","last_clean":"2024-04-20T11:02:16.441404+0000","last_became_active":"2024-04-20T10:56:22.904118+0000","last_became_peered":"2024-04-20T10:56:22.904118+0000","last_unstale":"2024-04-20T11:02:16.441404+0000","last_undegraded":"2024-04-20T11:02:16.441404+0000","last_fullsized":"2024-04-20T11:02:16.441404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:06.204840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":63538,"num_write":372,"num_write_kb":113421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"27'78","reported_seq":113,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:01.406304+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T11:02:01.406304+0000","last_peered":"2024-04-20T11:02:01.406304+0000","last_clean":"2024-04-20T11:02:01.406304+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T11:02:01.406304+0000","last_undegraded":"2024-04-20T11:02:01.406304+0000","last_fullsized":"2024-04-20T11:02:01.406304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":10,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"27'438","reported_seq":486,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.451206+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T11:02:16.451206+0000","last_peered":"2024-04-20T11:02:16.451206+0000","last_clean":"2024-04-20T11:02:16.451206+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T11:02:16.451206+0000","last_undegraded":"2024-04-20T11:02:16.451206+0000","last_fullsized":"2024-04-20T11:02:16.451206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":68877,"num_write":411,"num_write_kb":59157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"27'361","reported_seq":397,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.948421+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T11:02:17.948421+0000","last_peered":"2024-04-20T11:02:17.948421+0000","last_clean":"2024-04-20T11:02:17.948421+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T11:02:17.948421+0000","last_undegraded":"2024-04-20T11:02:17.948421+0000","last_fullsized":"2024-04-20T11:02:17.948421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":307,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"27'471","reported_seq":525,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.473146+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T11:02:16.473146+0000","last_peered":"2024-04-20T11:02:16.473146+0000","last_clean":"2024-04-20T11:02:16.473146+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T11:02:16.473146+0000","last_undegraded":"2024-04-20T11:02:16.473146+0000","last_fullsized":"2024-04-20T11:02:16.473146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":59233,"num_write":434,"num_write_kb":71822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'75","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.471665+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T11:02:11.471665+0000","last_peered":"2024-04-20T11:02:11.471665+0000","last_clean":"2024-04-20T11:02:11.471665+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T11:02:11.471665+0000","last_undegraded":"2024-04-20T11:02:11.471665+0000","last_fullsized":"2024-04-20T11:02:11.471665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"27'408","reported_seq":469,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.444200+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T11:02:16.444200+0000","last_peered":"2024-04-20T11:02:16.444200+0000","last_clean":"2024-04-20T11:02:16.444200+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T11:02:16.444200+0000","last_undegraded":"2024-04-20T11:02:16.444200+0000","last_fullsized":"2024-04-20T11:02:16.444200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":108675,"num_write":383,"num_write_kb":121193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'262","reported_seq":296,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:02.418420+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T11:02:02.418420+0000","last_peered":"2024-04-20T11:02:02.418420+0000","last_clean":"2024-04-20T11:02:02.418420+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T11:02:02.418420+0000","last_undegraded":"2024-04-20T11:02:02.418420+0000","last_fullsized":"2024-04-20T11:02:02.418420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":218,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"27'384","reported_seq":425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.464192+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T11:02:16.464192+0000","last_peered":"2024-04-20T11:02:16.464192+0000","last_clean":"2024-04-20T11:02:16.464192+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T11:02:16.464192+0000","last_undegraded":"2024-04-20T11:02:16.464192+0000","last_fullsized":"2024-04-20T11:02:16.464192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":38645,"num_write":351,"num_write_kb":38372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"27'464","reported_seq":494,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.419741+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T11:02:07.419741+0000","last_peered":"2024-04-20T11:02:07.419741+0000","last_clean":"2024-04-20T11:02:07.419741+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T11:02:07.419741+0000","last_undegraded":"2024-04-20T11:02:07.419741+0000","last_fullsized":"2024-04-20T11:02:07.419741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"27'460","reported_seq":500,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.477443+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T11:02:16.477443+0000","last_peered":"2024-04-20T11:02:16.477443+0000","last_clean":"2024-04-20T11:02:16.477443+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T11:02:16.477443+0000","last_undegraded":"2024-04-20T11:02:16.477443+0000","last_fullsized":"2024-04-20T11:02:16.477443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":42813,"num_write":427,"num_write_kb":83769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"27'70","reported_seq":105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:05.439468+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T11:02:05.439468+0000","last_peered":"2024-04-20T11:02:05.439468+0000","last_clean":"2024-04-20T11:02:05.439468+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T11:02:05.439468+0000","last_undegraded":"2024-04-20T11:02:05.439468+0000","last_fullsized":"2024-04-20T11:02:05.439468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":8,"num_write":11,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"27'397","reported_seq":428,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.454049+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T11:02:11.454049+0000","last_peered":"2024-04-20T11:02:11.454049+0000","last_clean":"2024-04-20T11:02:11.454049+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T11:02:11.454049+0000","last_undegraded":"2024-04-20T11:02:11.454049+0000","last_fullsized":"2024-04-20T11:02:11.454049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6180,"num_write":356,"num_write_kb":9278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:01:52.405499+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T11:01:52.405499+0000","last_peered":"2024-04-20T11:01:52.405499+0000","last_clean":"2024-04-20T11:01:52.405499+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T11:01:52.405499+0000","last_undegraded":"2024-04-20T11:01:52.405499+0000","last_fullsized":"2024-04-20T11:01:52.405499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"27'77","reported_seq":115,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.490479+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T11:02:16.490479+0000","last_peered":"2024-04-20T11:02:16.490479+0000","last_clean":"2024-04-20T11:02:16.490479+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T11:02:16.490479+0000","last_undegraded":"2024-04-20T11:02:16.490479+0000","last_fullsized":"2024-04-20T11:02:16.490479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":18,"num_write":19,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'352","reported_seq":387,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.461407+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T11:02:12.461407+0000","last_peered":"2024-04-20T11:02:12.461407+0000","last_clean":"2024-04-20T11:02:12.461407+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T11:02:12.461407+0000","last_undegraded":"2024-04-20T11:02:12.461407+0000","last_fullsized":"2024-04-20T11:02:12.461407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":17282,"num_write":327,"num_write_kb":42097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"27'122","reported_seq":155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:02.416875+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T11:02:02.416875+0000","last_peered":"2024-04-20T11:02:02.416875+0000","last_clean":"2024-04-20T11:02:02.416875+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T11:02:02.416875+0000","last_undegraded":"2024-04-20T11:02:02.416875+0000","last_fullsized":"2024-04-20T11:02:02.416875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":64,"num_write_kb":8241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"27'380","reported_seq":421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.453605+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T11:02:12.453605+0000","last_peered":"2024-04-20T11:02:12.453605+0000","last_clean":"2024-04-20T11:02:12.453605+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T11:02:12.453605+0000","last_undegraded":"2024-04-20T11:02:12.453605+0000","last_fullsized":"2024-04-20T11:02:12.453605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":18713,"num_write":347,"num_write_kb":51198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"27'326","reported_seq":611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:18.437066+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T11:02:18.437066+0000","last_peered":"2024-04-20T11:02:18.437066+0000","last_clean":"2024-04-20T11:02:18.437066+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T11:02:18.437066+0000","last_undegraded":"2024-04-20T11:02:18.437066+0000","last_fullsized":"2024-04-20T11:02:18.437066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462167,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":258,"num_read_kb":1597,"num_write":262,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"27'410","reported_seq":457,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.447219+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T11:02:11.447219+0000","last_peered":"2024-04-20T11:02:11.447219+0000","last_clean":"2024-04-20T11:02:11.447219+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T11:02:11.447219+0000","last_undegraded":"2024-04-20T11:02:11.447219+0000","last_fullsized":"2024-04-20T11:02:11.447219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":47086,"num_write":383,"num_write_kb":51294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"27'799","reported_seq":838,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:01:59.759462+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T11:01:59.759462+0000","last_peered":"2024-04-20T11:01:59.759462+0000","last_clean":"2024-04-20T11:01:59.759462+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T11:01:59.759462+0000","last_undegraded":"2024-04-20T11:01:59.759462+0000","last_fullsized":"2024-04-20T11:01:59.759462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":731,"num_write_kb":8599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"27'421","reported_seq":471,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.454524+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T11:02:16.454524+0000","last_peered":"2024-04-20T11:02:16.454524+0000","last_clean":"2024-04-20T11:02:16.454524+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T11:02:16.454524+0000","last_undegraded":"2024-04-20T11:02:16.454524+0000","last_fullsized":"2024-04-20T11:02:16.454524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":65250,"num_write":382,"num_write_kb":83821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"27'75","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.455107+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T11:02:16.455107+0000","last_peered":"2024-04-20T11:02:16.455107+0000","last_clean":"2024-04-20T11:02:16.455107+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T11:02:16.455107+0000","last_undegraded":"2024-04-20T11:02:16.455107+0000","last_fullsized":"2024-04-20T11:02:16.455107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"27'421","reported_seq":454,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.469611+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T11:02:16.469611+0000","last_peered":"2024-04-20T11:02:16.469611+0000","last_clean":"2024-04-20T11:02:16.469611+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T11:02:16.469611+0000","last_undegraded":"2024-04-20T11:02:16.469611+0000","last_fullsized":"2024-04-20T11:02:16.469611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":375,"num_write_kb":62820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"27'85","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.454748+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T11:02:16.454748+0000","last_peered":"2024-04-20T11:02:16.454748+0000","last_clean":"2024-04-20T11:02:16.454748+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T11:02:16.454748+0000","last_undegraded":"2024-04-20T11:02:16.454748+0000","last_fullsized":"2024-04-20T11:02:16.454748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":29,"num_write_kb":4110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"27'382","reported_seq":421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.457128+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T11:02:16.457128+0000","last_peered":"2024-04-20T11:02:16.457128+0000","last_clean":"2024-04-20T11:02:16.457128+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T11:02:16.457128+0000","last_undegraded":"2024-04-20T11:02:16.457128+0000","last_fullsized":"2024-04-20T11:02:16.457128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":25761,"num_write":337,"num_write_kb":50347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"27'345","reported_seq":382,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.957484+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T11:02:17.957484+0000","last_peered":"2024-04-20T11:02:17.957484+0000","last_clean":"2024-04-20T11:02:17.957484+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T11:02:17.957484+0000","last_undegraded":"2024-04-20T11:02:17.957484+0000","last_fullsized":"2024-04-20T11:02:17.957484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":667,"num_write_kb":1875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"27'340","reported_seq":376,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.759865+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T11:02:14.759865+0000","last_peered":"2024-04-20T11:02:14.759865+0000","last_clean":"2024-04-20T11:02:14.759865+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T11:02:14.759865+0000","last_undegraded":"2024-04-20T11:02:14.759865+0000","last_fullsized":"2024-04-20T11:02:14.759865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15872,"num_write":302,"num_write_kb":13258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"27'209","reported_seq":247,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:22.826423+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T11:02:22.826423+0000","last_peered":"2024-04-20T11:02:22.826423+0000","last_clean":"2024-04-20T11:02:22.826423+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T11:02:22.826423+0000","last_undegraded":"2024-04-20T11:02:22.826423+0000","last_fullsized":"2024-04-20T11:02:22.826423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3862695,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":147,"num_write_kb":3851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'451","reported_seq":521,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.480280+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T11:02:16.480280+0000","last_peered":"2024-04-20T11:02:16.480280+0000","last_clean":"2024-04-20T11:02:16.480280+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T11:02:16.480280+0000","last_undegraded":"2024-04-20T11:02:16.480280+0000","last_fullsized":"2024-04-20T11:02:16.480280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":106048,"num_write":421,"num_write_kb":76099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"27'559","reported_seq":592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.947780+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T11:02:17.947780+0000","last_peered":"2024-04-20T11:02:17.947780+0000","last_clean":"2024-04-20T11:02:17.947780+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T11:02:17.947780+0000","last_undegraded":"2024-04-20T11:02:17.947780+0000","last_fullsized":"2024-04-20T11:02:17.947780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":504,"num_write_kb":4393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'459","reported_seq":504,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.466237+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T11:02:16.466237+0000","last_peered":"2024-04-20T11:02:16.466237+0000","last_clean":"2024-04-20T11:02:16.466237+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T11:02:16.466237+0000","last_undegraded":"2024-04-20T11:02:16.466237+0000","last_fullsized":"2024-04-20T11:02:16.466237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":55402,"num_write":430,"num_write_kb":80382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'246","reported_seq":286,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.947314+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T11:02:17.947314+0000","last_peered":"2024-04-20T11:02:17.947314+0000","last_clean":"2024-04-20T11:02:17.947314+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T11:02:17.947314+0000","last_undegraded":"2024-04-20T11:02:17.947314+0000","last_fullsized":"2024-04-20T11:02:17.947314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":197,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'401","reported_seq":442,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.486930+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T11:02:11.486930+0000","last_peered":"2024-04-20T11:02:11.486930+0000","last_clean":"2024-04-20T11:02:11.486930+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T11:02:11.486930+0000","last_undegraded":"2024-04-20T11:02:11.486930+0000","last_fullsized":"2024-04-20T11:02:11.486930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":45668,"num_write":367,"num_write_kb":46153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'230","reported_seq":263,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.768794+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T11:02:14.768794+0000","last_peered":"2024-04-20T11:02:14.768794+0000","last_clean":"2024-04-20T11:02:14.768794+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T11:02:14.768794+0000","last_undegraded":"2024-04-20T11:02:14.768794+0000","last_fullsized":"2024-04-20T11:02:14.768794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":165,"num_write_kb":4183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"27'464","reported_seq":527,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:08.358362+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T11:02:08.358362+0000","last_peered":"2024-04-20T11:02:08.358362+0000","last_clean":"2024-04-20T11:02:08.358362+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T11:02:08.358362+0000","last_undegraded":"2024-04-20T11:02:08.358362+0000","last_fullsized":"2024-04-20T11:02:08.358362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":121848,"num_write":422,"num_write_kb":125716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'140","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.942782+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T11:02:17.942782+0000","last_peered":"2024-04-20T11:02:17.942782+0000","last_clean":"2024-04-20T11:02:17.942782+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T11:02:17.942782+0000","last_undegraded":"2024-04-20T11:02:17.942782+0000","last_fullsized":"2024-04-20T11:02:17.942782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":89,"num_write_kb":8247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"27'435","reported_seq":482,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.480165+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T11:02:16.480165+0000","last_peered":"2024-04-20T11:02:16.480165+0000","last_clean":"2024-04-20T11:02:16.480165+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T11:02:16.480165+0000","last_undegraded":"2024-04-20T11:02:16.480165+0000","last_fullsized":"2024-04-20T11:02:16.480165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":46429,"num_write":408,"num_write_kb":79692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"27'378","reported_seq":407,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.458430+0000","last_change":"2024-04-20T10:56:22.904797+0000","last_active":"2024-04-20T11:02:16.458430+0000","last_peered":"2024-04-20T11:02:16.458430+0000","last_clean":"2024-04-20T11:02:16.458430+0000","last_became_active":"2024-04-20T10:56:22.904460+0000","last_became_peered":"2024-04-20T10:56:22.904460+0000","last_unstale":"2024-04-20T11:02:16.458430+0000","last_undegraded":"2024-04-20T11:02:16.458430+0000","last_fullsized":"2024-04-20T11:02:16.458430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:55.531408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15562,"num_write":337,"num_write_kb":42622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'264","reported_seq":300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.428494+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T11:02:07.428494+0000","last_peered":"2024-04-20T11:02:07.428494+0000","last_clean":"2024-04-20T11:02:07.428494+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T11:02:07.428494+0000","last_undegraded":"2024-04-20T11:02:07.428494+0000","last_fullsized":"2024-04-20T11:02:07.428494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":212,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'380","reported_seq":421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.439845+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T11:02:12.439845+0000","last_peered":"2024-04-20T11:02:12.439845+0000","last_clean":"2024-04-20T11:02:12.439845+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T11:02:12.439845+0000","last_undegraded":"2024-04-20T11:02:12.439845+0000","last_fullsized":"2024-04-20T11:02:12.439845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":31020,"num_write":340,"num_write_kb":46796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"27'219","reported_seq":263,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.427078+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T11:02:07.427078+0000","last_peered":"2024-04-20T11:02:07.427078+0000","last_clean":"2024-04-20T11:02:07.427078+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T11:02:07.427078+0000","last_undegraded":"2024-04-20T11:02:07.427078+0000","last_fullsized":"2024-04-20T11:02:07.427078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":39,"num_read_kb":43,"num_write":158,"num_write_kb":12367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"27'377","reported_seq":430,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.462821+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T11:02:12.462821+0000","last_peered":"2024-04-20T11:02:12.462821+0000","last_clean":"2024-04-20T11:02:12.462821+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T11:02:12.462821+0000","last_undegraded":"2024-04-20T11:02:12.462821+0000","last_fullsized":"2024-04-20T11:02:12.462821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":72309,"num_write":351,"num_write_kb":75679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'88","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.474013+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T11:02:11.474013+0000","last_peered":"2024-04-20T11:02:11.474013+0000","last_clean":"2024-04-20T11:02:11.474013+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T11:02:11.474013+0000","last_undegraded":"2024-04-20T11:02:11.474013+0000","last_fullsized":"2024-04-20T11:02:11.474013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'429","reported_seq":474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.959031+0000","last_change":"2024-04-20T10:56:22.913007+0000","last_active":"2024-04-20T11:02:16.959031+0000","last_peered":"2024-04-20T11:02:16.959031+0000","last_clean":"2024-04-20T11:02:16.959031+0000","last_became_active":"2024-04-20T10:56:22.912718+0000","last_became_peered":"2024-04-20T10:56:22.912718+0000","last_unstale":"2024-04-20T11:02:16.959031+0000","last_undegraded":"2024-04-20T11:02:16.959031+0000","last_fullsized":"2024-04-20T11:02:16.959031+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:35.954727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":67508,"num_write":400,"num_write_kb":96285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1f","version":"27'70","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:08.364621+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T11:02:08.364621+0000","last_peered":"2024-04-20T11:02:08.364621+0000","last_clean":"2024-04-20T11:02:08.364621+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T11:02:08.364621+0000","last_undegraded":"2024-04-20T11:02:08.364621+0000","last_fullsized":"2024-04-20T11:02:08.364621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":25,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1110,"num_read_kb":3290197,"num_write":24364,"num_write_kb":4196200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26562,"ondisk_log_size":26562,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":152132834,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":619,"num_read_kb":1973,"num_write":6421,"num_write_kb":154889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":304513024,"data_stored":304265668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7799,"ondisk_log_size":7799,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115468,"kb_used_data":57924,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94256372,"statfs":{"total":96636764160,"available":96518524928,"internally_reserved":0,"allocated":59314176,"data_stored":59096434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574936,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80316,"kb_used_data":49140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94291524,"statfs":{"total":96636764160,"available":96554520576,"internally_reserved":0,"allocated":50319360,"data_stored":50116448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86548,"kb_used_data":29004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94285292,"statfs":{"total":96636764160,"available":96548139008,"internally_reserved":0,"allocated":29700096,"data_stored":29479029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102908,"kb_used_data":45364,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94268932,"statfs":{"total":96636764160,"available":96531386368,"internally_reserved":0,"allocated":46452736,"data_stored":46253643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74004,"kb_used_data":34508,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94297836,"statfs":{"total":96636764160,"available":96560984064,"internally_reserved":0,"allocated":35336192,"data_stored":35133324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41048,"kb_used_data":14032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330792,"statfs":{"total":96636764160,"available":96594731008,"internally_reserved":0,"allocated":14368768,"data_stored":14161398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94656,"kb_used_data":41272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94277184,"statfs":{"total":96636764160,"available":96539836416,"internally_reserved":0,"allocated":42262528,"data_stored":42059541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574937,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64900,"kb_used_data":29564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94306940,"statfs":{"total":96636764160,"available":96570306560,"internally_reserved":0,"allocated":30273536,"data_stored":30066339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29388800,"data_stored":29361002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41971712,"data_stored":41944572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14077952,"data_stored":14046429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35045376,"data_stored":35018355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46161920,"data_stored":46138674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29409280,"data_stored":29364060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50028544,"data_stored":50001479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":58429440,"data_stored":58391097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:02:27.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:02:27.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.353+0000 7fe2a683f640 1 Processor -- start 2024-04-20T11:02:27.353 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.353+0000 7fe2a683f640 1 -- start start 2024-04-20T11:02:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe2a683f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe2a0151f40 0x7fe2a006f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe2a683f640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe2a015e990 0x7fe2a006fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe2a683f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a00c9970 con 0x7fe2a0152320 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe2a683f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a0070210 con 0x7fe2a0151f40 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe2a683f640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a015b750 con 0x7fe2a015e990 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe29ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe2a0151f40 0x7fe2a006f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe29ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe2a0151f40 0x7fe2a006f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38652/0 (socket says 172.21.15.27:38652) 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe29ffff640 1 -- 172.21.15.27:0/1021523291 learned_addr learned my addr 172.21.15.27:0/1021523291 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe2a4ddc640 1 --2- 172.21.15.27:0/1021523291 >> v2:172.21.15.137:3300/0 conn(0x7fe2a015e990 0x7fe2a006fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe29ffff640 1 -- 172.21.15.27:0/1021523291 >> v2:172.21.15.137:3300/0 conn(0x7fe2a015e990 msgr2=0x7fe2a006fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe29ffff640 1 --2- 172.21.15.27:0/1021523291 >> v2:172.21.15.137:3300/0 conn(0x7fe2a015e990 0x7fe2a006fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe29ffff640 1 -- 172.21.15.27:0/1021523291 >> v1:172.21.15.137:6789/0 conn(0x7fe2a0152320 legacy=0x7fe2a006ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.354+0000 7fe29ffff640 1 -- 172.21.15.27:0/1021523291 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a015ba30 con 0x7fe2a0151f40 2024-04-20T11:02:27.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.355+0000 7fe29ffff640 1 --2- 172.21.15.27:0/1021523291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a0151f40 0x7fe2a006f5c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0073180 tx=0x7fe29402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290c7a5c40391fff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:27.356 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.355+0000 7fe29effd640 1 -- 172.21.15.27:0/1021523291 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29402f850 con 0x7fe2a0151f40 2024-04-20T11:02:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.355+0000 7fe29effd640 1 -- 172.21.15.27:0/1021523291 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29402f9f0 con 0x7fe2a0151f40 2024-04-20T11:02:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.355+0000 7fe29effd640 1 -- 172.21.15.27:0/1021523291 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29402fdc0 con 0x7fe2a0151f40 2024-04-20T11:02:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.356+0000 7fe2a683f640 1 -- 172.21.15.27:0/1021523291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a0151f40 msgr2=0x7fe2a006f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.356+0000 7fe2a683f640 1 --2- 172.21.15.27:0/1021523291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a0151f40 0x7fe2a006f5c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0073180 tx=0x7fe29402ba30 comp rx=0 tx=0).stop 2024-04-20T11:02:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.356+0000 7fe2a683f640 1 -- 172.21.15.27:0/1021523291 shutdown_connections 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.356+0000 7fe2a683f640 1 --2- 172.21.15.27:0/1021523291 >> v2:172.21.15.137:3300/0 conn(0x7fe2a015e990 0x7fe2a006fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.356+0000 7fe2a683f640 1 --2- 172.21.15.27:0/1021523291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a0151f40 0x7fe2a006f5c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.356+0000 7fe2a683f640 1 -- 172.21.15.27:0/1021523291 >> 172.21.15.27:0/1021523291 conn(0x7fe2a00cee90 msgr2=0x7fe2a00cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.356+0000 7fe2a683f640 1 -- 172.21.15.27:0/1021523291 shutdown_connections 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.356+0000 7fe2a683f640 1 -- 172.21.15.27:0/1021523291 wait complete. 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.357+0000 7fe2a683f640 1 Processor -- start 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.357+0000 7fe2a683f640 1 -- start start 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a683f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a0151f40 0x7fe2a014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a683f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2a0152320 0x7fe2a0150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a683f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a015e990 0x7fe2a01f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a683f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a015c2d0 con 0x7fe2a015e990 2024-04-20T11:02:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a683f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a015bff0 con 0x7fe2a0151f40 2024-04-20T11:02:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a683f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a006f0a0 con 0x7fe2a0152320 2024-04-20T11:02:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a4ddc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a015e990 0x7fe2a01f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a4ddc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a015e990 0x7fe2a01f15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38656/0 (socket says 172.21.15.27:38656) 2024-04-20T11:02:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a4ddc640 1 -- 172.21.15.27:0/1867736246 learned_addr learned my addr 172.21.15.27:0/1867736246 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe29ffff640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a0151f40 0x7fe2a014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe29f7fe640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2a0152320 0x7fe2a0150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a4ddc640 1 -- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2a0152320 msgr2=0x7fe2a0150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a4ddc640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2a0152320 0x7fe2a0150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a4ddc640 1 -- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a0151f40 msgr2=0x7fe2a014b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a4ddc640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a0151f40 0x7fe2a014b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a4ddc640 1 -- 172.21.15.27:0/1867736246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a0151570 con 0x7fe2a015e990 2024-04-20T11:02:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe2a4ddc640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a015e990 0x7fe2a01f15e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe298009d30 tx=0x7fe29800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.358+0000 7fe29cff9640 1 -- 172.21.15.27:0/1867736246 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298018020 con 0x7fe2a015e990 2024-04-20T11:02:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.359+0000 7fe29cff9640 1 -- 172.21.15.27:0/1867736246 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe298014480 con 0x7fe2a015e990 2024-04-20T11:02:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.359+0000 7fe29cff9640 1 -- 172.21.15.27:0/1867736246 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298014870 con 0x7fe2a015e990 2024-04-20T11:02:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.359+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a01f4cf0 con 0x7fe2a015e990 2024-04-20T11:02:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.359+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a01f52b0 con 0x7fe2a015e990 2024-04-20T11:02:27.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.360+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a004fcf0 con 0x7fe2a015e990 2024-04-20T11:02:27.363 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.360+0000 7fe29cff9640 1 -- 172.21.15.27:0/1867736246 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe298005d10 con 0x7fe2a015e990 2024-04-20T11:02:27.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.361+0000 7fe29cff9640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe28c06c8f0 0x7fe28c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.361+0000 7fe29cff9640 1 -- 172.21.15.27:0/1867736246 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fe29808c950 con 0x7fe2a015e990 2024-04-20T11:02:27.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.361+0000 7fe29ffff640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe28c06c8f0 0x7fe28c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.361+0000 7fe29ffff640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe28c06c8f0 0x7fe28c06edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe294036000 tx=0x7fe294008030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:27.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.365+0000 7fe29cff9640 1 -- 172.21.15.27:0/1867736246 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe298022030 con 0x7fe2a015e990 2024-04-20T11:02:27.548 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.548+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2a0150590 con 0x7fe28c06c8f0 2024-04-20T11:02:27.554 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.555+0000 7fe29cff9640 1 -- 172.21.15.27:0/1867736246 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253823 (secure 0 0 0) 0x7fe29402f820 con 0x7fe28c06c8f0 2024-04-20T11:02:27.557 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:27.558 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe28c06c8f0 msgr2=0x7fe28c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe28c06c8f0 0x7fe28c06edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe294036000 tx=0x7fe294008030 comp rx=0 tx=0).stop 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a015e990 msgr2=0x7fe2a01f15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a015e990 0x7fe2a01f15e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe298009d30 tx=0x7fe29800c490 comp rx=0 tx=0).stop 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 shutdown_connections 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe28c06c8f0 0x7fe28c06edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a015e990 0x7fe2a01f15e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe2a0152320 0x7fe2a0150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 --2- 172.21.15.27:0/1867736246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a0151f40 0x7fe2a014b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 >> 172.21.15.27:0/1867736246 conn(0x7fe2a00cee90 msgr2=0x7fe2a006d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 shutdown_connections 2024-04-20T11:02:27.561 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.558+0000 7fe2a683f640 1 -- 172.21.15.27:0/1867736246 wait complete. 2024-04-20T11:02:27.569 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-04-20T11:02:25.792397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":152723202,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":1845,"num_read_kb":3292270,"num_write":30944,"num_write_kb":4353505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34478,"ondisk_log_size":34478,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":659848,"kb_used_data":300808,"kb_used_omap":12,"kb_used_meta":358963,"kb_avail":754314872,"statfs":{"total":773094113280,"available":772418428928,"internally_reserved":0,"allocated":308027392,"data_stored":306366156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":367578673},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3210049,"num_objects":-403,"num_object_clones":0,"num_object_copies":-806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-403,"num_whiteouts":0,"num_read":74,"num_read_kb":117,"num_write":480,"num_write_kb":1281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002609"},"pg_stats":[{"pgid":"3.3f","version":"27'417","reported_seq":460,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.434387+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T11:02:12.434387+0000","last_peered":"2024-04-20T11:02:12.434387+0000","last_clean":"2024-04-20T11:02:12.434387+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T11:02:12.434387+0000","last_undegraded":"2024-04-20T11:02:12.434387+0000","last_fullsized":"2024-04-20T11:02:12.434387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":31614,"num_write":375,"num_write_kb":59119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"27'425","reported_seq":470,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.442935+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T11:02:16.442935+0000","last_peered":"2024-04-20T11:02:16.442935+0000","last_clean":"2024-04-20T11:02:16.442935+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T11:02:16.442935+0000","last_undegraded":"2024-04-20T11:02:16.442935+0000","last_fullsized":"2024-04-20T11:02:16.442935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":40656,"num_write":384,"num_write_kb":54681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"27'399","reported_seq":447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.459090+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T11:02:12.459090+0000","last_peered":"2024-04-20T11:02:12.459090+0000","last_clean":"2024-04-20T11:02:12.459090+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T11:02:12.459090+0000","last_undegraded":"2024-04-20T11:02:12.459090+0000","last_fullsized":"2024-04-20T11:02:12.459090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":42823,"num_write":366,"num_write_kb":47539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'384","reported_seq":426,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.471314+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T11:02:16.471314+0000","last_peered":"2024-04-20T11:02:16.471314+0000","last_clean":"2024-04-20T11:02:16.471314+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T11:02:16.471314+0000","last_undegraded":"2024-04-20T11:02:16.471314+0000","last_fullsized":"2024-04-20T11:02:16.471314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":22108,"num_write":353,"num_write_kb":40048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'426","reported_seq":472,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.445404+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T11:02:16.445404+0000","last_peered":"2024-04-20T11:02:16.445404+0000","last_clean":"2024-04-20T11:02:16.445404+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T11:02:16.445404+0000","last_undegraded":"2024-04-20T11:02:16.445404+0000","last_fullsized":"2024-04-20T11:02:16.445404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":63832,"num_write":387,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'365","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.453897+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T11:02:11.453897+0000","last_peered":"2024-04-20T11:02:11.453897+0000","last_clean":"2024-04-20T11:02:11.453897+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T11:02:11.453897+0000","last_undegraded":"2024-04-20T11:02:11.453897+0000","last_fullsized":"2024-04-20T11:02:11.453897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":22263,"num_write":332,"num_write_kb":22043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'431","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.452173+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T11:02:12.452173+0000","last_peered":"2024-04-20T11:02:12.452173+0000","last_clean":"2024-04-20T11:02:12.452173+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T11:02:12.452173+0000","last_undegraded":"2024-04-20T11:02:12.452173+0000","last_fullsized":"2024-04-20T11:02:12.452173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":74379,"num_write":399,"num_write_kb":92171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'360","reported_seq":400,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.478262+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T11:02:16.478262+0000","last_peered":"2024-04-20T11:02:16.478262+0000","last_clean":"2024-04-20T11:02:16.478262+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T11:02:16.478262+0000","last_undegraded":"2024-04-20T11:02:16.478262+0000","last_fullsized":"2024-04-20T11:02:16.478262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":32797,"num_write":314,"num_write_kb":38008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"27'422","reported_seq":468,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.442111+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T11:02:16.442111+0000","last_peered":"2024-04-20T11:02:16.442111+0000","last_clean":"2024-04-20T11:02:16.442111+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T11:02:16.442111+0000","last_undegraded":"2024-04-20T11:02:16.442111+0000","last_fullsized":"2024-04-20T11:02:16.442111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":52787,"num_write":396,"num_write_kb":59707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"27'409","reported_seq":465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.456181+0000","last_change":"2024-04-20T10:56:22.913401+0000","last_active":"2024-04-20T11:02:16.456181+0000","last_peered":"2024-04-20T11:02:16.456181+0000","last_clean":"2024-04-20T11:02:16.456181+0000","last_became_active":"2024-04-20T10:56:22.913113+0000","last_became_peered":"2024-04-20T10:56:22.913113+0000","last_unstale":"2024-04-20T11:02:16.456181+0000","last_undegraded":"2024-04-20T11:02:16.456181+0000","last_fullsized":"2024-04-20T11:02:16.456181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:17.248882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":77299,"num_write":373,"num_write_kb":63557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"27'381","reported_seq":425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.456654+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T11:02:16.456654+0000","last_peered":"2024-04-20T11:02:16.456654+0000","last_clean":"2024-04-20T11:02:16.456654+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T11:02:16.456654+0000","last_undegraded":"2024-04-20T11:02:16.456654+0000","last_fullsized":"2024-04-20T11:02:16.456654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":50462,"num_write":358,"num_write_kb":46678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'387","reported_seq":434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.452830+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T11:02:16.452830+0000","last_peered":"2024-04-20T11:02:16.452830+0000","last_clean":"2024-04-20T11:02:16.452830+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T11:02:16.452830+0000","last_undegraded":"2024-04-20T11:02:16.452830+0000","last_fullsized":"2024-04-20T11:02:16.452830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":64677,"num_write":357,"num_write_kb":71584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"27'405","reported_seq":454,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.452369+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T11:02:12.452369+0000","last_peered":"2024-04-20T11:02:12.452369+0000","last_clean":"2024-04-20T11:02:12.452369+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T11:02:12.452369+0000","last_undegraded":"2024-04-20T11:02:12.452369+0000","last_fullsized":"2024-04-20T11:02:12.452369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":59589,"num_write":367,"num_write_kb":71789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"27'414","reported_seq":451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.450507+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T11:02:16.450507+0000","last_peered":"2024-04-20T11:02:16.450507+0000","last_clean":"2024-04-20T11:02:16.450507+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T11:02:16.450507+0000","last_undegraded":"2024-04-20T11:02:16.450507+0000","last_fullsized":"2024-04-20T11:02:16.450507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":30435,"num_write":362,"num_write_kb":42694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"27'412","reported_seq":453,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.441267+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T11:02:11.441267+0000","last_peered":"2024-04-20T11:02:11.441267+0000","last_clean":"2024-04-20T11:02:11.441267+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T11:02:11.441267+0000","last_undegraded":"2024-04-20T11:02:11.441267+0000","last_fullsized":"2024-04-20T11:02:11.441267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":33945,"num_write":377,"num_write_kb":75986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"27'442","reported_seq":495,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.475976+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T11:02:16.475976+0000","last_peered":"2024-04-20T11:02:16.475976+0000","last_clean":"2024-04-20T11:02:16.475976+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T11:02:16.475976+0000","last_undegraded":"2024-04-20T11:02:16.475976+0000","last_fullsized":"2024-04-20T11:02:16.475976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":54533,"num_write":414,"num_write_kb":39250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"27'419","reported_seq":469,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.472619+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T11:02:16.472619+0000","last_peered":"2024-04-20T11:02:16.472619+0000","last_clean":"2024-04-20T11:02:16.472619+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T11:02:16.472619+0000","last_undegraded":"2024-04-20T11:02:16.472619+0000","last_fullsized":"2024-04-20T11:02:16.472619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":68755,"num_write":396,"num_write_kb":79570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"27'419","reported_seq":466,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.483508+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T11:02:16.483508+0000","last_peered":"2024-04-20T11:02:16.483508+0000","last_clean":"2024-04-20T11:02:16.483508+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T11:02:16.483508+0000","last_undegraded":"2024-04-20T11:02:16.483508+0000","last_fullsized":"2024-04-20T11:02:16.483508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53186,"num_write":383,"num_write_kb":88017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"27'432","reported_seq":484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.453682+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T11:02:16.453682+0000","last_peered":"2024-04-20T11:02:16.453682+0000","last_clean":"2024-04-20T11:02:16.453682+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T11:02:16.453682+0000","last_undegraded":"2024-04-20T11:02:16.453682+0000","last_fullsized":"2024-04-20T11:02:16.453682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":86853,"num_write":391,"num_write_kb":112571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"27'447","reported_seq":488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.489722+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T11:02:11.489722+0000","last_peered":"2024-04-20T11:02:11.489722+0000","last_clean":"2024-04-20T11:02:11.489722+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T11:02:11.489722+0000","last_undegraded":"2024-04-20T11:02:11.489722+0000","last_fullsized":"2024-04-20T11:02:11.489722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36592,"num_write":417,"num_write_kb":59154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"27'412","reported_seq":461,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.451071+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T11:02:16.451071+0000","last_peered":"2024-04-20T11:02:16.451071+0000","last_clean":"2024-04-20T11:02:16.451071+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T11:02:16.451071+0000","last_undegraded":"2024-04-20T11:02:16.451071+0000","last_fullsized":"2024-04-20T11:02:16.451071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":55674,"num_write":374,"num_write_kb":59056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"27'435","reported_seq":480,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.431501+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T11:02:12.431501+0000","last_peered":"2024-04-20T11:02:12.431501+0000","last_clean":"2024-04-20T11:02:12.431501+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T11:02:12.431501+0000","last_undegraded":"2024-04-20T11:02:12.431501+0000","last_fullsized":"2024-04-20T11:02:12.431501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":66341,"num_write":402,"num_write_kb":96389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'401","reported_seq":437,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.447612+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T11:02:16.447612+0000","last_peered":"2024-04-20T11:02:16.447612+0000","last_clean":"2024-04-20T11:02:16.447612+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T11:02:16.447612+0000","last_undegraded":"2024-04-20T11:02:16.447612+0000","last_fullsized":"2024-04-20T11:02:16.447612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":25926,"num_write":360,"num_write_kb":50488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'466","reported_seq":519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.484754+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T11:02:16.484754+0000","last_peered":"2024-04-20T11:02:16.484754+0000","last_clean":"2024-04-20T11:02:16.484754+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T11:02:16.484754+0000","last_undegraded":"2024-04-20T11:02:16.484754+0000","last_fullsized":"2024-04-20T11:02:16.484754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":61071,"num_write":433,"num_write_kb":92498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"27'403","reported_seq":438,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.462861+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T11:02:16.462861+0000","last_peered":"2024-04-20T11:02:16.462861+0000","last_clean":"2024-04-20T11:02:16.462861+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T11:02:16.462861+0000","last_undegraded":"2024-04-20T11:02:16.462861+0000","last_fullsized":"2024-04-20T11:02:16.462861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":21468,"num_write":380,"num_write_kb":33760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"27'445","reported_seq":477,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.458342+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T11:02:16.458342+0000","last_peered":"2024-04-20T11:02:16.458342+0000","last_clean":"2024-04-20T11:02:16.458342+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T11:02:16.458342+0000","last_undegraded":"2024-04-20T11:02:16.458342+0000","last_fullsized":"2024-04-20T11:02:16.458342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8561,"num_write":404,"num_write_kb":47121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'426","reported_seq":481,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.768817+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T11:02:14.768817+0000","last_peered":"2024-04-20T11:02:14.768817+0000","last_clean":"2024-04-20T11:02:14.768817+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T11:02:14.768817+0000","last_undegraded":"2024-04-20T11:02:14.768817+0000","last_fullsized":"2024-04-20T11:02:14.768817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":82311,"num_write":397,"num_write_kb":100756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"27'402","reported_seq":451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.486267+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T11:02:16.486267+0000","last_peered":"2024-04-20T11:02:16.486267+0000","last_clean":"2024-04-20T11:02:16.486267+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T11:02:16.486267+0000","last_undegraded":"2024-04-20T11:02:16.486267+0000","last_fullsized":"2024-04-20T11:02:16.486267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":40362,"num_write":368,"num_write_kb":83934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'430","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.477185+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T11:02:11.477185+0000","last_peered":"2024-04-20T11:02:11.477185+0000","last_clean":"2024-04-20T11:02:11.477185+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T11:02:11.477185+0000","last_undegraded":"2024-04-20T11:02:11.477185+0000","last_fullsized":"2024-04-20T11:02:11.477185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":51907,"num_write":395,"num_write_kb":76367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"27'471","reported_seq":523,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.452623+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T11:02:16.452623+0000","last_peered":"2024-04-20T11:02:16.452623+0000","last_clean":"2024-04-20T11:02:16.452623+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T11:02:16.452623+0000","last_undegraded":"2024-04-20T11:02:16.452623+0000","last_fullsized":"2024-04-20T11:02:16.452623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":45044,"num_write":427,"num_write_kb":43197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"27'413","reported_seq":450,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.458689+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T11:02:16.458689+0000","last_peered":"2024-04-20T11:02:16.458689+0000","last_clean":"2024-04-20T11:02:16.458689+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T11:02:16.458689+0000","last_undegraded":"2024-04-20T11:02:16.458689+0000","last_fullsized":"2024-04-20T11:02:16.458689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":30452,"num_write":380,"num_write_kb":32013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"27'445","reported_seq":481,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.447290+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T11:02:16.447290+0000","last_peered":"2024-04-20T11:02:16.447290+0000","last_clean":"2024-04-20T11:02:16.447290+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T11:02:16.447290+0000","last_undegraded":"2024-04-20T11:02:16.447290+0000","last_fullsized":"2024-04-20T11:02:16.447290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":23674,"num_write":409,"num_write_kb":56716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"27'390","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.453596+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T11:02:16.453596+0000","last_peered":"2024-04-20T11:02:16.453596+0000","last_clean":"2024-04-20T11:02:16.453596+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T11:02:16.453596+0000","last_undegraded":"2024-04-20T11:02:16.453596+0000","last_fullsized":"2024-04-20T11:02:16.453596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36784,"num_write":356,"num_write_kb":38078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'160","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.426958+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T11:02:07.426958+0000","last_peered":"2024-04-20T11:02:07.426958+0000","last_clean":"2024-04-20T11:02:07.426958+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T11:02:07.426958+0000","last_undegraded":"2024-04-20T11:02:07.426958+0000","last_fullsized":"2024-04-20T11:02:07.426958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195298,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":111,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'395","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.447080+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T11:02:16.447080+0000","last_peered":"2024-04-20T11:02:16.447080+0000","last_clean":"2024-04-20T11:02:16.447080+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T11:02:16.447080+0000","last_undegraded":"2024-04-20T11:02:16.447080+0000","last_fullsized":"2024-04-20T11:02:16.447080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":26712,"num_write":367,"num_write_kb":67448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"27'322","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.494755+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T11:02:11.494755+0000","last_peered":"2024-04-20T11:02:11.494755+0000","last_clean":"2024-04-20T11:02:11.494755+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T11:02:11.494755+0000","last_undegraded":"2024-04-20T11:02:11.494755+0000","last_fullsized":"2024-04-20T11:02:11.494755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":260,"num_write_kb":12413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"27'95","reported_seq":129,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:05.455443+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T11:02:05.455443+0000","last_peered":"2024-04-20T11:02:05.455443+0000","last_clean":"2024-04-20T11:02:05.455443+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T11:02:05.455443+0000","last_undegraded":"2024-04-20T11:02:05.455443+0000","last_fullsized":"2024-04-20T11:02:05.455443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":20,"num_write":51,"num_write_kb":4136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'441","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.428113+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T11:02:12.428113+0000","last_peered":"2024-04-20T11:02:12.428113+0000","last_clean":"2024-04-20T11:02:12.428113+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T11:02:12.428113+0000","last_undegraded":"2024-04-20T11:02:12.428113+0000","last_fullsized":"2024-04-20T11:02:12.428113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":16783,"num_write":403,"num_write_kb":42765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"27'797","reported_seq":828,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.485457+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T11:02:16.485457+0000","last_peered":"2024-04-20T11:02:16.485457+0000","last_clean":"2024-04-20T11:02:16.485457+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T11:02:16.485457+0000","last_undegraded":"2024-04-20T11:02:16.485457+0000","last_fullsized":"2024-04-20T11:02:16.485457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":724,"num_write_kb":8631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"27'447","reported_seq":510,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.473487+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T11:02:16.473487+0000","last_peered":"2024-04-20T11:02:16.473487+0000","last_clean":"2024-04-20T11:02:16.473487+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T11:02:16.473487+0000","last_undegraded":"2024-04-20T11:02:16.473487+0000","last_fullsized":"2024-04-20T11:02:16.473487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":131398,"num_write":411,"num_write_kb":92477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"27'73","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.943331+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T11:02:17.943331+0000","last_peered":"2024-04-20T11:02:17.943331+0000","last_clean":"2024-04-20T11:02:17.943331+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T11:02:17.943331+0000","last_undegraded":"2024-04-20T11:02:17.943331+0000","last_fullsized":"2024-04-20T11:02:17.943331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":22,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"27'445","reported_seq":498,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.441375+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T11:02:12.441375+0000","last_peered":"2024-04-20T11:02:12.441375+0000","last_clean":"2024-04-20T11:02:12.441375+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T11:02:12.441375+0000","last_undegraded":"2024-04-20T11:02:12.441375+0000","last_fullsized":"2024-04-20T11:02:12.441375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":75132,"num_write":413,"num_write_kb":100522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'95","reported_seq":131,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.944720+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T11:02:17.944720+0000","last_peered":"2024-04-20T11:02:17.944720+0000","last_clean":"2024-04-20T11:02:17.944720+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T11:02:17.944720+0000","last_undegraded":"2024-04-20T11:02:17.944720+0000","last_fullsized":"2024-04-20T11:02:17.944720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":51,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"27'441","reported_seq":523,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.445313+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T11:02:16.445313+0000","last_peered":"2024-04-20T11:02:16.445313+0000","last_clean":"2024-04-20T11:02:16.445313+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T11:02:16.445313+0000","last_undegraded":"2024-04-20T11:02:16.445313+0000","last_fullsized":"2024-04-20T11:02:16.445313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":176347,"num_write":418,"num_write_kb":171699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"27'268","reported_seq":300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.162494+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T11:02:17.162494+0000","last_peered":"2024-04-20T11:02:17.162494+0000","last_clean":"2024-04-20T11:02:17.162494+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T11:02:17.162494+0000","last_undegraded":"2024-04-20T11:02:17.162494+0000","last_fullsized":"2024-04-20T11:02:17.162494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":225,"num_write_kb":4349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'397","reported_seq":444,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.438660+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T11:02:16.438660+0000","last_peered":"2024-04-20T11:02:16.438660+0000","last_clean":"2024-04-20T11:02:16.438660+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T11:02:16.438660+0000","last_undegraded":"2024-04-20T11:02:16.438660+0000","last_fullsized":"2024-04-20T11:02:16.438660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53530,"num_write":359,"num_write_kb":71690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"27'395","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.945316+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T11:02:17.945316+0000","last_peered":"2024-04-20T11:02:17.945316+0000","last_clean":"2024-04-20T11:02:17.945316+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T11:02:17.945316+0000","last_undegraded":"2024-04-20T11:02:17.945316+0000","last_fullsized":"2024-04-20T11:02:17.945316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":364,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"27'423","reported_seq":474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.471695+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T11:02:16.471695+0000","last_peered":"2024-04-20T11:02:16.471695+0000","last_clean":"2024-04-20T11:02:16.471695+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T11:02:16.471695+0000","last_undegraded":"2024-04-20T11:02:16.471695+0000","last_fullsized":"2024-04-20T11:02:16.471695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":72220,"num_write":383,"num_write_kb":72034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"27'357","reported_seq":396,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.946815+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T11:02:17.946815+0000","last_peered":"2024-04-20T11:02:17.946815+0000","last_clean":"2024-04-20T11:02:17.946815+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T11:02:17.946815+0000","last_undegraded":"2024-04-20T11:02:17.946815+0000","last_fullsized":"2024-04-20T11:02:17.946815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":299,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"27'397","reported_seq":434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.483200+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T11:02:16.483200+0000","last_peered":"2024-04-20T11:02:16.483200+0000","last_clean":"2024-04-20T11:02:16.483200+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T11:02:16.483200+0000","last_undegraded":"2024-04-20T11:02:16.483200+0000","last_fullsized":"2024-04-20T11:02:16.483200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":32812,"num_write":361,"num_write_kb":30179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"27'73","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.943894+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T11:02:17.943894+0000","last_peered":"2024-04-20T11:02:17.943894+0000","last_clean":"2024-04-20T11:02:17.943894+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T11:02:17.943894+0000","last_undegraded":"2024-04-20T11:02:17.943894+0000","last_fullsized":"2024-04-20T11:02:17.943894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":17,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'403","reported_seq":446,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.441404+0000","last_change":"2024-04-20T10:56:22.904413+0000","last_active":"2024-04-20T11:02:16.441404+0000","last_peered":"2024-04-20T11:02:16.441404+0000","last_clean":"2024-04-20T11:02:16.441404+0000","last_became_active":"2024-04-20T10:56:22.904118+0000","last_became_peered":"2024-04-20T10:56:22.904118+0000","last_unstale":"2024-04-20T11:02:16.441404+0000","last_undegraded":"2024-04-20T11:02:16.441404+0000","last_fullsized":"2024-04-20T11:02:16.441404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:06.204840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":63538,"num_write":372,"num_write_kb":113421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"27'78","reported_seq":113,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:01.406304+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T11:02:01.406304+0000","last_peered":"2024-04-20T11:02:01.406304+0000","last_clean":"2024-04-20T11:02:01.406304+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T11:02:01.406304+0000","last_undegraded":"2024-04-20T11:02:01.406304+0000","last_fullsized":"2024-04-20T11:02:01.406304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":10,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"27'438","reported_seq":486,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.451206+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T11:02:16.451206+0000","last_peered":"2024-04-20T11:02:16.451206+0000","last_clean":"2024-04-20T11:02:16.451206+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T11:02:16.451206+0000","last_undegraded":"2024-04-20T11:02:16.451206+0000","last_fullsized":"2024-04-20T11:02:16.451206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":68877,"num_write":411,"num_write_kb":59157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"27'361","reported_seq":397,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.948421+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T11:02:17.948421+0000","last_peered":"2024-04-20T11:02:17.948421+0000","last_clean":"2024-04-20T11:02:17.948421+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T11:02:17.948421+0000","last_undegraded":"2024-04-20T11:02:17.948421+0000","last_fullsized":"2024-04-20T11:02:17.948421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":307,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"27'471","reported_seq":525,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.473146+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T11:02:16.473146+0000","last_peered":"2024-04-20T11:02:16.473146+0000","last_clean":"2024-04-20T11:02:16.473146+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T11:02:16.473146+0000","last_undegraded":"2024-04-20T11:02:16.473146+0000","last_fullsized":"2024-04-20T11:02:16.473146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":59233,"num_write":434,"num_write_kb":71822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'75","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.471665+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T11:02:11.471665+0000","last_peered":"2024-04-20T11:02:11.471665+0000","last_clean":"2024-04-20T11:02:11.471665+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T11:02:11.471665+0000","last_undegraded":"2024-04-20T11:02:11.471665+0000","last_fullsized":"2024-04-20T11:02:11.471665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"27'408","reported_seq":469,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.444200+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T11:02:16.444200+0000","last_peered":"2024-04-20T11:02:16.444200+0000","last_clean":"2024-04-20T11:02:16.444200+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T11:02:16.444200+0000","last_undegraded":"2024-04-20T11:02:16.444200+0000","last_fullsized":"2024-04-20T11:02:16.444200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":108675,"num_write":383,"num_write_kb":121193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'262","reported_seq":296,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:02.418420+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T11:02:02.418420+0000","last_peered":"2024-04-20T11:02:02.418420+0000","last_clean":"2024-04-20T11:02:02.418420+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T11:02:02.418420+0000","last_undegraded":"2024-04-20T11:02:02.418420+0000","last_fullsized":"2024-04-20T11:02:02.418420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":218,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"27'384","reported_seq":425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.464192+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T11:02:16.464192+0000","last_peered":"2024-04-20T11:02:16.464192+0000","last_clean":"2024-04-20T11:02:16.464192+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T11:02:16.464192+0000","last_undegraded":"2024-04-20T11:02:16.464192+0000","last_fullsized":"2024-04-20T11:02:16.464192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":38645,"num_write":351,"num_write_kb":38372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"27'464","reported_seq":494,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.419741+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T11:02:07.419741+0000","last_peered":"2024-04-20T11:02:07.419741+0000","last_clean":"2024-04-20T11:02:07.419741+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T11:02:07.419741+0000","last_undegraded":"2024-04-20T11:02:07.419741+0000","last_fullsized":"2024-04-20T11:02:07.419741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"27'460","reported_seq":500,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.477443+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T11:02:16.477443+0000","last_peered":"2024-04-20T11:02:16.477443+0000","last_clean":"2024-04-20T11:02:16.477443+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T11:02:16.477443+0000","last_undegraded":"2024-04-20T11:02:16.477443+0000","last_fullsized":"2024-04-20T11:02:16.477443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":42813,"num_write":427,"num_write_kb":83769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"27'70","reported_seq":105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:05.439468+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T11:02:05.439468+0000","last_peered":"2024-04-20T11:02:05.439468+0000","last_clean":"2024-04-20T11:02:05.439468+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T11:02:05.439468+0000","last_undegraded":"2024-04-20T11:02:05.439468+0000","last_fullsized":"2024-04-20T11:02:05.439468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":8,"num_write":11,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"27'397","reported_seq":428,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.454049+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T11:02:11.454049+0000","last_peered":"2024-04-20T11:02:11.454049+0000","last_clean":"2024-04-20T11:02:11.454049+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T11:02:11.454049+0000","last_undegraded":"2024-04-20T11:02:11.454049+0000","last_fullsized":"2024-04-20T11:02:11.454049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6180,"num_write":356,"num_write_kb":9278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:01:52.405499+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T11:01:52.405499+0000","last_peered":"2024-04-20T11:01:52.405499+0000","last_clean":"2024-04-20T11:01:52.405499+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T11:01:52.405499+0000","last_undegraded":"2024-04-20T11:01:52.405499+0000","last_fullsized":"2024-04-20T11:01:52.405499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"27'77","reported_seq":115,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.490479+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T11:02:16.490479+0000","last_peered":"2024-04-20T11:02:16.490479+0000","last_clean":"2024-04-20T11:02:16.490479+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T11:02:16.490479+0000","last_undegraded":"2024-04-20T11:02:16.490479+0000","last_fullsized":"2024-04-20T11:02:16.490479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":18,"num_write":19,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'352","reported_seq":387,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.461407+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T11:02:12.461407+0000","last_peered":"2024-04-20T11:02:12.461407+0000","last_clean":"2024-04-20T11:02:12.461407+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T11:02:12.461407+0000","last_undegraded":"2024-04-20T11:02:12.461407+0000","last_fullsized":"2024-04-20T11:02:12.461407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":17282,"num_write":327,"num_write_kb":42097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"27'122","reported_seq":155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:02.416875+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T11:02:02.416875+0000","last_peered":"2024-04-20T11:02:02.416875+0000","last_clean":"2024-04-20T11:02:02.416875+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T11:02:02.416875+0000","last_undegraded":"2024-04-20T11:02:02.416875+0000","last_fullsized":"2024-04-20T11:02:02.416875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":64,"num_write_kb":8241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"27'380","reported_seq":421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.453605+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T11:02:12.453605+0000","last_peered":"2024-04-20T11:02:12.453605+0000","last_clean":"2024-04-20T11:02:12.453605+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T11:02:12.453605+0000","last_undegraded":"2024-04-20T11:02:12.453605+0000","last_fullsized":"2024-04-20T11:02:12.453605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":18713,"num_write":347,"num_write_kb":51198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"27'326","reported_seq":611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:18.437066+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T11:02:18.437066+0000","last_peered":"2024-04-20T11:02:18.437066+0000","last_clean":"2024-04-20T11:02:18.437066+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T11:02:18.437066+0000","last_undegraded":"2024-04-20T11:02:18.437066+0000","last_fullsized":"2024-04-20T11:02:18.437066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462167,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":258,"num_read_kb":1597,"num_write":262,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"27'410","reported_seq":457,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.447219+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T11:02:11.447219+0000","last_peered":"2024-04-20T11:02:11.447219+0000","last_clean":"2024-04-20T11:02:11.447219+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T11:02:11.447219+0000","last_undegraded":"2024-04-20T11:02:11.447219+0000","last_fullsized":"2024-04-20T11:02:11.447219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":47086,"num_write":383,"num_write_kb":51294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"27'799","reported_seq":838,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:01:59.759462+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T11:01:59.759462+0000","last_peered":"2024-04-20T11:01:59.759462+0000","last_clean":"2024-04-20T11:01:59.759462+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T11:01:59.759462+0000","last_undegraded":"2024-04-20T11:01:59.759462+0000","last_fullsized":"2024-04-20T11:01:59.759462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":731,"num_write_kb":8599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"27'421","reported_seq":471,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.454524+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T11:02:16.454524+0000","last_peered":"2024-04-20T11:02:16.454524+0000","last_clean":"2024-04-20T11:02:16.454524+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T11:02:16.454524+0000","last_undegraded":"2024-04-20T11:02:16.454524+0000","last_fullsized":"2024-04-20T11:02:16.454524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":65250,"num_write":382,"num_write_kb":83821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"27'75","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.455107+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T11:02:16.455107+0000","last_peered":"2024-04-20T11:02:16.455107+0000","last_clean":"2024-04-20T11:02:16.455107+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T11:02:16.455107+0000","last_undegraded":"2024-04-20T11:02:16.455107+0000","last_fullsized":"2024-04-20T11:02:16.455107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"27'421","reported_seq":454,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.469611+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T11:02:16.469611+0000","last_peered":"2024-04-20T11:02:16.469611+0000","last_clean":"2024-04-20T11:02:16.469611+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T11:02:16.469611+0000","last_undegraded":"2024-04-20T11:02:16.469611+0000","last_fullsized":"2024-04-20T11:02:16.469611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":375,"num_write_kb":62820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"27'85","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.454748+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T11:02:16.454748+0000","last_peered":"2024-04-20T11:02:16.454748+0000","last_clean":"2024-04-20T11:02:16.454748+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T11:02:16.454748+0000","last_undegraded":"2024-04-20T11:02:16.454748+0000","last_fullsized":"2024-04-20T11:02:16.454748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":29,"num_write_kb":4110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"27'382","reported_seq":421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.457128+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T11:02:16.457128+0000","last_peered":"2024-04-20T11:02:16.457128+0000","last_clean":"2024-04-20T11:02:16.457128+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T11:02:16.457128+0000","last_undegraded":"2024-04-20T11:02:16.457128+0000","last_fullsized":"2024-04-20T11:02:16.457128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":25761,"num_write":337,"num_write_kb":50347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"27'345","reported_seq":382,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.957484+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T11:02:17.957484+0000","last_peered":"2024-04-20T11:02:17.957484+0000","last_clean":"2024-04-20T11:02:17.957484+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T11:02:17.957484+0000","last_undegraded":"2024-04-20T11:02:17.957484+0000","last_fullsized":"2024-04-20T11:02:17.957484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":667,"num_write_kb":1875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"27'340","reported_seq":376,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.759865+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T11:02:14.759865+0000","last_peered":"2024-04-20T11:02:14.759865+0000","last_clean":"2024-04-20T11:02:14.759865+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T11:02:14.759865+0000","last_undegraded":"2024-04-20T11:02:14.759865+0000","last_fullsized":"2024-04-20T11:02:14.759865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15872,"num_write":302,"num_write_kb":13258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"27'209","reported_seq":247,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:22.826423+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T11:02:22.826423+0000","last_peered":"2024-04-20T11:02:22.826423+0000","last_clean":"2024-04-20T11:02:22.826423+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T11:02:22.826423+0000","last_undegraded":"2024-04-20T11:02:22.826423+0000","last_fullsized":"2024-04-20T11:02:22.826423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3862695,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":147,"num_write_kb":3851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'451","reported_seq":521,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.480280+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T11:02:16.480280+0000","last_peered":"2024-04-20T11:02:16.480280+0000","last_clean":"2024-04-20T11:02:16.480280+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T11:02:16.480280+0000","last_undegraded":"2024-04-20T11:02:16.480280+0000","last_fullsized":"2024-04-20T11:02:16.480280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":106048,"num_write":421,"num_write_kb":76099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"27'559","reported_seq":592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.947780+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T11:02:17.947780+0000","last_peered":"2024-04-20T11:02:17.947780+0000","last_clean":"2024-04-20T11:02:17.947780+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T11:02:17.947780+0000","last_undegraded":"2024-04-20T11:02:17.947780+0000","last_fullsized":"2024-04-20T11:02:17.947780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":504,"num_write_kb":4393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'459","reported_seq":504,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.466237+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T11:02:16.466237+0000","last_peered":"2024-04-20T11:02:16.466237+0000","last_clean":"2024-04-20T11:02:16.466237+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T11:02:16.466237+0000","last_undegraded":"2024-04-20T11:02:16.466237+0000","last_fullsized":"2024-04-20T11:02:16.466237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":55402,"num_write":430,"num_write_kb":80382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'246","reported_seq":286,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.947314+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T11:02:17.947314+0000","last_peered":"2024-04-20T11:02:17.947314+0000","last_clean":"2024-04-20T11:02:17.947314+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T11:02:17.947314+0000","last_undegraded":"2024-04-20T11:02:17.947314+0000","last_fullsized":"2024-04-20T11:02:17.947314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":197,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'401","reported_seq":442,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.486930+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T11:02:11.486930+0000","last_peered":"2024-04-20T11:02:11.486930+0000","last_clean":"2024-04-20T11:02:11.486930+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T11:02:11.486930+0000","last_undegraded":"2024-04-20T11:02:11.486930+0000","last_fullsized":"2024-04-20T11:02:11.486930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":45668,"num_write":367,"num_write_kb":46153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'230","reported_seq":263,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.768794+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T11:02:14.768794+0000","last_peered":"2024-04-20T11:02:14.768794+0000","last_clean":"2024-04-20T11:02:14.768794+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T11:02:14.768794+0000","last_undegraded":"2024-04-20T11:02:14.768794+0000","last_fullsized":"2024-04-20T11:02:14.768794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":165,"num_write_kb":4183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"27'464","reported_seq":527,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:08.358362+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T11:02:08.358362+0000","last_peered":"2024-04-20T11:02:08.358362+0000","last_clean":"2024-04-20T11:02:08.358362+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T11:02:08.358362+0000","last_undegraded":"2024-04-20T11:02:08.358362+0000","last_fullsized":"2024-04-20T11:02:08.358362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":121848,"num_write":422,"num_write_kb":125716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'140","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.942782+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T11:02:17.942782+0000","last_peered":"2024-04-20T11:02:17.942782+0000","last_clean":"2024-04-20T11:02:17.942782+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T11:02:17.942782+0000","last_undegraded":"2024-04-20T11:02:17.942782+0000","last_fullsized":"2024-04-20T11:02:17.942782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":89,"num_write_kb":8247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"27'435","reported_seq":482,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.480165+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T11:02:16.480165+0000","last_peered":"2024-04-20T11:02:16.480165+0000","last_clean":"2024-04-20T11:02:16.480165+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T11:02:16.480165+0000","last_undegraded":"2024-04-20T11:02:16.480165+0000","last_fullsized":"2024-04-20T11:02:16.480165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":46429,"num_write":408,"num_write_kb":79692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"27'378","reported_seq":407,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.458430+0000","last_change":"2024-04-20T10:56:22.904797+0000","last_active":"2024-04-20T11:02:16.458430+0000","last_peered":"2024-04-20T11:02:16.458430+0000","last_clean":"2024-04-20T11:02:16.458430+0000","last_became_active":"2024-04-20T10:56:22.904460+0000","last_became_peered":"2024-04-20T10:56:22.904460+0000","last_unstale":"2024-04-20T11:02:16.458430+0000","last_undegraded":"2024-04-20T11:02:16.458430+0000","last_fullsized":"2024-04-20T11:02:16.458430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:55.531408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15562,"num_write":337,"num_write_kb":42622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'264","reported_seq":300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.428494+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T11:02:07.428494+0000","last_peered":"2024-04-20T11:02:07.428494+0000","last_clean":"2024-04-20T11:02:07.428494+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T11:02:07.428494+0000","last_undegraded":"2024-04-20T11:02:07.428494+0000","last_fullsized":"2024-04-20T11:02:07.428494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":212,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'380","reported_seq":421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.439845+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T11:02:12.439845+0000","last_peered":"2024-04-20T11:02:12.439845+0000","last_clean":"2024-04-20T11:02:12.439845+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T11:02:12.439845+0000","last_undegraded":"2024-04-20T11:02:12.439845+0000","last_fullsized":"2024-04-20T11:02:12.439845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":31020,"num_write":340,"num_write_kb":46796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"27'219","reported_seq":263,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.427078+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T11:02:07.427078+0000","last_peered":"2024-04-20T11:02:07.427078+0000","last_clean":"2024-04-20T11:02:07.427078+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T11:02:07.427078+0000","last_undegraded":"2024-04-20T11:02:07.427078+0000","last_fullsized":"2024-04-20T11:02:07.427078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":39,"num_read_kb":43,"num_write":158,"num_write_kb":12367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"27'377","reported_seq":430,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.462821+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T11:02:12.462821+0000","last_peered":"2024-04-20T11:02:12.462821+0000","last_clean":"2024-04-20T11:02:12.462821+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T11:02:12.462821+0000","last_undegraded":"2024-04-20T11:02:12.462821+0000","last_fullsized":"2024-04-20T11:02:12.462821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":72309,"num_write":351,"num_write_kb":75679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'88","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.474013+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T11:02:11.474013+0000","last_peered":"2024-04-20T11:02:11.474013+0000","last_clean":"2024-04-20T11:02:11.474013+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T11:02:11.474013+0000","last_undegraded":"2024-04-20T11:02:11.474013+0000","last_fullsized":"2024-04-20T11:02:11.474013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'429","reported_seq":474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.959031+0000","last_change":"2024-04-20T10:56:22.913007+0000","last_active":"2024-04-20T11:02:16.959031+0000","last_peered":"2024-04-20T11:02:16.959031+0000","last_clean":"2024-04-20T11:02:16.959031+0000","last_became_active":"2024-04-20T10:56:22.912718+0000","last_became_peered":"2024-04-20T10:56:22.912718+0000","last_unstale":"2024-04-20T11:02:16.959031+0000","last_undegraded":"2024-04-20T11:02:16.959031+0000","last_fullsized":"2024-04-20T11:02:16.959031+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:35.954727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":67508,"num_write":400,"num_write_kb":96285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1f","version":"27'70","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:08.364621+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T11:02:08.364621+0000","last_peered":"2024-04-20T11:02:08.364621+0000","last_clean":"2024-04-20T11:02:08.364621+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T11:02:08.364621+0000","last_undegraded":"2024-04-20T11:02:08.364621+0000","last_fullsized":"2024-04-20T11:02:08.364621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":25,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1110,"num_read_kb":3290197,"num_write":24364,"num_write_kb":4196200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26562,"ondisk_log_size":26562,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":152132834,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":619,"num_read_kb":1973,"num_write":6421,"num_write_kb":154889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":304513024,"data_stored":304265668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7799,"ondisk_log_size":7799,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115468,"kb_used_data":57924,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94256372,"statfs":{"total":96636764160,"available":96518524928,"internally_reserved":0,"allocated":59314176,"data_stored":59096434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574936,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80316,"kb_used_data":49140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94291524,"statfs":{"total":96636764160,"available":96554520576,"internally_reserved":0,"allocated":50319360,"data_stored":50116448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86548,"kb_used_data":29004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94285292,"statfs":{"total":96636764160,"available":96548139008,"internally_reserved":0,"allocated":29700096,"data_stored":29479029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102908,"kb_used_data":45364,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94268932,"statfs":{"total":96636764160,"available":96531386368,"internally_reserved":0,"allocated":46452736,"data_stored":46253643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74004,"kb_used_data":34508,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94297836,"statfs":{"total":96636764160,"available":96560984064,"internally_reserved":0,"allocated":35336192,"data_stored":35133324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41048,"kb_used_data":14032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330792,"statfs":{"total":96636764160,"available":96594731008,"internally_reserved":0,"allocated":14368768,"data_stored":14161398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94656,"kb_used_data":41272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94277184,"statfs":{"total":96636764160,"available":96539836416,"internally_reserved":0,"allocated":42262528,"data_stored":42059541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574937,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64900,"kb_used_data":29564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94306940,"statfs":{"total":96636764160,"available":96570306560,"internally_reserved":0,"allocated":30273536,"data_stored":30066339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29388800,"data_stored":29361002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41971712,"data_stored":41944572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14077952,"data_stored":14046429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35045376,"data_stored":35018355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46161920,"data_stored":46138674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29409280,"data_stored":29364060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50028544,"data_stored":50001479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":58429440,"data_stored":58391097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:02:27.573 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T11:02:27.573 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:02:27.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.710+0000 7ff5c6f3b640 1 Processor -- start 2024-04-20T11:02:27.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.711+0000 7ff5c6f3b640 1 -- start start 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.711+0000 7ff5c6f3b640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff5c0155950 0x7ff5c00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.711+0000 7ff5c6f3b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff5c00cc330 0x7ff5c00b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.711+0000 7ff5c6f3b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c015b9c0 con 0x7ff5c00b5a30 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.711+0000 7ff5c6f3b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c015bca0 con 0x7ff5c00cc330 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.711+0000 7ff5c6f3b640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c015bf80 con 0x7ff5c0155950 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.711+0000 7ff5bffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff5c00cc330 0x7ff5c00b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.711+0000 7ff5c54d8640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff5c00b5a30 0x7ff5c00d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58640/0 (socket says 172.21.15.27:58640) 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.712+0000 7ff5c54d8640 1 -- 172.21.15.27:0/4234719487 learned_addr learned my addr 172.21.15.27:0/4234719487 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.712+0000 7ff5c4cd7640 1 --2- 172.21.15.27:0/4234719487 >> v2:172.21.15.137:3300/0 conn(0x7ff5c0155950 0x7ff5c00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.712+0000 7ff5bffff640 1 -- 172.21.15.27:0/4234719487 >> v2:172.21.15.137:3300/0 conn(0x7ff5c0155950 msgr2=0x7ff5c00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:27.712 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.712+0000 7ff5bffff640 1 --2- 172.21.15.27:0/4234719487 >> v2:172.21.15.137:3300/0 conn(0x7ff5c0155950 0x7ff5c00cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.712+0000 7ff5bffff640 1 -- 172.21.15.27:0/4234719487 >> v1:172.21.15.27:6789/0 conn(0x7ff5c00b5a30 legacy=0x7ff5c00d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.712+0000 7ff5bf7fe640 1 -- 172.21.15.27:0/4234719487 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670290983 0 0) 0x7ff5c015b9c0 con 0x7ff5c00b5a30 2024-04-20T11:02:27.713 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.712+0000 7ff5bffff640 1 -- 172.21.15.27:0/4234719487 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c015c260 con 0x7ff5c00cc330 2024-04-20T11:02:27.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5bffff640 1 --2- 172.21.15.27:0/4234719487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 0x7ff5c00b54f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ce7c0 tx=0x7ff5ac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a91ef80fbe8fba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:27.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5bf7fe640 1 -- 172.21.15.27:0/4234719487 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac034070 con 0x7ff5c00cc330 2024-04-20T11:02:27.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5bf7fe640 1 -- 172.21.15.27:0/4234719487 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac030610 con 0x7ff5c00cc330 2024-04-20T11:02:27.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5bf7fe640 1 -- 172.21.15.27:0/4234719487 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0309e0 con 0x7ff5c00cc330 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/4234719487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 msgr2=0x7ff5c00b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5c6f3b640 1 --2- 172.21.15.27:0/4234719487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 0x7ff5c00b54f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ce7c0 tx=0x7ff5ac02b960 comp rx=0 tx=0).stop 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/4234719487 shutdown_connections 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5c6f3b640 1 --2- 172.21.15.27:0/4234719487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 0x7ff5c00b54f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5c6f3b640 1 --2- 172.21.15.27:0/4234719487 >> v2:172.21.15.137:3300/0 conn(0x7ff5c0155950 0x7ff5c00cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/4234719487 >> 172.21.15.27:0/4234719487 conn(0x7ff5c00676c0 msgr2=0x7ff5c0067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/4234719487 shutdown_connections 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.713+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/4234719487 wait complete. 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.714+0000 7ff5c6f3b640 1 Processor -- start 2024-04-20T11:02:27.715 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.715+0000 7ff5c6f3b640 1 -- start start 2024-04-20T11:02:27.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.715+0000 7ff5c6f3b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff5c00b5a30 0x7ff5c01f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.715+0000 7ff5c6f3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 0x7ff5c01548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.715+0000 7ff5c6f3b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c0155950 0x7ff5c015e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.715+0000 7ff5c6f3b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c015fb90 con 0x7ff5c00cc330 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.715+0000 7ff5c6f3b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00b5e40 con 0x7ff5c0155950 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.715+0000 7ff5c6f3b640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c015fd10 con 0x7ff5c00b5a30 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.715+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 0x7ff5c01548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 0x7ff5c01548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38680/0 (socket says 172.21.15.27:38680) 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bffff640 1 -- 172.21.15.27:0/2867702953 learned_addr learned my addr 172.21.15.27:0/2867702953 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5c54d8640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c0155950 0x7ff5c015e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5c4cd7640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff5c00b5a30 0x7ff5c01f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bffff640 1 -- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff5c00b5a30 msgr2=0x7ff5c01f1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bffff640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff5c00b5a30 0x7ff5c01f1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bffff640 1 -- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c0155950 msgr2=0x7ff5c015e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.718 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bffff640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c0155950 0x7ff5c015e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bffff640 1 -- 172.21.15.27:0/2867702953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c0155120 con 0x7ff5c00cc330 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5c54d8640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c0155950 0x7ff5c015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bffff640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 0x7ff5c01548e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac02bf70 tx=0x7ff5ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bd7fa640 1 -- 172.21.15.27:0/2867702953 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac034040 con 0x7ff5c00cc330 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bd7fa640 1 -- 172.21.15.27:0/2867702953 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac039a10 con 0x7ff5c00cc330 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c0152c80 con 0x7ff5c00cc330 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5bd7fa640 1 -- 172.21.15.27:0/2867702953 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac039e20 con 0x7ff5c00cc330 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.716+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c0153240 con 0x7ff5c00cc330 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.717+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c004fcf0 con 0x7ff5c00cc330 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.718+0000 7ff5bd7fa640 1 -- 172.21.15.27:0/2867702953 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5ac037070 con 0x7ff5c00cc330 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.718+0000 7ff5bd7fa640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff5a406c8f0 0x7ff5a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.718+0000 7ff5bd7fa640 1 -- 172.21.15.27:0/2867702953 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7ff5ac0ad870 con 0x7ff5c00cc330 2024-04-20T11:02:27.719 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.718+0000 7ff5c4cd7640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff5a406c8f0 0x7ff5a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:27.720 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.719+0000 7ff5c4cd7640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff5a406c8f0 0x7ff5a406edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4006fd0 tx=0x7ff5b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:27.722 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.722+0000 7ff5bd7fa640 1 -- 172.21.15.27:0/2867702953 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff5ac07bb10 con 0x7ff5c00cc330 2024-04-20T11:02:27.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.905+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5c0153d20 con 0x7ff5a406c8f0 2024-04-20T11:02:27.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.912+0000 7ff5bd7fa640 1 -- 172.21.15.27:0/2867702953 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253820 (secure 0 0 0) 0x7ff5b400e4b0 con 0x7ff5a406c8f0 2024-04-20T11:02:27.914 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:27.915 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T11:02:27.916 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff5a406c8f0 msgr2=0x7ff5a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff5a406c8f0 0x7ff5a406edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4006fd0 tx=0x7ff5b4008040 comp rx=0 tx=0).stop 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 msgr2=0x7ff5c01548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 0x7ff5c01548e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac02bf70 tx=0x7ff5ac0046c0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 shutdown_connections 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff5a406c8f0 0x7ff5a406edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5c0155950 0x7ff5c015e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c00cc330 0x7ff5c01548e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 --2- 172.21.15.27:0/2867702953 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff5c00b5a30 0x7ff5c01f1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 >> 172.21.15.27:0/2867702953 conn(0x7ff5c00676c0 msgr2=0x7ff5c00c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 shutdown_connections 2024-04-20T11:02:27.917 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:27.915+0000 7ff5c6f3b640 1 -- 172.21.15.27:0/2867702953 wait complete. 2024-04-20T11:02:27.926 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-04-20T11:02:27.792985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":152723202,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":1845,"num_read_kb":3292270,"num_write":30944,"num_write_kb":4353505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34478,"ondisk_log_size":34478,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":659848,"kb_used_data":300808,"kb_used_omap":12,"kb_used_meta":358963,"kb_avail":754314872,"statfs":{"total":773094113280,"available":772418428928,"internally_reserved":0,"allocated":308027392,"data_stored":306366156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":367578673},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-973845,"num_objects":-152,"num_object_clones":0,"num_object_copies":-304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-152,"num_whiteouts":0,"num_read":68,"num_read_kb":88,"num_write":183,"num_write_kb":201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002888"},"pg_stats":[{"pgid":"3.3f","version":"27'417","reported_seq":460,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.434387+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T11:02:12.434387+0000","last_peered":"2024-04-20T11:02:12.434387+0000","last_clean":"2024-04-20T11:02:12.434387+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T11:02:12.434387+0000","last_undegraded":"2024-04-20T11:02:12.434387+0000","last_fullsized":"2024-04-20T11:02:12.434387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":31614,"num_write":375,"num_write_kb":59119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"27'425","reported_seq":470,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.442935+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T11:02:16.442935+0000","last_peered":"2024-04-20T11:02:16.442935+0000","last_clean":"2024-04-20T11:02:16.442935+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T11:02:16.442935+0000","last_undegraded":"2024-04-20T11:02:16.442935+0000","last_fullsized":"2024-04-20T11:02:16.442935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":40656,"num_write":384,"num_write_kb":54681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"27'399","reported_seq":447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.459090+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T11:02:12.459090+0000","last_peered":"2024-04-20T11:02:12.459090+0000","last_clean":"2024-04-20T11:02:12.459090+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T11:02:12.459090+0000","last_undegraded":"2024-04-20T11:02:12.459090+0000","last_fullsized":"2024-04-20T11:02:12.459090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:12.188584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":42823,"num_write":366,"num_write_kb":47539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'384","reported_seq":426,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.471314+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T11:02:16.471314+0000","last_peered":"2024-04-20T11:02:16.471314+0000","last_clean":"2024-04-20T11:02:16.471314+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T11:02:16.471314+0000","last_undegraded":"2024-04-20T11:02:16.471314+0000","last_fullsized":"2024-04-20T11:02:16.471314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":22108,"num_write":353,"num_write_kb":40048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'426","reported_seq":472,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.445404+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T11:02:16.445404+0000","last_peered":"2024-04-20T11:02:16.445404+0000","last_clean":"2024-04-20T11:02:16.445404+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T11:02:16.445404+0000","last_undegraded":"2024-04-20T11:02:16.445404+0000","last_fullsized":"2024-04-20T11:02:16.445404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":63832,"num_write":387,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'365","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.453897+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T11:02:11.453897+0000","last_peered":"2024-04-20T11:02:11.453897+0000","last_clean":"2024-04-20T11:02:11.453897+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T11:02:11.453897+0000","last_undegraded":"2024-04-20T11:02:11.453897+0000","last_fullsized":"2024-04-20T11:02:11.453897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":22263,"num_write":332,"num_write_kb":22043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'431","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.452173+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T11:02:12.452173+0000","last_peered":"2024-04-20T11:02:12.452173+0000","last_clean":"2024-04-20T11:02:12.452173+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T11:02:12.452173+0000","last_undegraded":"2024-04-20T11:02:12.452173+0000","last_fullsized":"2024-04-20T11:02:12.452173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":74379,"num_write":399,"num_write_kb":92171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'360","reported_seq":400,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.478262+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T11:02:16.478262+0000","last_peered":"2024-04-20T11:02:16.478262+0000","last_clean":"2024-04-20T11:02:16.478262+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T11:02:16.478262+0000","last_undegraded":"2024-04-20T11:02:16.478262+0000","last_fullsized":"2024-04-20T11:02:16.478262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":32797,"num_write":314,"num_write_kb":38008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"27'422","reported_seq":468,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.442111+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T11:02:16.442111+0000","last_peered":"2024-04-20T11:02:16.442111+0000","last_clean":"2024-04-20T11:02:16.442111+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T11:02:16.442111+0000","last_undegraded":"2024-04-20T11:02:16.442111+0000","last_fullsized":"2024-04-20T11:02:16.442111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":52787,"num_write":396,"num_write_kb":59707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"27'409","reported_seq":465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.456181+0000","last_change":"2024-04-20T10:56:22.913401+0000","last_active":"2024-04-20T11:02:16.456181+0000","last_peered":"2024-04-20T11:02:16.456181+0000","last_clean":"2024-04-20T11:02:16.456181+0000","last_became_active":"2024-04-20T10:56:22.913113+0000","last_became_peered":"2024-04-20T10:56:22.913113+0000","last_unstale":"2024-04-20T11:02:16.456181+0000","last_undegraded":"2024-04-20T11:02:16.456181+0000","last_fullsized":"2024-04-20T11:02:16.456181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:17.248882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":77299,"num_write":373,"num_write_kb":63557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"27'381","reported_seq":425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.456654+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T11:02:16.456654+0000","last_peered":"2024-04-20T11:02:16.456654+0000","last_clean":"2024-04-20T11:02:16.456654+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T11:02:16.456654+0000","last_undegraded":"2024-04-20T11:02:16.456654+0000","last_fullsized":"2024-04-20T11:02:16.456654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":50462,"num_write":358,"num_write_kb":46678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'387","reported_seq":434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.452830+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T11:02:16.452830+0000","last_peered":"2024-04-20T11:02:16.452830+0000","last_clean":"2024-04-20T11:02:16.452830+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T11:02:16.452830+0000","last_undegraded":"2024-04-20T11:02:16.452830+0000","last_fullsized":"2024-04-20T11:02:16.452830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":64677,"num_write":357,"num_write_kb":71584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"27'405","reported_seq":454,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.452369+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T11:02:12.452369+0000","last_peered":"2024-04-20T11:02:12.452369+0000","last_clean":"2024-04-20T11:02:12.452369+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T11:02:12.452369+0000","last_undegraded":"2024-04-20T11:02:12.452369+0000","last_fullsized":"2024-04-20T11:02:12.452369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":59589,"num_write":367,"num_write_kb":71789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"27'414","reported_seq":451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.450507+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T11:02:16.450507+0000","last_peered":"2024-04-20T11:02:16.450507+0000","last_clean":"2024-04-20T11:02:16.450507+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T11:02:16.450507+0000","last_undegraded":"2024-04-20T11:02:16.450507+0000","last_fullsized":"2024-04-20T11:02:16.450507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:46:07.681991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":30435,"num_write":362,"num_write_kb":42694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"27'412","reported_seq":453,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.441267+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T11:02:11.441267+0000","last_peered":"2024-04-20T11:02:11.441267+0000","last_clean":"2024-04-20T11:02:11.441267+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T11:02:11.441267+0000","last_undegraded":"2024-04-20T11:02:11.441267+0000","last_fullsized":"2024-04-20T11:02:11.441267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":33945,"num_write":377,"num_write_kb":75986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"27'442","reported_seq":495,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.475976+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T11:02:16.475976+0000","last_peered":"2024-04-20T11:02:16.475976+0000","last_clean":"2024-04-20T11:02:16.475976+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T11:02:16.475976+0000","last_undegraded":"2024-04-20T11:02:16.475976+0000","last_fullsized":"2024-04-20T11:02:16.475976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":54533,"num_write":414,"num_write_kb":39250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"27'419","reported_seq":469,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.472619+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T11:02:16.472619+0000","last_peered":"2024-04-20T11:02:16.472619+0000","last_clean":"2024-04-20T11:02:16.472619+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T11:02:16.472619+0000","last_undegraded":"2024-04-20T11:02:16.472619+0000","last_fullsized":"2024-04-20T11:02:16.472619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":68755,"num_write":396,"num_write_kb":79570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"27'419","reported_seq":466,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.483508+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T11:02:16.483508+0000","last_peered":"2024-04-20T11:02:16.483508+0000","last_clean":"2024-04-20T11:02:16.483508+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T11:02:16.483508+0000","last_undegraded":"2024-04-20T11:02:16.483508+0000","last_fullsized":"2024-04-20T11:02:16.483508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53186,"num_write":383,"num_write_kb":88017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"27'432","reported_seq":484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.453682+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T11:02:16.453682+0000","last_peered":"2024-04-20T11:02:16.453682+0000","last_clean":"2024-04-20T11:02:16.453682+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T11:02:16.453682+0000","last_undegraded":"2024-04-20T11:02:16.453682+0000","last_fullsized":"2024-04-20T11:02:16.453682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":86853,"num_write":391,"num_write_kb":112571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"27'447","reported_seq":488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.489722+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T11:02:11.489722+0000","last_peered":"2024-04-20T11:02:11.489722+0000","last_clean":"2024-04-20T11:02:11.489722+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T11:02:11.489722+0000","last_undegraded":"2024-04-20T11:02:11.489722+0000","last_fullsized":"2024-04-20T11:02:11.489722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36592,"num_write":417,"num_write_kb":59154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"27'412","reported_seq":461,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.451071+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T11:02:16.451071+0000","last_peered":"2024-04-20T11:02:16.451071+0000","last_clean":"2024-04-20T11:02:16.451071+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T11:02:16.451071+0000","last_undegraded":"2024-04-20T11:02:16.451071+0000","last_fullsized":"2024-04-20T11:02:16.451071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:35:15.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":55674,"num_write":374,"num_write_kb":59056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"27'435","reported_seq":480,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.431501+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T11:02:12.431501+0000","last_peered":"2024-04-20T11:02:12.431501+0000","last_clean":"2024-04-20T11:02:12.431501+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T11:02:12.431501+0000","last_undegraded":"2024-04-20T11:02:12.431501+0000","last_fullsized":"2024-04-20T11:02:12.431501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":66341,"num_write":402,"num_write_kb":96389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'401","reported_seq":437,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.447612+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T11:02:16.447612+0000","last_peered":"2024-04-20T11:02:16.447612+0000","last_clean":"2024-04-20T11:02:16.447612+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T11:02:16.447612+0000","last_undegraded":"2024-04-20T11:02:16.447612+0000","last_fullsized":"2024-04-20T11:02:16.447612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":25926,"num_write":360,"num_write_kb":50488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'466","reported_seq":519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.484754+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T11:02:16.484754+0000","last_peered":"2024-04-20T11:02:16.484754+0000","last_clean":"2024-04-20T11:02:16.484754+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T11:02:16.484754+0000","last_undegraded":"2024-04-20T11:02:16.484754+0000","last_fullsized":"2024-04-20T11:02:16.484754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":61071,"num_write":433,"num_write_kb":92498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"27'403","reported_seq":438,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.462861+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T11:02:16.462861+0000","last_peered":"2024-04-20T11:02:16.462861+0000","last_clean":"2024-04-20T11:02:16.462861+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T11:02:16.462861+0000","last_undegraded":"2024-04-20T11:02:16.462861+0000","last_fullsized":"2024-04-20T11:02:16.462861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":21468,"num_write":380,"num_write_kb":33760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"27'445","reported_seq":477,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.458342+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T11:02:16.458342+0000","last_peered":"2024-04-20T11:02:16.458342+0000","last_clean":"2024-04-20T11:02:16.458342+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T11:02:16.458342+0000","last_undegraded":"2024-04-20T11:02:16.458342+0000","last_fullsized":"2024-04-20T11:02:16.458342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8561,"num_write":404,"num_write_kb":47121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'426","reported_seq":481,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.768817+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T11:02:14.768817+0000","last_peered":"2024-04-20T11:02:14.768817+0000","last_clean":"2024-04-20T11:02:14.768817+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T11:02:14.768817+0000","last_undegraded":"2024-04-20T11:02:14.768817+0000","last_fullsized":"2024-04-20T11:02:14.768817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":82311,"num_write":397,"num_write_kb":100756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"27'402","reported_seq":451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.486267+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T11:02:16.486267+0000","last_peered":"2024-04-20T11:02:16.486267+0000","last_clean":"2024-04-20T11:02:16.486267+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T11:02:16.486267+0000","last_undegraded":"2024-04-20T11:02:16.486267+0000","last_fullsized":"2024-04-20T11:02:16.486267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":40362,"num_write":368,"num_write_kb":83934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'430","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.477185+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T11:02:11.477185+0000","last_peered":"2024-04-20T11:02:11.477185+0000","last_clean":"2024-04-20T11:02:11.477185+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T11:02:11.477185+0000","last_undegraded":"2024-04-20T11:02:11.477185+0000","last_fullsized":"2024-04-20T11:02:11.477185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:42.372650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":51907,"num_write":395,"num_write_kb":76367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"27'471","reported_seq":523,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.452623+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T11:02:16.452623+0000","last_peered":"2024-04-20T11:02:16.452623+0000","last_clean":"2024-04-20T11:02:16.452623+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T11:02:16.452623+0000","last_undegraded":"2024-04-20T11:02:16.452623+0000","last_fullsized":"2024-04-20T11:02:16.452623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":45044,"num_write":427,"num_write_kb":43197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"27'413","reported_seq":450,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.458689+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T11:02:16.458689+0000","last_peered":"2024-04-20T11:02:16.458689+0000","last_clean":"2024-04-20T11:02:16.458689+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T11:02:16.458689+0000","last_undegraded":"2024-04-20T11:02:16.458689+0000","last_fullsized":"2024-04-20T11:02:16.458689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":30452,"num_write":380,"num_write_kb":32013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"27'445","reported_seq":481,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.447290+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T11:02:16.447290+0000","last_peered":"2024-04-20T11:02:16.447290+0000","last_clean":"2024-04-20T11:02:16.447290+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T11:02:16.447290+0000","last_undegraded":"2024-04-20T11:02:16.447290+0000","last_fullsized":"2024-04-20T11:02:16.447290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":23674,"num_write":409,"num_write_kb":56716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"27'390","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.453596+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T11:02:16.453596+0000","last_peered":"2024-04-20T11:02:16.453596+0000","last_clean":"2024-04-20T11:02:16.453596+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T11:02:16.453596+0000","last_undegraded":"2024-04-20T11:02:16.453596+0000","last_fullsized":"2024-04-20T11:02:16.453596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36784,"num_write":356,"num_write_kb":38078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'160","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.426958+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T11:02:07.426958+0000","last_peered":"2024-04-20T11:02:07.426958+0000","last_clean":"2024-04-20T11:02:07.426958+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T11:02:07.426958+0000","last_undegraded":"2024-04-20T11:02:07.426958+0000","last_fullsized":"2024-04-20T11:02:07.426958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195298,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":111,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'395","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.447080+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T11:02:16.447080+0000","last_peered":"2024-04-20T11:02:16.447080+0000","last_clean":"2024-04-20T11:02:16.447080+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T11:02:16.447080+0000","last_undegraded":"2024-04-20T11:02:16.447080+0000","last_fullsized":"2024-04-20T11:02:16.447080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:46:20.914239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":26712,"num_write":367,"num_write_kb":67448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"27'322","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.494755+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T11:02:11.494755+0000","last_peered":"2024-04-20T11:02:11.494755+0000","last_clean":"2024-04-20T11:02:11.494755+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T11:02:11.494755+0000","last_undegraded":"2024-04-20T11:02:11.494755+0000","last_fullsized":"2024-04-20T11:02:11.494755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":260,"num_write_kb":12413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"27'95","reported_seq":129,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:05.455443+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T11:02:05.455443+0000","last_peered":"2024-04-20T11:02:05.455443+0000","last_clean":"2024-04-20T11:02:05.455443+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T11:02:05.455443+0000","last_undegraded":"2024-04-20T11:02:05.455443+0000","last_fullsized":"2024-04-20T11:02:05.455443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":20,"num_write":51,"num_write_kb":4136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'441","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.428113+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T11:02:12.428113+0000","last_peered":"2024-04-20T11:02:12.428113+0000","last_clean":"2024-04-20T11:02:12.428113+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T11:02:12.428113+0000","last_undegraded":"2024-04-20T11:02:12.428113+0000","last_fullsized":"2024-04-20T11:02:12.428113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":16783,"num_write":403,"num_write_kb":42765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"27'797","reported_seq":828,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.485457+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T11:02:16.485457+0000","last_peered":"2024-04-20T11:02:16.485457+0000","last_clean":"2024-04-20T11:02:16.485457+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T11:02:16.485457+0000","last_undegraded":"2024-04-20T11:02:16.485457+0000","last_fullsized":"2024-04-20T11:02:16.485457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":724,"num_write_kb":8631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"27'447","reported_seq":510,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.473487+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T11:02:16.473487+0000","last_peered":"2024-04-20T11:02:16.473487+0000","last_clean":"2024-04-20T11:02:16.473487+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T11:02:16.473487+0000","last_undegraded":"2024-04-20T11:02:16.473487+0000","last_fullsized":"2024-04-20T11:02:16.473487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":131398,"num_write":411,"num_write_kb":92477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"27'73","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.943331+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T11:02:17.943331+0000","last_peered":"2024-04-20T11:02:17.943331+0000","last_clean":"2024-04-20T11:02:17.943331+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T11:02:17.943331+0000","last_undegraded":"2024-04-20T11:02:17.943331+0000","last_fullsized":"2024-04-20T11:02:17.943331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":22,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"27'445","reported_seq":498,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.441375+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T11:02:12.441375+0000","last_peered":"2024-04-20T11:02:12.441375+0000","last_clean":"2024-04-20T11:02:12.441375+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T11:02:12.441375+0000","last_undegraded":"2024-04-20T11:02:12.441375+0000","last_fullsized":"2024-04-20T11:02:12.441375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":75132,"num_write":413,"num_write_kb":100522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'95","reported_seq":131,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.944720+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T11:02:17.944720+0000","last_peered":"2024-04-20T11:02:17.944720+0000","last_clean":"2024-04-20T11:02:17.944720+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T11:02:17.944720+0000","last_undegraded":"2024-04-20T11:02:17.944720+0000","last_fullsized":"2024-04-20T11:02:17.944720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":51,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"27'441","reported_seq":523,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.445313+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T11:02:16.445313+0000","last_peered":"2024-04-20T11:02:16.445313+0000","last_clean":"2024-04-20T11:02:16.445313+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T11:02:16.445313+0000","last_undegraded":"2024-04-20T11:02:16.445313+0000","last_fullsized":"2024-04-20T11:02:16.445313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.891384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":176347,"num_write":418,"num_write_kb":171699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"27'268","reported_seq":300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.162494+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T11:02:17.162494+0000","last_peered":"2024-04-20T11:02:17.162494+0000","last_clean":"2024-04-20T11:02:17.162494+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T11:02:17.162494+0000","last_undegraded":"2024-04-20T11:02:17.162494+0000","last_fullsized":"2024-04-20T11:02:17.162494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":225,"num_write_kb":4349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'397","reported_seq":444,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.438660+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T11:02:16.438660+0000","last_peered":"2024-04-20T11:02:16.438660+0000","last_clean":"2024-04-20T11:02:16.438660+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T11:02:16.438660+0000","last_undegraded":"2024-04-20T11:02:16.438660+0000","last_fullsized":"2024-04-20T11:02:16.438660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53530,"num_write":359,"num_write_kb":71690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"27'395","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.945316+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T11:02:17.945316+0000","last_peered":"2024-04-20T11:02:17.945316+0000","last_clean":"2024-04-20T11:02:17.945316+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T11:02:17.945316+0000","last_undegraded":"2024-04-20T11:02:17.945316+0000","last_fullsized":"2024-04-20T11:02:17.945316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":364,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"27'423","reported_seq":474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.471695+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T11:02:16.471695+0000","last_peered":"2024-04-20T11:02:16.471695+0000","last_clean":"2024-04-20T11:02:16.471695+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T11:02:16.471695+0000","last_undegraded":"2024-04-20T11:02:16.471695+0000","last_fullsized":"2024-04-20T11:02:16.471695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":72220,"num_write":383,"num_write_kb":72034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"27'357","reported_seq":396,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.946815+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T11:02:17.946815+0000","last_peered":"2024-04-20T11:02:17.946815+0000","last_clean":"2024-04-20T11:02:17.946815+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T11:02:17.946815+0000","last_undegraded":"2024-04-20T11:02:17.946815+0000","last_fullsized":"2024-04-20T11:02:17.946815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:46:01.275989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":299,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"27'397","reported_seq":434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.483200+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T11:02:16.483200+0000","last_peered":"2024-04-20T11:02:16.483200+0000","last_clean":"2024-04-20T11:02:16.483200+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T11:02:16.483200+0000","last_undegraded":"2024-04-20T11:02:16.483200+0000","last_fullsized":"2024-04-20T11:02:16.483200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:33:43.082923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":32812,"num_write":361,"num_write_kb":30179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"27'73","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.943894+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T11:02:17.943894+0000","last_peered":"2024-04-20T11:02:17.943894+0000","last_clean":"2024-04-20T11:02:17.943894+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T11:02:17.943894+0000","last_undegraded":"2024-04-20T11:02:17.943894+0000","last_fullsized":"2024-04-20T11:02:17.943894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":17,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'403","reported_seq":446,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.441404+0000","last_change":"2024-04-20T10:56:22.904413+0000","last_active":"2024-04-20T11:02:16.441404+0000","last_peered":"2024-04-20T11:02:16.441404+0000","last_clean":"2024-04-20T11:02:16.441404+0000","last_became_active":"2024-04-20T10:56:22.904118+0000","last_became_peered":"2024-04-20T10:56:22.904118+0000","last_unstale":"2024-04-20T11:02:16.441404+0000","last_undegraded":"2024-04-20T11:02:16.441404+0000","last_fullsized":"2024-04-20T11:02:16.441404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:06.204840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":63538,"num_write":372,"num_write_kb":113421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"27'78","reported_seq":113,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:01.406304+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T11:02:01.406304+0000","last_peered":"2024-04-20T11:02:01.406304+0000","last_clean":"2024-04-20T11:02:01.406304+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T11:02:01.406304+0000","last_undegraded":"2024-04-20T11:02:01.406304+0000","last_fullsized":"2024-04-20T11:02:01.406304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":10,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"27'438","reported_seq":486,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.451206+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T11:02:16.451206+0000","last_peered":"2024-04-20T11:02:16.451206+0000","last_clean":"2024-04-20T11:02:16.451206+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T11:02:16.451206+0000","last_undegraded":"2024-04-20T11:02:16.451206+0000","last_fullsized":"2024-04-20T11:02:16.451206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":68877,"num_write":411,"num_write_kb":59157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"27'361","reported_seq":397,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.948421+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T11:02:17.948421+0000","last_peered":"2024-04-20T11:02:17.948421+0000","last_clean":"2024-04-20T11:02:17.948421+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T11:02:17.948421+0000","last_undegraded":"2024-04-20T11:02:17.948421+0000","last_fullsized":"2024-04-20T11:02:17.948421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":307,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"27'471","reported_seq":525,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.473146+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T11:02:16.473146+0000","last_peered":"2024-04-20T11:02:16.473146+0000","last_clean":"2024-04-20T11:02:16.473146+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T11:02:16.473146+0000","last_undegraded":"2024-04-20T11:02:16.473146+0000","last_fullsized":"2024-04-20T11:02:16.473146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":59233,"num_write":434,"num_write_kb":71822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'75","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.471665+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T11:02:11.471665+0000","last_peered":"2024-04-20T11:02:11.471665+0000","last_clean":"2024-04-20T11:02:11.471665+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T11:02:11.471665+0000","last_undegraded":"2024-04-20T11:02:11.471665+0000","last_fullsized":"2024-04-20T11:02:11.471665+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"27'408","reported_seq":469,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.444200+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T11:02:16.444200+0000","last_peered":"2024-04-20T11:02:16.444200+0000","last_clean":"2024-04-20T11:02:16.444200+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T11:02:16.444200+0000","last_undegraded":"2024-04-20T11:02:16.444200+0000","last_fullsized":"2024-04-20T11:02:16.444200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":108675,"num_write":383,"num_write_kb":121193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'262","reported_seq":296,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:02.418420+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T11:02:02.418420+0000","last_peered":"2024-04-20T11:02:02.418420+0000","last_clean":"2024-04-20T11:02:02.418420+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T11:02:02.418420+0000","last_undegraded":"2024-04-20T11:02:02.418420+0000","last_fullsized":"2024-04-20T11:02:02.418420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":218,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"27'384","reported_seq":425,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.464192+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T11:02:16.464192+0000","last_peered":"2024-04-20T11:02:16.464192+0000","last_clean":"2024-04-20T11:02:16.464192+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T11:02:16.464192+0000","last_undegraded":"2024-04-20T11:02:16.464192+0000","last_fullsized":"2024-04-20T11:02:16.464192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":38645,"num_write":351,"num_write_kb":38372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"27'464","reported_seq":494,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.419741+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T11:02:07.419741+0000","last_peered":"2024-04-20T11:02:07.419741+0000","last_clean":"2024-04-20T11:02:07.419741+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T11:02:07.419741+0000","last_undegraded":"2024-04-20T11:02:07.419741+0000","last_fullsized":"2024-04-20T11:02:07.419741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"27'460","reported_seq":500,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.477443+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T11:02:16.477443+0000","last_peered":"2024-04-20T11:02:16.477443+0000","last_clean":"2024-04-20T11:02:16.477443+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T11:02:16.477443+0000","last_undegraded":"2024-04-20T11:02:16.477443+0000","last_fullsized":"2024-04-20T11:02:16.477443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":42813,"num_write":427,"num_write_kb":83769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"27'70","reported_seq":105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:05.439468+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T11:02:05.439468+0000","last_peered":"2024-04-20T11:02:05.439468+0000","last_clean":"2024-04-20T11:02:05.439468+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T11:02:05.439468+0000","last_undegraded":"2024-04-20T11:02:05.439468+0000","last_fullsized":"2024-04-20T11:02:05.439468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":8,"num_write":11,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"27'397","reported_seq":428,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.454049+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T11:02:11.454049+0000","last_peered":"2024-04-20T11:02:11.454049+0000","last_clean":"2024-04-20T11:02:11.454049+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T11:02:11.454049+0000","last_undegraded":"2024-04-20T11:02:11.454049+0000","last_fullsized":"2024-04-20T11:02:11.454049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6180,"num_write":356,"num_write_kb":9278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:01:52.405499+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T11:01:52.405499+0000","last_peered":"2024-04-20T11:01:52.405499+0000","last_clean":"2024-04-20T11:01:52.405499+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T11:01:52.405499+0000","last_undegraded":"2024-04-20T11:01:52.405499+0000","last_fullsized":"2024-04-20T11:01:52.405499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"27'77","reported_seq":115,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.490479+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T11:02:16.490479+0000","last_peered":"2024-04-20T11:02:16.490479+0000","last_clean":"2024-04-20T11:02:16.490479+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T11:02:16.490479+0000","last_undegraded":"2024-04-20T11:02:16.490479+0000","last_fullsized":"2024-04-20T11:02:16.490479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":18,"num_write":19,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'352","reported_seq":387,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.461407+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T11:02:12.461407+0000","last_peered":"2024-04-20T11:02:12.461407+0000","last_clean":"2024-04-20T11:02:12.461407+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T11:02:12.461407+0000","last_undegraded":"2024-04-20T11:02:12.461407+0000","last_fullsized":"2024-04-20T11:02:12.461407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":17282,"num_write":327,"num_write_kb":42097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"27'122","reported_seq":155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:02.416875+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T11:02:02.416875+0000","last_peered":"2024-04-20T11:02:02.416875+0000","last_clean":"2024-04-20T11:02:02.416875+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T11:02:02.416875+0000","last_undegraded":"2024-04-20T11:02:02.416875+0000","last_fullsized":"2024-04-20T11:02:02.416875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":64,"num_write_kb":8241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"27'380","reported_seq":421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.453605+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T11:02:12.453605+0000","last_peered":"2024-04-20T11:02:12.453605+0000","last_clean":"2024-04-20T11:02:12.453605+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T11:02:12.453605+0000","last_undegraded":"2024-04-20T11:02:12.453605+0000","last_fullsized":"2024-04-20T11:02:12.453605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:22:54.229740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":18713,"num_write":347,"num_write_kb":51198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"27'326","reported_seq":611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:18.437066+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T11:02:18.437066+0000","last_peered":"2024-04-20T11:02:18.437066+0000","last_clean":"2024-04-20T11:02:18.437066+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T11:02:18.437066+0000","last_undegraded":"2024-04-20T11:02:18.437066+0000","last_fullsized":"2024-04-20T11:02:18.437066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462167,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":258,"num_read_kb":1597,"num_write":262,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"27'410","reported_seq":457,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.447219+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T11:02:11.447219+0000","last_peered":"2024-04-20T11:02:11.447219+0000","last_clean":"2024-04-20T11:02:11.447219+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T11:02:11.447219+0000","last_undegraded":"2024-04-20T11:02:11.447219+0000","last_fullsized":"2024-04-20T11:02:11.447219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":47086,"num_write":383,"num_write_kb":51294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"27'799","reported_seq":838,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:01:59.759462+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T11:01:59.759462+0000","last_peered":"2024-04-20T11:01:59.759462+0000","last_clean":"2024-04-20T11:01:59.759462+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T11:01:59.759462+0000","last_undegraded":"2024-04-20T11:01:59.759462+0000","last_fullsized":"2024-04-20T11:01:59.759462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":731,"num_write_kb":8599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"27'421","reported_seq":471,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.454524+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T11:02:16.454524+0000","last_peered":"2024-04-20T11:02:16.454524+0000","last_clean":"2024-04-20T11:02:16.454524+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T11:02:16.454524+0000","last_undegraded":"2024-04-20T11:02:16.454524+0000","last_fullsized":"2024-04-20T11:02:16.454524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":65250,"num_write":382,"num_write_kb":83821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"27'75","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.455107+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T11:02:16.455107+0000","last_peered":"2024-04-20T11:02:16.455107+0000","last_clean":"2024-04-20T11:02:16.455107+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T11:02:16.455107+0000","last_undegraded":"2024-04-20T11:02:16.455107+0000","last_fullsized":"2024-04-20T11:02:16.455107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"27'421","reported_seq":454,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.469611+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T11:02:16.469611+0000","last_peered":"2024-04-20T11:02:16.469611+0000","last_clean":"2024-04-20T11:02:16.469611+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T11:02:16.469611+0000","last_undegraded":"2024-04-20T11:02:16.469611+0000","last_fullsized":"2024-04-20T11:02:16.469611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":375,"num_write_kb":62820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"27'85","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.454748+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T11:02:16.454748+0000","last_peered":"2024-04-20T11:02:16.454748+0000","last_clean":"2024-04-20T11:02:16.454748+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T11:02:16.454748+0000","last_undegraded":"2024-04-20T11:02:16.454748+0000","last_fullsized":"2024-04-20T11:02:16.454748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":29,"num_write_kb":4110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"27'382","reported_seq":421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.457128+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T11:02:16.457128+0000","last_peered":"2024-04-20T11:02:16.457128+0000","last_clean":"2024-04-20T11:02:16.457128+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T11:02:16.457128+0000","last_undegraded":"2024-04-20T11:02:16.457128+0000","last_fullsized":"2024-04-20T11:02:16.457128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":25761,"num_write":337,"num_write_kb":50347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"27'345","reported_seq":382,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.957484+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T11:02:17.957484+0000","last_peered":"2024-04-20T11:02:17.957484+0000","last_clean":"2024-04-20T11:02:17.957484+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T11:02:17.957484+0000","last_undegraded":"2024-04-20T11:02:17.957484+0000","last_fullsized":"2024-04-20T11:02:17.957484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":667,"num_write_kb":1875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"27'340","reported_seq":376,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.759865+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T11:02:14.759865+0000","last_peered":"2024-04-20T11:02:14.759865+0000","last_clean":"2024-04-20T11:02:14.759865+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T11:02:14.759865+0000","last_undegraded":"2024-04-20T11:02:14.759865+0000","last_fullsized":"2024-04-20T11:02:14.759865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15872,"num_write":302,"num_write_kb":13258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"27'209","reported_seq":247,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:22.826423+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T11:02:22.826423+0000","last_peered":"2024-04-20T11:02:22.826423+0000","last_clean":"2024-04-20T11:02:22.826423+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T11:02:22.826423+0000","last_undegraded":"2024-04-20T11:02:22.826423+0000","last_fullsized":"2024-04-20T11:02:22.826423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3862695,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":147,"num_write_kb":3851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'451","reported_seq":521,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.480280+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T11:02:16.480280+0000","last_peered":"2024-04-20T11:02:16.480280+0000","last_clean":"2024-04-20T11:02:16.480280+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T11:02:16.480280+0000","last_undegraded":"2024-04-20T11:02:16.480280+0000","last_fullsized":"2024-04-20T11:02:16.480280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:59.714249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":106048,"num_write":421,"num_write_kb":76099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"27'559","reported_seq":592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.947780+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T11:02:17.947780+0000","last_peered":"2024-04-20T11:02:17.947780+0000","last_clean":"2024-04-20T11:02:17.947780+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T11:02:17.947780+0000","last_undegraded":"2024-04-20T11:02:17.947780+0000","last_fullsized":"2024-04-20T11:02:17.947780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":504,"num_write_kb":4393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'459","reported_seq":504,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.466237+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T11:02:16.466237+0000","last_peered":"2024-04-20T11:02:16.466237+0000","last_clean":"2024-04-20T11:02:16.466237+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T11:02:16.466237+0000","last_undegraded":"2024-04-20T11:02:16.466237+0000","last_fullsized":"2024-04-20T11:02:16.466237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":55402,"num_write":430,"num_write_kb":80382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'246","reported_seq":286,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.947314+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T11:02:17.947314+0000","last_peered":"2024-04-20T11:02:17.947314+0000","last_clean":"2024-04-20T11:02:17.947314+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T11:02:17.947314+0000","last_undegraded":"2024-04-20T11:02:17.947314+0000","last_fullsized":"2024-04-20T11:02:17.947314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":197,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'401","reported_seq":442,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.486930+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T11:02:11.486930+0000","last_peered":"2024-04-20T11:02:11.486930+0000","last_clean":"2024-04-20T11:02:11.486930+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T11:02:11.486930+0000","last_undegraded":"2024-04-20T11:02:11.486930+0000","last_fullsized":"2024-04-20T11:02:11.486930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":45668,"num_write":367,"num_write_kb":46153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'230","reported_seq":263,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.768794+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T11:02:14.768794+0000","last_peered":"2024-04-20T11:02:14.768794+0000","last_clean":"2024-04-20T11:02:14.768794+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T11:02:14.768794+0000","last_undegraded":"2024-04-20T11:02:14.768794+0000","last_fullsized":"2024-04-20T11:02:14.768794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":165,"num_write_kb":4183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"27'464","reported_seq":527,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:08.358362+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T11:02:08.358362+0000","last_peered":"2024-04-20T11:02:08.358362+0000","last_clean":"2024-04-20T11:02:08.358362+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T11:02:08.358362+0000","last_undegraded":"2024-04-20T11:02:08.358362+0000","last_fullsized":"2024-04-20T11:02:08.358362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":121848,"num_write":422,"num_write_kb":125716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'140","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.942782+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T11:02:17.942782+0000","last_peered":"2024-04-20T11:02:17.942782+0000","last_clean":"2024-04-20T11:02:17.942782+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T11:02:17.942782+0000","last_undegraded":"2024-04-20T11:02:17.942782+0000","last_fullsized":"2024-04-20T11:02:17.942782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":89,"num_write_kb":8247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"27'435","reported_seq":482,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.480165+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T11:02:16.480165+0000","last_peered":"2024-04-20T11:02:16.480165+0000","last_clean":"2024-04-20T11:02:16.480165+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T11:02:16.480165+0000","last_undegraded":"2024-04-20T11:02:16.480165+0000","last_fullsized":"2024-04-20T11:02:16.480165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":46429,"num_write":408,"num_write_kb":79692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"27'378","reported_seq":407,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.458430+0000","last_change":"2024-04-20T10:56:22.904797+0000","last_active":"2024-04-20T11:02:16.458430+0000","last_peered":"2024-04-20T11:02:16.458430+0000","last_clean":"2024-04-20T11:02:16.458430+0000","last_became_active":"2024-04-20T10:56:22.904460+0000","last_became_peered":"2024-04-20T10:56:22.904460+0000","last_unstale":"2024-04-20T11:02:16.458430+0000","last_undegraded":"2024-04-20T11:02:16.458430+0000","last_fullsized":"2024-04-20T11:02:16.458430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:55.531408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15562,"num_write":337,"num_write_kb":42622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'264","reported_seq":300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.428494+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T11:02:07.428494+0000","last_peered":"2024-04-20T11:02:07.428494+0000","last_clean":"2024-04-20T11:02:07.428494+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T11:02:07.428494+0000","last_undegraded":"2024-04-20T11:02:07.428494+0000","last_fullsized":"2024-04-20T11:02:07.428494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":212,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'380","reported_seq":421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.439845+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T11:02:12.439845+0000","last_peered":"2024-04-20T11:02:12.439845+0000","last_clean":"2024-04-20T11:02:12.439845+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T11:02:12.439845+0000","last_undegraded":"2024-04-20T11:02:12.439845+0000","last_fullsized":"2024-04-20T11:02:12.439845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":31020,"num_write":340,"num_write_kb":46796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"27'219","reported_seq":263,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.427078+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T11:02:07.427078+0000","last_peered":"2024-04-20T11:02:07.427078+0000","last_clean":"2024-04-20T11:02:07.427078+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T11:02:07.427078+0000","last_undegraded":"2024-04-20T11:02:07.427078+0000","last_fullsized":"2024-04-20T11:02:07.427078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":39,"num_read_kb":43,"num_write":158,"num_write_kb":12367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"27'377","reported_seq":430,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.462821+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T11:02:12.462821+0000","last_peered":"2024-04-20T11:02:12.462821+0000","last_clean":"2024-04-20T11:02:12.462821+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T11:02:12.462821+0000","last_undegraded":"2024-04-20T11:02:12.462821+0000","last_fullsized":"2024-04-20T11:02:12.462821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:17.208701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":72309,"num_write":351,"num_write_kb":75679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'88","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.474013+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T11:02:11.474013+0000","last_peered":"2024-04-20T11:02:11.474013+0000","last_clean":"2024-04-20T11:02:11.474013+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T11:02:11.474013+0000","last_undegraded":"2024-04-20T11:02:11.474013+0000","last_fullsized":"2024-04-20T11:02:11.474013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'429","reported_seq":474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.959031+0000","last_change":"2024-04-20T10:56:22.913007+0000","last_active":"2024-04-20T11:02:16.959031+0000","last_peered":"2024-04-20T11:02:16.959031+0000","last_clean":"2024-04-20T11:02:16.959031+0000","last_became_active":"2024-04-20T10:56:22.912718+0000","last_became_peered":"2024-04-20T10:56:22.912718+0000","last_unstale":"2024-04-20T11:02:16.959031+0000","last_undegraded":"2024-04-20T11:02:16.959031+0000","last_fullsized":"2024-04-20T11:02:16.959031+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:35.954727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":67508,"num_write":400,"num_write_kb":96285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1f","version":"27'70","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:08.364621+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T11:02:08.364621+0000","last_peered":"2024-04-20T11:02:08.364621+0000","last_clean":"2024-04-20T11:02:08.364621+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T11:02:08.364621+0000","last_undegraded":"2024-04-20T11:02:08.364621+0000","last_fullsized":"2024-04-20T11:02:08.364621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:20:44.332109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":25,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1110,"num_read_kb":3290197,"num_write":24364,"num_write_kb":4196200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26562,"ondisk_log_size":26562,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":152132834,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":619,"num_read_kb":1973,"num_write":6421,"num_write_kb":154889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":304513024,"data_stored":304265668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7799,"ondisk_log_size":7799,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115468,"kb_used_data":57924,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94256372,"statfs":{"total":96636764160,"available":96518524928,"internally_reserved":0,"allocated":59314176,"data_stored":59096434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574936,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80316,"kb_used_data":49140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94291524,"statfs":{"total":96636764160,"available":96554520576,"internally_reserved":0,"allocated":50319360,"data_stored":50116448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86548,"kb_used_data":29004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94285292,"statfs":{"total":96636764160,"available":96548139008,"internally_reserved":0,"allocated":29700096,"data_stored":29479029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102908,"kb_used_data":45364,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94268932,"statfs":{"total":96636764160,"available":96531386368,"internally_reserved":0,"allocated":46452736,"data_stored":46253643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74004,"kb_used_data":34508,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94297836,"statfs":{"total":96636764160,"available":96560984064,"internally_reserved":0,"allocated":35336192,"data_stored":35133324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41048,"kb_used_data":14032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330792,"statfs":{"total":96636764160,"available":96594731008,"internally_reserved":0,"allocated":14368768,"data_stored":14161398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94656,"kb_used_data":41272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94277184,"statfs":{"total":96636764160,"available":96539836416,"internally_reserved":0,"allocated":42262528,"data_stored":42059541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574937,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64900,"kb_used_data":29564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94306940,"statfs":{"total":96636764160,"available":96570306560,"internally_reserved":0,"allocated":30273536,"data_stored":30066339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29388800,"data_stored":29361002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41971712,"data_stored":41944572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14077952,"data_stored":14046429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35045376,"data_stored":35018355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46161920,"data_stored":46138674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29409280,"data_stored":29364060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50028544,"data_stored":50001479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":58429440,"data_stored":58391097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:02:27.929 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:02:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.067+0000 7fc308a5c640 1 Processor -- start 2024-04-20T11:02:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.067+0000 7fc308a5c640 1 -- start start 2024-04-20T11:02:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.068+0000 7fc308a5c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152640 0x7fc3040719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.068+0000 7fc308a5c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc3040cd6f0 0x7fc304158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.068+0000 7fc308a5c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc30415ba80 con 0x7fc304152260 2024-04-20T11:02:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.068+0000 7fc308a5c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fc30415bd60 con 0x7fc3040cd6f0 2024-04-20T11:02:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.068+0000 7fc308a5c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc30415c040 con 0x7fc304152640 2024-04-20T11:02:28.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc301d9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152640 0x7fc3040719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:28.071 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc30259c640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fc304152260 0x7fc3040c86e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:59860/0 (socket says 172.21.15.27:59860) 2024-04-20T11:02:28.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc302d9d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fc3040cd6f0 0x7fc304158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:28.072 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc30259c640 1 -- 172.21.15.27:0/199697482 learned_addr learned my addr 172.21.15.27:0/199697482 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc30159a640 1 -- 172.21.15.27:0/199697482 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206861582 0 0) 0x7fc30415ba80 con 0x7fc304152260 2024-04-20T11:02:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc30159a640 1 -- 172.21.15.27:0/199697482 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2f0003520 con 0x7fc304152260 2024-04-20T11:02:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc301d9b640 1 -- 172.21.15.27:0/199697482 >> v2:172.21.15.137:3300/0 conn(0x7fc3040cd6f0 msgr2=0x7fc304158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc301d9b640 1 --2- 172.21.15.27:0/199697482 >> v2:172.21.15.137:3300/0 conn(0x7fc3040cd6f0 0x7fc304158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc301d9b640 1 -- 172.21.15.27:0/199697482 >> v1:172.21.15.137:6789/0 conn(0x7fc304152260 legacy=0x7fc3040c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc301d9b640 1 -- 172.21.15.27:0/199697482 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30415c320 con 0x7fc304152640 2024-04-20T11:02:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.069+0000 7fc302d9d640 1 --2- 172.21.15.27:0/199697482 >> v2:172.21.15.137:3300/0 conn(0x7fc3040cd6f0 0x7fc304158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:02:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.070+0000 7fc301d9b640 1 --2- 172.21.15.27:0/199697482 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152640 0x7fc3040719c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0097a0 tx=0x7fc2ec02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270fe96f6f0e3419 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:28.073 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.070+0000 7fc30159a640 1 -- 172.21.15.27:0/199697482 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec034070 con 0x7fc304152640 2024-04-20T11:02:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.070+0000 7fc30159a640 1 -- 172.21.15.27:0/199697482 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2ec030610 con 0x7fc304152640 2024-04-20T11:02:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.070+0000 7fc30159a640 1 -- 172.21.15.27:0/199697482 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0309c0 con 0x7fc304152640 2024-04-20T11:02:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.071+0000 7fc308a5c640 1 -- 172.21.15.27:0/199697482 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152640 msgr2=0x7fc3040719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.071+0000 7fc308a5c640 1 --2- 172.21.15.27:0/199697482 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152640 0x7fc3040719c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0097a0 tx=0x7fc2ec02b960 comp rx=0 tx=0).stop 2024-04-20T11:02:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.071+0000 7fc308a5c640 1 -- 172.21.15.27:0/199697482 shutdown_connections 2024-04-20T11:02:28.074 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.071+0000 7fc308a5c640 1 --2- 172.21.15.27:0/199697482 >> v2:172.21.15.137:3300/0 conn(0x7fc3040cd6f0 0x7fc304158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:28.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.071+0000 7fc308a5c640 1 --2- 172.21.15.27:0/199697482 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152640 0x7fc3040719c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:28.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.071+0000 7fc308a5c640 1 -- 172.21.15.27:0/199697482 >> 172.21.15.27:0/199697482 conn(0x7fc3040ccbd0 msgr2=0x7fc3040ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:28.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.071+0000 7fc308a5c640 1 -- 172.21.15.27:0/199697482 shutdown_connections 2024-04-20T11:02:28.075 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.071+0000 7fc308a5c640 1 -- 172.21.15.27:0/199697482 wait complete. 2024-04-20T11:02:28.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.072+0000 7fc308a5c640 1 Processor -- start 2024-04-20T11:02:28.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.072+0000 7fc308a5c640 1 -- start start 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc308a5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3040cd6f0 0x7fc304141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc308a5c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152260 0x7fc304162de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc308a5c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc304152640 0x7fc304163320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc308a5c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc30415c600 con 0x7fc304152640 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc308a5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc304074260 con 0x7fc3040cd6f0 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc308a5c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fc30415b720 con 0x7fc304152260 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc302d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc304152640 0x7fc304163320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc302d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc304152640 0x7fc304163320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38690/0 (socket says 172.21.15.27:38690) 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc302d9d640 1 -- 172.21.15.27:0/3731836785 learned_addr learned my addr 172.21.15.27:0/3731836785 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc301d9b640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152260 0x7fc304162de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc30259c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3040cd6f0 0x7fc304141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc302d9d640 1 -- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152260 msgr2=0x7fc304162de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc302d9d640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152260 0x7fc304162de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:28.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc302d9d640 1 -- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3040cd6f0 msgr2=0x7fc304141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc302d9d640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3040cd6f0 0x7fc304141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc302d9d640 1 -- 172.21.15.27:0/3731836785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3041410b0 con 0x7fc304152640 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc301d9b640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152260 0x7fc304162de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.073+0000 7fc30259c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3040cd6f0 0x7fc304141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.074+0000 7fc302d9d640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc304152640 0x7fc304163320 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009e60 tx=0x7fc2f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.074+0000 7fc2faffd640 1 -- 172.21.15.27:0/3731836785 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4018030 con 0x7fc304152640 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.074+0000 7fc2faffd640 1 -- 172.21.15.27:0/3731836785 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f400cdf0 con 0x7fc304152640 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.074+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30414d710 con 0x7fc304152640 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.074+0000 7fc2faffd640 1 -- 172.21.15.27:0/3731836785 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4003bb0 con 0x7fc304152640 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.074+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30414dcd0 con 0x7fc304152640 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.075+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30404fcf0 con 0x7fc304152640 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.075+0000 7fc2faffd640 1 -- 172.21.15.27:0/3731836785 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2f401b070 con 0x7fc304152640 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.076+0000 7fc2faffd640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc2e406c920 0x7fc2e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.076+0000 7fc2faffd640 1 -- 172.21.15.27:0/3731836785 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fc2f408c650 con 0x7fc304152640 2024-04-20T11:02:28.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.076+0000 7fc30259c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc2e406c920 0x7fc2e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:28.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.076+0000 7fc30259c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc2e406c920 0x7fc2e406ede0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc007fd0 tx=0x7fc2fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:28.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.080+0000 7fc2faffd640 1 -- 172.21.15.27:0/3731836785 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc2f4010040 con 0x7fc304152640 2024-04-20T11:02:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.268+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc304072220 con 0x7fc304152640 2024-04-20T11:02:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.269+0000 7fc2faffd640 1 -- 172.21.15.27:0/3731836785 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+17470 (secure 0 0 0) 0x7fc2f405a2a0 con 0x7fc304152640 2024-04-20T11:02:28.269 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:28.269 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T11:01:52.359149+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6836/4195543922":"2024-04-21T11:01:52.359070+0000","172.21.15.27:6833/4195543922":"2024-04-21T11:01:52.359070+0000","172.21.15.27:6837/1680990583":"2024-04-21T11:00:42.348974+0000","172.21.15.27:6834/1680990583":"2024-04-21T11:00:42.348974+0000","172.21.15.27:6832/859496751":"2024-04-21T10:59:37.339287+0000","172.21.15.137:6835/1164978945":"2024-04-21T10:58:22.329607+0000","172.21.15.137:6834/1164978945":"2024-04-21T10:58:22.329607+0000","172.21.15.27:6835/859496751":"2024-04-21T10:59:37.339287+0000","172.21.15.137:6838/1990627502":"2024-04-21T10:57:27.321673+0000","172.21.15.137:6836/1990627502":"2024-04-21T10:57:27.321673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:02:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc2e406c920 msgr2=0x7fc2e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc2e406c920 0x7fc2e406ede0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc007fd0 tx=0x7fc2fc009040 comp rx=0 tx=0).stop 2024-04-20T11:02:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc304152640 msgr2=0x7fc304163320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc304152640 0x7fc304163320 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009e60 tx=0x7fc2f400c490 comp rx=0 tx=0).stop 2024-04-20T11:02:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 shutdown_connections 2024-04-20T11:02:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fc2e406c920 0x7fc2e406ede0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc304152640 0x7fc304163320 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc304152260 0x7fc304162de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 --2- 172.21.15.27:0/3731836785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3040cd6f0 0x7fc304141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 >> 172.21.15.27:0/3731836785 conn(0x7fc3040ccbd0 msgr2=0x7fc3040ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 shutdown_connections 2024-04-20T11:02:28.272 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:28.271+0000 7fc308a5c640 1 -- 172.21.15.27:0/3731836785 wait complete. 2024-04-20T11:02:29.283 INFO:tasks.ceph:Scrubbing osd.0 2024-04-20T11:02:29.283 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:02:29.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.402+0000 7f469284e640 1 Processor -- start 2024-04-20T11:02:29.401 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.402+0000 7f469284e640 1 -- start start 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.402+0000 7f469284e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f468c0cb940 0x7f468c166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.402+0000 7f469284e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468c0ce710 con 0x7f468c155610 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.402+0000 7f469284e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f468c0c60a0 con 0x7f468c0cd6e0 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.402+0000 7f469284e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f468c0c6380 con 0x7f468c0cb940 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f468bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f468c155610 0x7f468c06da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58652/0 (socket says 172.21.15.27:58652) 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f468bfff640 1 -- 172.21.15.27:0/845817977 learned_addr learned my addr 172.21.15.27:0/845817977 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f4690deb640 1 --2- 172.21.15.27:0/845817977 >> v2:172.21.15.137:3300/0 conn(0x7f468c0cb940 0x7f468c166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f468affd640 1 -- 172.21.15.27:0/845817977 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1296634769 0 0) 0x7f468c0ce710 con 0x7f468c155610 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f468affd640 1 -- 172.21.15.27:0/845817977 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4674003520 con 0x7f468c155610 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f468affd640 1 -- 172.21.15.27:0/845817977 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1898065579 0 0) 0x7f468c0c60a0 con 0x7f468c0cd6e0 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f468affd640 1 -- 172.21.15.27:0/845817977 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f468c0ce710 con 0x7f468c0cd6e0 2024-04-20T11:02:29.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f468affd640 1 -- 172.21.15.27:0/845817977 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3718966399 0 0) 0x7f4674003520 con 0x7f468c155610 2024-04-20T11:02:29.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f4690deb640 1 -- 172.21.15.27:0/845817977 >> v1:172.21.15.137:6789/0 conn(0x7f468c0cd6e0 legacy=0x7f468c15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f4690deb640 1 -- 172.21.15.27:0/845817977 >> v1:172.21.15.27:6789/0 conn(0x7f468c155610 legacy=0x7f468c06da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.403+0000 7f4690deb640 1 -- 172.21.15.27:0/845817977 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c0c6660 con 0x7f468c0cb940 2024-04-20T11:02:29.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.404+0000 7f4690deb640 1 --2- 172.21.15.27:0/845817977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f468c0cb940 0x7f468c166d60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4684009d90 tx=0x7f468402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=925ca23eb5cf433d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.404+0000 7f468affd640 1 -- 172.21.15.27:0/845817977 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684039070 con 0x7f468c0cb940 2024-04-20T11:02:29.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.404+0000 7f468affd640 1 -- 172.21.15.27:0/845817977 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f468402c9d0 con 0x7f468c0cb940 2024-04-20T11:02:29.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.404+0000 7f468affd640 1 -- 172.21.15.27:0/845817977 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c001670 con 0x7f468c0cb940 2024-04-20T11:02:29.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.405+0000 7f469284e640 1 -- 172.21.15.27:0/845817977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f468c0cb940 msgr2=0x7f468c166d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.405+0000 7f469284e640 1 --2- 172.21.15.27:0/845817977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f468c0cb940 0x7f468c166d60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4684009d90 tx=0x7f468402bea0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.405+0000 7f469284e640 1 -- 172.21.15.27:0/845817977 shutdown_connections 2024-04-20T11:02:29.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.405+0000 7f469284e640 1 --2- 172.21.15.27:0/845817977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f468c0cb940 0x7f468c166d60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.405+0000 7f469284e640 1 -- 172.21.15.27:0/845817977 >> 172.21.15.27:0/845817977 conn(0x7f468c06c310 msgr2=0x7f468c06cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:29.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.405+0000 7f469284e640 1 -- 172.21.15.27:0/845817977 shutdown_connections 2024-04-20T11:02:29.406 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.405+0000 7f469284e640 1 -- 172.21.15.27:0/845817977 wait complete. 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.406+0000 7f469284e640 1 Processor -- start 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.406+0000 7f469284e640 1 -- start start 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f469284e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468c0cb940 0x7f468c1f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f469284e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f468c0cd6e0 0x7f468c1f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f469284e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f468c155610 0x7f468c149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f469284e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f468c06e2e0 con 0x7f468c0cb940 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f469284e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f468c06e160 con 0x7f468c0cd6e0 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f469284e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f468c1559f0 con 0x7f468c155610 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468c0cb940 0x7f468c1f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468c0cb940 0x7f468c1f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38696/0 (socket says 172.21.15.27:38696) 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468bfff640 1 -- 172.21.15.27:0/968432875 learned_addr learned my addr 172.21.15.27:0/968432875 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f4690deb640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f468c155610 0x7f468c149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468b7fe640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f468c0cd6e0 0x7f468c1f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468bfff640 1 -- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f468c155610 msgr2=0x7f468c149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468bfff640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f468c155610 0x7f468c149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468bfff640 1 -- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f468c0cd6e0 msgr2=0x7f468c1f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468bfff640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f468c0cd6e0 0x7f468c1f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468bfff640 1 -- 172.21.15.27:0/968432875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c1f5130 con 0x7f468c0cb940 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f4690deb640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f468c155610 0x7f468c149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.407+0000 7f468b7fe640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f468c0cd6e0 0x7f468c1f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.408+0000 7f468bfff640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468c0cb940 0x7f468c1f44f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c780 tx=0x7f467c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.408+0000 7f4688ff9640 1 -- 172.21.15.27:0/968432875 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c017070 con 0x7f468c0cb940 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.408+0000 7f4688ff9640 1 -- 172.21.15.27:0/968432875 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467c005af0 con 0x7f468c0cb940 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.408+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468c151b80 con 0x7f468c0cb940 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.408+0000 7f4688ff9640 1 -- 172.21.15.27:0/968432875 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c007070 con 0x7f468c0cb940 2024-04-20T11:02:29.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.408+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468c0ca510 con 0x7f468c0cb940 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.409+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f468c0ca8f0 con 0x7f468c0cb940 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.409+0000 7f4688ff9640 1 -- 172.21.15.27:0/968432875 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f467c01a070 con 0x7f468c0cb940 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.410+0000 7f4688ff9640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f467806c8f0 0x7f467806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.410+0000 7f4688ff9640 1 -- 172.21.15.27:0/968432875 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f467c08b2d0 con 0x7f468c0cb940 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.410+0000 7f468b7fe640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f467806c8f0 0x7f467806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.410+0000 7f4688ff9640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f4678074f90 0x7f4678077410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.410+0000 7f4688ff9640 1 -- 172.21.15.27:0/968432875 --> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f467c00ae50 con 0x7f4678074f90 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.410+0000 7f4688ff9640 1 -- 172.21.15.27:0/968432875 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f467c092050 con 0x7f468c0cb940 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.410+0000 7f4690deb640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f4678074f90 0x7f4678077410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.411+0000 7f468b7fe640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f467806c8f0 0x7f467806edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4680007fd0 tx=0x7f4680009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.411+0000 7f4690deb640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f4678074f90 0x7f4678077410 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.411 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.411+0000 7f4688ff9640 1 -- 172.21.15.27:0/968432875 <== osd.0 v2:172.21.15.27:6808/2513808316 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f467c00ae50 con 0x7f4678074f90 2024-04-20T11:02:29.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 --> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f468c0cab60 con 0x7f4678074f90 2024-04-20T11:02:29.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f4688ff9640 1 -- 172.21.15.27:0/968432875 <== osd.0 v2:172.21.15.27:6808/2513808316 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f468c0cab60 con 0x7f4678074f90 2024-04-20T11:02:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T11:02:29.427 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:02:29.427 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f4678074f90 msgr2=0x7f4678077410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f469284e640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.27:6808/2513808316,v1:172.21.15.27:6809/2513808316] conn(0x7f4678074f90 0x7f4678077410 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f467806c8f0 msgr2=0x7f467806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f469284e640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f467806c8f0 0x7f467806edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4680007fd0 tx=0x7f4680009040 comp rx=0 tx=0).stop 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468c0cb940 msgr2=0x7f468c1f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f469284e640 1 --2- 172.21.15.27:0/968432875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f468c0cb940 0x7f468c1f44f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c780 tx=0x7f467c00cc40 comp rx=0 tx=0).stop 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f4690deb640 1 -- 172.21.15.27:0/968432875 reap_dead start 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 shutdown_connections 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.425+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 >> 172.21.15.27:0/968432875 conn(0x7f468c06c310 msgr2=0x7f468c0b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.426+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 shutdown_connections 2024-04-20T11:02:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.426+0000 7f469284e640 1 -- 172.21.15.27:0/968432875 wait complete. 2024-04-20T11:02:29.436 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-20T11:02:29.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.538+0000 7fdafab3d640 1 Processor -- start 2024-04-20T11:02:29.538 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.539+0000 7fdafab3d640 1 -- start start 2024-04-20T11:02:29.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.539+0000 7fdafab3d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf4155950 0x7fdaf40cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.539+0000 7fdafab3d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdaf40cc330 0x7fdaf40b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.539+0000 7fdafab3d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf40b5ea0 con 0x7fdaf40b5a30 2024-04-20T11:02:29.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.539+0000 7fdafab3d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf415b9c0 con 0x7fdaf40cc330 2024-04-20T11:02:29.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.539+0000 7fdafab3d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf415bca0 con 0x7fdaf4155950 2024-04-20T11:02:29.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdaf40cc330 0x7fdaf40b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdaf40cc330 0x7fdaf40b54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38710/0 (socket says 172.21.15.27:38710) 2024-04-20T11:02:29.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf3fff640 1 -- 172.21.15.27:0/236231720 learned_addr learned my addr 172.21.15.27:0/236231720 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:29.541 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf90da640 1 --1- 172.21.15.27:0/236231720 >> v1:172.21.15.27:6789/0 conn(0x7fdaf40b5a30 0x7fdaf40d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58666/0 (socket says 172.21.15.27:58666) 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf88d9640 1 --2- 172.21.15.27:0/236231720 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf4155950 0x7fdaf40cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf3fff640 1 -- 172.21.15.27:0/236231720 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf4155950 msgr2=0x7fdaf40cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/236231720 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf4155950 0x7fdaf40cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf3fff640 1 -- 172.21.15.27:0/236231720 >> v1:172.21.15.27:6789/0 conn(0x7fdaf40b5a30 legacy=0x7fdaf40d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf3fff640 1 -- 172.21.15.27:0/236231720 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf415bf80 con 0x7fdaf40cc330 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf37fe640 1 -- 172.21.15.27:0/236231720 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1248653188 0 0) 0x7fdaf40b5ea0 con 0x7fdaf40b5a30 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.540+0000 7fdaf88d9640 1 --2- 172.21.15.27:0/236231720 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf4155950 0x7fdaf40cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/236231720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 0x7fdaf40b54f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009870 tx=0x7fdae002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582c78f03ebf6810 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdaf37fe640 1 -- 172.21.15.27:0/236231720 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae002f840 con 0x7fdaf40cc330 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdaf37fe640 1 -- 172.21.15.27:0/236231720 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae002f9e0 con 0x7fdaf40cc330 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdaf37fe640 1 -- 172.21.15.27:0/236231720 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae002fdb0 con 0x7fdaf40cc330 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdafab3d640 1 -- 172.21.15.27:0/236231720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 msgr2=0x7fdaf40b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdafab3d640 1 --2- 172.21.15.27:0/236231720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 0x7fdaf40b54f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009870 tx=0x7fdae002ba20 comp rx=0 tx=0).stop 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdafab3d640 1 -- 172.21.15.27:0/236231720 shutdown_connections 2024-04-20T11:02:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdafab3d640 1 --2- 172.21.15.27:0/236231720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 0x7fdaf40b54f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdafab3d640 1 --2- 172.21.15.27:0/236231720 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf4155950 0x7fdaf40cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdafab3d640 1 -- 172.21.15.27:0/236231720 >> 172.21.15.27:0/236231720 conn(0x7fdaf40676c0 msgr2=0x7fdaf4067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:29.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdafab3d640 1 -- 172.21.15.27:0/236231720 shutdown_connections 2024-04-20T11:02:29.543 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.541+0000 7fdafab3d640 1 -- 172.21.15.27:0/236231720 wait complete. 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.543+0000 7fdafab3d640 1 Processor -- start 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.543+0000 7fdafab3d640 1 -- start start 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.543+0000 7fdafab3d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf40b5a30 0x7fdaf40b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.543+0000 7fdafab3d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 0x7fdaf414fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.543+0000 7fdafab3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaf4155950 0x7fdaf4154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.543+0000 7fdafab3d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf415fb90 con 0x7fdaf40cc330 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.543+0000 7fdafab3d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf415c540 con 0x7fdaf4155950 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.543+0000 7fdafab3d640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf415fd10 con 0x7fdaf40b5a30 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 0x7fdaf414fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 0x7fdaf414fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38718/0 (socket says 172.21.15.27:38718) 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf3fff640 1 -- 172.21.15.27:0/2436624691 learned_addr learned my addr 172.21.15.27:0/2436624691 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:29.545 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf90da640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaf4155950 0x7fdaf4154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf88d9640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf40b5a30 0x7fdaf40b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf3fff640 1 -- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf40b5a30 msgr2=0x7fdaf40b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf40b5a30 0x7fdaf40b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf3fff640 1 -- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaf4155950 msgr2=0x7fdaf4154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaf4155950 0x7fdaf4154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf3fff640 1 -- 172.21.15.27:0/2436624691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf4150ff0 con 0x7fdaf40cc330 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf3fff640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 0x7fdaf414fb80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdae002ba60 tx=0x7fdae0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf17fa640 1 -- 172.21.15.27:0/2436624691 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0047070 con 0x7fdaf40cc330 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf17fa640 1 -- 172.21.15.27:0/2436624691 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae0042480 con 0x7fdaf40cc330 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf41512d0 con 0x7fdaf40cc330 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdaf17fa640 1 -- 172.21.15.27:0/2436624691 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0042870 con 0x7fdaf40cc330 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.544+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf414d390 con 0x7fdaf40cc330 2024-04-20T11:02:29.546 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.545+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf415ecc0 con 0x7fdaf40cc330 2024-04-20T11:02:29.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.546+0000 7fdaf17fa640 1 -- 172.21.15.27:0/2436624691 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdae004a070 con 0x7fdaf40cc330 2024-04-20T11:02:29.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.546+0000 7fdaf17fa640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdad806c8f0 0x7fdad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.546+0000 7fdaf17fa640 1 -- 172.21.15.27:0/2436624691 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fdae00adfb0 con 0x7fdaf40cc330 2024-04-20T11:02:29.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.546+0000 7fdaf88d9640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdad806c8f0 0x7fdad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.547 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.547+0000 7fdaf88d9640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdad806c8f0 0x7fdad806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdae800a990 tx=0x7fdae8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.550 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.550+0000 7fdaf17fa640 1 -- 172.21.15.27:0/2436624691 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdaf415ecc0 con 0x7fdaf40cc330 2024-04-20T11:02:29.735 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.735+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fdaf404fcf0 con 0x7fdad806c8f0 2024-04-20T11:02:29.736 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.736+0000 7fdaf17fa640 1 -- 172.21.15.27:0/2436624691 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdaf404fcf0 con 0x7fdad806c8f0 2024-04-20T11:02:29.736 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 0 to deep-scrub 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.738+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdad806c8f0 msgr2=0x7fdad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdad806c8f0 0x7fdad806edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdae800a990 tx=0x7fdae8008040 comp rx=0 tx=0).stop 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 msgr2=0x7fdaf414fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 0x7fdaf414fb80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdae002ba60 tx=0x7fdae0004730 comp rx=0 tx=0).stop 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 shutdown_connections 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdad806c8f0 0x7fdad806edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaf4155950 0x7fdaf4154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaf40cc330 0x7fdaf414fb80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 --2- 172.21.15.27:0/2436624691 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdaf40b5a30 0x7fdaf40b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 >> 172.21.15.27:0/2436624691 conn(0x7fdaf40676c0 msgr2=0x7fdaf40d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 shutdown_connections 2024-04-20T11:02:29.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.739+0000 7fdafab3d640 1 -- 172.21.15.27:0/2436624691 wait complete. 2024-04-20T11:02:29.749 INFO:tasks.ceph:Scrubbing osd.1 2024-04-20T11:02:29.750 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:02:29.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.858+0000 7f6925031640 1 Processor -- start 2024-04-20T11:02:29.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.859+0000 7f6925031640 1 -- start start 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.859+0000 7f6925031640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c8600 0x7f6920158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.859+0000 7f6925031640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f69200c89e0 0x7f6920151f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.859+0000 7f6925031640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69200c9120 con 0x7f69200cd370 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.859+0000 7f6925031640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f692006bad0 con 0x7f69200c89e0 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.859+0000 7f6925031640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f692006bdb0 con 0x7f69200c8600 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f691ed9d640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c8600 0x7f6920158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f691f59e640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f69200cd370 0x7f69200c5480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:59878/0 (socket says 172.21.15.27:59878) 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f6917fff640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f69200c89e0 0x7f6920151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f691f59e640 1 -- 172.21.15.27:0/2628208182 learned_addr learned my addr 172.21.15.27:0/2628208182 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:29.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f691e59c640 1 -- 172.21.15.27:0/2628208182 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261045543 0 0) 0x7f69200c9120 con 0x7f69200cd370 2024-04-20T11:02:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f691e59c640 1 -- 172.21.15.27:0/2628208182 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f690c003520 con 0x7f69200cd370 2024-04-20T11:02:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f691ed9d640 1 -- 172.21.15.27:0/2628208182 >> v2:172.21.15.137:3300/0 conn(0x7f69200c89e0 msgr2=0x7f6920151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f691ed9d640 1 --2- 172.21.15.27:0/2628208182 >> v2:172.21.15.137:3300/0 conn(0x7f69200c89e0 0x7f6920151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f691ed9d640 1 -- 172.21.15.27:0/2628208182 >> v1:172.21.15.137:6789/0 conn(0x7f69200cd370 legacy=0x7f69200c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f691ed9d640 1 -- 172.21.15.27:0/2628208182 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692006c090 con 0x7f69200c8600 2024-04-20T11:02:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.860+0000 7f6917fff640 1 --2- 172.21.15.27:0/2628208182 >> v2:172.21.15.137:3300/0 conn(0x7f69200c89e0 0x7f6920151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:02:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.861+0000 7f691ed9d640 1 --2- 172.21.15.27:0/2628208182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c8600 0x7f6920158b60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69100097a0 tx=0x7f691002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb245a9fb2e5e5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.862 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.861+0000 7f691e59c640 1 -- 172.21.15.27:0/2628208182 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910034070 con 0x7f69200c8600 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.861+0000 7f691e59c640 1 -- 172.21.15.27:0/2628208182 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6910030610 con 0x7f69200c8600 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.861+0000 7f691e59c640 1 -- 172.21.15.27:0/2628208182 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69100309c0 con 0x7f69200c8600 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.862+0000 7f6925031640 1 -- 172.21.15.27:0/2628208182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c8600 msgr2=0x7f6920158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.862+0000 7f6925031640 1 --2- 172.21.15.27:0/2628208182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c8600 0x7f6920158b60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69100097a0 tx=0x7f691002b960 comp rx=0 tx=0).stop 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.862+0000 7f6925031640 1 -- 172.21.15.27:0/2628208182 shutdown_connections 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.862+0000 7f6925031640 1 --2- 172.21.15.27:0/2628208182 >> v2:172.21.15.137:3300/0 conn(0x7f69200c89e0 0x7f6920151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.862+0000 7f6925031640 1 --2- 172.21.15.27:0/2628208182 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c8600 0x7f6920158b60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.862+0000 7f6925031640 1 -- 172.21.15.27:0/2628208182 >> 172.21.15.27:0/2628208182 conn(0x7f69200cc880 msgr2=0x7f69200ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.862+0000 7f6925031640 1 -- 172.21.15.27:0/2628208182 shutdown_connections 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.862+0000 7f6925031640 1 -- 172.21.15.27:0/2628208182 wait complete. 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.863+0000 7f6925031640 1 Processor -- start 2024-04-20T11:02:29.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.863+0000 7f6925031640 1 -- start start 2024-04-20T11:02:29.865 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f6925031640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69200c8600 0x7f6920157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f6925031640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c89e0 0x7f6920158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f6925031640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69200cd370 0x7f6920071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f6925031640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6920070e50 con 0x7f69200c8600 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f6925031640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f692006c650 con 0x7f69200cd370 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f6925031640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f6920070fd0 con 0x7f69200c89e0 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f691ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69200c8600 0x7f6920157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f691ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69200c8600 0x7f6920157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38724/0 (socket says 172.21.15.27:38724) 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f691ed9d640 1 -- 172.21.15.27:0/1868091602 learned_addr learned my addr 172.21.15.27:0/1868091602 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f6917fff640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c89e0 0x7f6920158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.866 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f691f59e640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69200cd370 0x7f6920071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f691ed9d640 1 -- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c89e0 msgr2=0x7f6920158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f691ed9d640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f69200c89e0 0x7f6920158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f691ed9d640 1 -- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69200cd370 msgr2=0x7f6920071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f691ed9d640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69200cd370 0x7f6920071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.864+0000 7f691ed9d640 1 -- 172.21.15.27:0/1868091602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6920072f90 con 0x7f69200c8600 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.865+0000 7f691ed9d640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69200c8600 0x7f6920157bd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f691002bf70 tx=0x7f69100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.865+0000 7f69177fe640 1 -- 172.21.15.27:0/1868091602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910034040 con 0x7f69200c8600 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.865+0000 7f69177fe640 1 -- 172.21.15.27:0/1868091602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691003aaa0 con 0x7f69200c8600 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.865+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6920073230 con 0x7f69200c8600 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.865+0000 7f69177fe640 1 -- 172.21.15.27:0/1868091602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910006e10 con 0x7f69200c8600 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.865+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692014ec50 con 0x7f69200c8600 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.866+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f692014ee90 con 0x7f69200c8600 2024-04-20T11:02:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.866+0000 7f69177fe640 1 -- 172.21.15.27:0/1868091602 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6910037070 con 0x7f69200c8600 2024-04-20T11:02:29.868 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.867+0000 7f69177fe640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f690006c920 0x7f690006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.867+0000 7f69177fe640 1 -- 172.21.15.27:0/1868091602 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f69100ad900 con 0x7f69200c8600 2024-04-20T11:02:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.867+0000 7f69177fe640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7f6900074fd0 0x7f6900077450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.867+0000 7f69177fe640 1 -- 172.21.15.27:0/1868091602 --> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6910004490 con 0x7f6900074fd0 2024-04-20T11:02:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.867+0000 7f6917fff640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f690006c920 0x7f690006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.867+0000 7f69177fe640 1 -- 172.21.15.27:0/1868091602 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69100b22c0 con 0x7f69200c8600 2024-04-20T11:02:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.867+0000 7f691f59e640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7f6900074fd0 0x7f6900077450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.867+0000 7f691f59e640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7f6900074fd0 0x7f6900077450 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.867+0000 7f6917fff640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f690006c920 0x7f690006ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6908009560 tx=0x7f6908009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.868+0000 7f69177fe640 1 -- 172.21.15.27:0/1868091602 <== osd.1 v2:172.21.15.27:6800/600824511 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6910004490 con 0x7f6900074fd0 2024-04-20T11:02:29.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.881+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 --> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f692004fcf0 con 0x7f6900074fd0 2024-04-20T11:02:29.881 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.881+0000 7f69177fe640 1 -- 172.21.15.27:0/1868091602 <== osd.1 v2:172.21.15.27:6800/600824511 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f692004fcf0 con 0x7f6900074fd0 2024-04-20T11:02:29.882 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T11:02:29.882 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:02:29.882 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7f6900074fd0 msgr2=0x7f6900077450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.27:6800/600824511,v1:172.21.15.27:6801/600824511] conn(0x7f6900074fd0 0x7f6900077450 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f690006c920 msgr2=0x7f690006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f690006c920 0x7f690006ede0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6908009560 tx=0x7f6908009190 comp rx=0 tx=0).stop 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69200c8600 msgr2=0x7f6920157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 --2- 172.21.15.27:0/1868091602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69200c8600 0x7f6920157bd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f691002bf70 tx=0x7f69100046c0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f691f59e640 1 -- 172.21.15.27:0/1868091602 reap_dead start 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 shutdown_connections 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 >> 172.21.15.27:0/1868091602 conn(0x7f69200cc880 msgr2=0x7f69200c53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 shutdown_connections 2024-04-20T11:02:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.882+0000 7f6925031640 1 -- 172.21.15.27:0/1868091602 wait complete. 2024-04-20T11:02:29.892 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-20T11:02:29.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.996+0000 7f175e115640 1 Processor -- start 2024-04-20T11:02:29.996 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.996+0000 7f175e115640 1 -- start start 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f175e115640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f1758155950 0x7f17580cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f175e115640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f17580cc330 0x7f17580b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f175e115640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f175815b9c0 con 0x7f17580b5a30 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f175e115640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f175815bca0 con 0x7f17580cc330 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f175e115640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f175815bf80 con 0x7f1758155950 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f1756ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f17580cc330 0x7f17580b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f1757fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f17580b5a30 0x7f17580d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58672/0 (socket says 172.21.15.27:58672) 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f1757fff640 1 -- 172.21.15.27:0/3717177480 learned_addr learned my addr 172.21.15.27:0/3717177480 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f17577fe640 1 --2- 172.21.15.27:0/3717177480 >> v2:172.21.15.137:3300/0 conn(0x7f1758155950 0x7f17580cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f17567fc640 1 -- 172.21.15.27:0/3717177480 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 236104042 0 0) 0x7f175815b9c0 con 0x7f17580b5a30 2024-04-20T11:02:29.998 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f17567fc640 1 -- 172.21.15.27:0/3717177480 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1740003520 con 0x7f17580b5a30 2024-04-20T11:02:29.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f1756ffd640 1 -- 172.21.15.27:0/3717177480 >> v2:172.21.15.137:3300/0 conn(0x7f1758155950 msgr2=0x7f17580cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:29.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f1756ffd640 1 --2- 172.21.15.27:0/3717177480 >> v2:172.21.15.137:3300/0 conn(0x7f1758155950 0x7f17580cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:29.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f1756ffd640 1 -- 172.21.15.27:0/3717177480 >> v1:172.21.15.27:6789/0 conn(0x7f17580b5a30 legacy=0x7f17580d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:29.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.997+0000 7f1756ffd640 1 -- 172.21.15.27:0/3717177480 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175815c260 con 0x7f17580cc330 2024-04-20T11:02:29.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.998+0000 7f1756ffd640 1 --2- 172.21.15.27:0/3717177480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 0x7f17580b54f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f17580ce7c0 tx=0x7f174c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b0c08d21a38e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:29.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.998+0000 7f17567fc640 1 -- 172.21.15.27:0/3717177480 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c034070 con 0x7f17580cc330 2024-04-20T11:02:29.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.998+0000 7f17567fc640 1 -- 172.21.15.27:0/3717177480 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f174c030610 con 0x7f17580cc330 2024-04-20T11:02:29.999 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.998+0000 7f17567fc640 1 -- 172.21.15.27:0/3717177480 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0309e0 con 0x7f17580cc330 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.999+0000 7f175e115640 1 -- 172.21.15.27:0/3717177480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 msgr2=0x7f17580b54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.999+0000 7f175e115640 1 --2- 172.21.15.27:0/3717177480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 0x7f17580b54f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f17580ce7c0 tx=0x7f174c02b960 comp rx=0 tx=0).stop 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.999+0000 7f175e115640 1 -- 172.21.15.27:0/3717177480 shutdown_connections 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.999+0000 7f175e115640 1 --2- 172.21.15.27:0/3717177480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 0x7f17580b54f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.999+0000 7f175e115640 1 --2- 172.21.15.27:0/3717177480 >> v2:172.21.15.137:3300/0 conn(0x7f1758155950 0x7f17580cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.999+0000 7f175e115640 1 -- 172.21.15.27:0/3717177480 >> 172.21.15.27:0/3717177480 conn(0x7f17580676c0 msgr2=0x7f1758067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.999+0000 7f175e115640 1 -- 172.21.15.27:0/3717177480 shutdown_connections 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:29.999+0000 7f175e115640 1 -- 172.21.15.27:0/3717177480 wait complete. 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.000+0000 7f175e115640 1 Processor -- start 2024-04-20T11:02:30.002 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.000+0000 7f175e115640 1 -- start start 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f175e115640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17580b5a30 0x7f17581f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f175e115640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 0x7f175814db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f175e115640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1758155950 0x7f17580cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f175e115640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f175815fb90 con 0x7f17580cc330 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f175e115640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f17580b5e40 con 0x7f1758155950 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f175e115640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f175815fd10 con 0x7f17580b5a30 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1756ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 0x7f175814db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1756ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 0x7f175814db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38734/0 (socket says 172.21.15.27:38734) 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1756ffd640 1 -- 172.21.15.27:0/305553488 learned_addr learned my addr 172.21.15.27:0/305553488 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1757fff640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1758155950 0x7f17580cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f17577fe640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17580b5a30 0x7f17581f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.003 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1756ffd640 1 -- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17580b5a30 msgr2=0x7f17581f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1756ffd640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17580b5a30 0x7f17581f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1756ffd640 1 -- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1758155950 msgr2=0x7f17580cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1756ffd640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1758155950 0x7f17580cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1756ffd640 1 -- 172.21.15.27:0/305553488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17581f1c80 con 0x7f17580cc330 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f1756ffd640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 0x7f175814db40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f174c02bf70 tx=0x7f174c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f173ffff640 1 -- 172.21.15.27:0/305553488 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c034040 con 0x7f17580cc330 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.001+0000 7f173ffff640 1 -- 172.21.15.27:0/305553488 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f174c030410 con 0x7f17580cc330 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.002+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17581f1f60 con 0x7f17580cc330 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.002+0000 7f173ffff640 1 -- 172.21.15.27:0/305553488 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c006e10 con 0x7f17580cc330 2024-04-20T11:02:30.004 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.002+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175815ca10 con 0x7f17580cc330 2024-04-20T11:02:30.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.003+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f175804fcf0 con 0x7f17580cc330 2024-04-20T11:02:30.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.003+0000 7f173ffff640 1 -- 172.21.15.27:0/305553488 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f174c037070 con 0x7f17580cc330 2024-04-20T11:02:30.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.004+0000 7f173ffff640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f174406c8f0 0x7f174406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.004+0000 7f173ffff640 1 -- 172.21.15.27:0/305553488 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f174c0ad6d0 con 0x7f17580cc330 2024-04-20T11:02:30.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.004+0000 7f17577fe640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f174406c8f0 0x7f174406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.005 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.004+0000 7f17577fe640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f174406c8f0 0x7f174406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1748009a30 tx=0x7f1748008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.008 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.008+0000 7f173ffff640 1 -- 172.21.15.27:0/305553488 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f174c07b970 con 0x7f17580cc330 2024-04-20T11:02:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.195+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f17580b4030 con 0x7f174406c8f0 2024-04-20T11:02:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.196+0000 7f173ffff640 1 -- 172.21.15.27:0/305553488 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f17580b4030 con 0x7f174406c8f0 2024-04-20T11:02:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 1 to deep-scrub 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f174406c8f0 msgr2=0x7f174406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f174406c8f0 0x7f174406edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1748009a30 tx=0x7f1748008040 comp rx=0 tx=0).stop 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 msgr2=0x7f175814db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 0x7f175814db40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f174c02bf70 tx=0x7f174c0046c0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 shutdown_connections 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f174406c8f0 0x7f174406edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1758155950 0x7f17580cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17580cc330 0x7f175814db40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 --2- 172.21.15.27:0/305553488 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f17580b5a30 0x7f17581f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 >> 172.21.15.27:0/305553488 conn(0x7f17580676c0 msgr2=0x7f1758067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 shutdown_connections 2024-04-20T11:02:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.198+0000 7f175e115640 1 -- 172.21.15.27:0/305553488 wait complete. 2024-04-20T11:02:30.208 INFO:tasks.ceph:Scrubbing osd.2 2024-04-20T11:02:30.209 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:02:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.313+0000 7f5a32b75640 1 Processor -- start 2024-04-20T11:02:30.313 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.313+0000 7f5a32b75640 1 -- start start 2024-04-20T11:02:30.314 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.314+0000 7f5a32b75640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a2c0cd6e0 0x7f5a2c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.314+0000 7f5a32b75640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2c0ce710 con 0x7f5a2c0cb940 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.314+0000 7f5a32b75640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2c0c60a0 con 0x7f5a2c155610 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.314+0000 7f5a32b75640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f5a2c0c6380 con 0x7f5a2c0cd6e0 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.314+0000 7f5a2bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a2c0cd6e0 0x7f5a2c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.314+0000 7f5a2bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5a2c0cd6e0 0x7f5a2c15b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38744/0 (socket says 172.21.15.27:38744) 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.314+0000 7f5a2bfff640 1 -- 172.21.15.27:0/2086084879 learned_addr learned my addr 172.21.15.27:0/2086084879 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.314+0000 7f5a31112640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5a2c0cb940 0x7f5a2c165d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58674/0 (socket says 172.21.15.27:58674) 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.315+0000 7f5a2bfff640 1 -- 172.21.15.27:0/2086084879 >> v1:172.21.15.137:6789/0 conn(0x7f5a2c155610 legacy=0x7f5a2c06da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.315+0000 7f5a2bfff640 1 -- 172.21.15.27:0/2086084879 >> v1:172.21.15.27:6789/0 conn(0x7f5a2c0cb940 legacy=0x7f5a2c165d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.315+0000 7f5a2b7fe640 1 -- 172.21.15.27:0/2086084879 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2636981182 0 0) 0x7f5a2c0ce710 con 0x7f5a2c0cb940 2024-04-20T11:02:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.315+0000 7f5a2bfff640 1 -- 172.21.15.27:0/2086084879 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c0c6660 con 0x7f5a2c0cd6e0 2024-04-20T11:02:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.315+0000 7f5a2bfff640 1 --2- 172.21.15.27:0/2086084879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c0cd6e0 0x7f5a2c15b410 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009850 tx=0x7f5a2002baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc315d42e92dcc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a2b7fe640 1 -- 172.21.15.27:0/2086084879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20004230 con 0x7f5a2c0cd6e0 2024-04-20T11:02:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a2b7fe640 1 -- 172.21.15.27:0/2086084879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a200043d0 con 0x7f5a2c0cd6e0 2024-04-20T11:02:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a2b7fe640 1 -- 172.21.15.27:0/2086084879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2002d860 con 0x7f5a2c0cd6e0 2024-04-20T11:02:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a32b75640 1 -- 172.21.15.27:0/2086084879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c0cd6e0 msgr2=0x7f5a2c15b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a32b75640 1 --2- 172.21.15.27:0/2086084879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c0cd6e0 0x7f5a2c15b410 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009850 tx=0x7f5a2002baa0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a32b75640 1 -- 172.21.15.27:0/2086084879 shutdown_connections 2024-04-20T11:02:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a32b75640 1 --2- 172.21.15.27:0/2086084879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c0cd6e0 0x7f5a2c15b410 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a32b75640 1 -- 172.21.15.27:0/2086084879 >> 172.21.15.27:0/2086084879 conn(0x7f5a2c06c310 msgr2=0x7f5a2c06cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a32b75640 1 -- 172.21.15.27:0/2086084879 shutdown_connections 2024-04-20T11:02:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.316+0000 7f5a32b75640 1 -- 172.21.15.27:0/2086084879 wait complete. 2024-04-20T11:02:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.317+0000 7f5a32b75640 1 Processor -- start 2024-04-20T11:02:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.317+0000 7f5a32b75640 1 -- start start 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a32b75640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0cb940 0x7f5a2c1f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a32b75640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a2c0cd6e0 0x7f5a2c149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a32b75640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c155610 0x7f5a2c151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a32b75640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c06e2e0 con 0x7f5a2c155610 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a32b75640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c06e160 con 0x7f5a2c0cb940 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a32b75640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2c1559f0 con 0x7f5a2c0cd6e0 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a31112640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c155610 0x7f5a2c151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a31112640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c155610 0x7f5a2c151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38760/0 (socket says 172.21.15.27:38760) 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a31112640 1 -- 172.21.15.27:0/3413103079 learned_addr learned my addr 172.21.15.27:0/3413103079 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:30.319 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a30911640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0cb940 0x7f5a2c1f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.318+0000 7f5a2bfff640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a2c0cd6e0 0x7f5a2c149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a31112640 1 -- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a2c0cd6e0 msgr2=0x7f5a2c149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a31112640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5a2c0cd6e0 0x7f5a2c149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a31112640 1 -- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0cb940 msgr2=0x7f5a2c1f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a31112640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0cb940 0x7f5a2c1f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a31112640 1 -- 172.21.15.27:0/3413103079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c14a740 con 0x7f5a2c155610 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a31112640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c155610 0x7f5a2c151c90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c410 tx=0x7f5a1c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a297fa640 1 -- 172.21.15.27:0/3413103079 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0074e0 con 0x7f5a2c155610 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a297fa640 1 -- 172.21.15.27:0/3413103079 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a1c0047e0 con 0x7f5a2c155610 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a297fa640 1 -- 172.21.15.27:0/3413103079 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c004bd0 con 0x7f5a2c155610 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2c1f12c0 con 0x7f5a2c155610 2024-04-20T11:02:30.320 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.319+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2c1f1880 con 0x7f5a2c155610 2024-04-20T11:02:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.320+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a2c154320 con 0x7f5a2c155610 2024-04-20T11:02:30.321 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.320+0000 7f5a297fa640 1 -- 172.21.15.27:0/3413103079 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a1c011070 con 0x7f5a2c155610 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.321+0000 7f5a297fa640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a1806c920 0x7f5a1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.321+0000 7f5a297fa640 1 -- 172.21.15.27:0/3413103079 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f5a1c08b8c0 con 0x7f5a2c155610 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.321+0000 7f5a297fa640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f5a18074fc0 0x7f5a18077440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.321+0000 7f5a297fa640 1 -- 172.21.15.27:0/3413103079 --> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a18077b10 con 0x7f5a18074fc0 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.321+0000 7f5a297fa640 1 -- 172.21.15.27:0/3413103079 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c0902c0 con 0x7f5a2c155610 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.321+0000 7f5a30911640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a1806c920 0x7f5a1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.321+0000 7f5a2bfff640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f5a18074fc0 0x7f5a18077440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.322+0000 7f5a2bfff640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f5a18074fc0 0x7f5a18077440 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.322+0000 7f5a30911640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a1806c920 0x7f5a1806ede0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a240012e0 tx=0x7f5a24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.322 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.322+0000 7f5a297fa640 1 -- 172.21.15.27:0/3413103079 <== osd.2 v2:172.21.15.27:6816/3544986990 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a18077b10 con 0x7f5a18074fc0 2024-04-20T11:02:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.335+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 --> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5a2c152420 con 0x7f5a18074fc0 2024-04-20T11:02:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a297fa640 1 -- 172.21.15.27:0/3413103079 <== osd.2 v2:172.21.15.27:6816/3544986990 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5a2c152420 con 0x7f5a18074fc0 2024-04-20T11:02:30.335 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T11:02:30.335 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:02:30.335 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T11:02:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f5a18074fc0 msgr2=0x7f5a18077440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.27:6816/3544986990,v1:172.21.15.27:6817/3544986990] conn(0x7f5a18074fc0 0x7f5a18077440 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a1806c920 msgr2=0x7f5a1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f5a1806c920 0x7f5a1806ede0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a240012e0 tx=0x7f5a24009040 comp rx=0 tx=0).stop 2024-04-20T11:02:30.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c155610 msgr2=0x7f5a2c151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 --2- 172.21.15.27:0/3413103079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a2c155610 0x7f5a2c151c90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00c410 tx=0x7f5a1c00c8d0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a31112640 1 -- 172.21.15.27:0/3413103079 reap_dead start 2024-04-20T11:02:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 shutdown_connections 2024-04-20T11:02:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 >> 172.21.15.27:0/3413103079 conn(0x7f5a2c06c310 msgr2=0x7f5a2c1e76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 shutdown_connections 2024-04-20T11:02:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.336+0000 7f5a32b75640 1 -- 172.21.15.27:0/3413103079 wait complete. 2024-04-20T11:02:30.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-20T11:02:30.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.448+0000 7fec16eba640 1 Processor -- start 2024-04-20T11:02:30.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.448+0000 7fec16eba640 1 -- start start 2024-04-20T11:02:30.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec16eba640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec1015ed10 0x7fec10073970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec16eba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fec10072130 0x7fec100c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.452 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec16eba640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fec100c8d30 0x7fec1015f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec16eba640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fec100c3bc0 con 0x7fec10072130 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec16eba640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fec100c4050 con 0x7fec100c8d30 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec16eba640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fec10073fc0 con 0x7fec1015ed10 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec0ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fec10072130 0x7fec100c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec0ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fec10072130 0x7fec100c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38776/0 (socket says 172.21.15.27:38776) 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec0ffff640 1 -- 172.21.15.27:0/3461697545 learned_addr learned my addr 172.21.15.27:0/3461697545 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec14c56640 1 --2- 172.21.15.27:0/3461697545 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec1015ed10 0x7fec10073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec15457640 1 --2- 172.21.15.27:0/3461697545 >> v2:172.21.15.137:3300/0 conn(0x7fec100c8d30 0x7fec1015f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec0ffff640 1 -- 172.21.15.27:0/3461697545 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec1015ed10 msgr2=0x7fec10073970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec0ffff640 1 --2- 172.21.15.27:0/3461697545 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec1015ed10 0x7fec10073970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec0ffff640 1 -- 172.21.15.27:0/3461697545 >> v2:172.21.15.137:3300/0 conn(0x7fec100c8d30 msgr2=0x7fec1015f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec0ffff640 1 --2- 172.21.15.27:0/3461697545 >> v2:172.21.15.137:3300/0 conn(0x7fec100c8d30 0x7fec1015f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.449+0000 7fec0ffff640 1 -- 172.21.15.27:0/3461697545 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec100c2fb0 con 0x7fec10072130 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.450+0000 7fec0ffff640 1 --2- 172.21.15.27:0/3461697545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec10072130 0x7fec100c87f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fec100cb910 tx=0x7febfc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2731bc38c7b1fd55 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.453 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.450+0000 7fec0f7fe640 1 -- 172.21.15.27:0/3461697545 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc035020 con 0x7fec10072130 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.450+0000 7fec0f7fe640 1 -- 172.21.15.27:0/3461697545 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febfc031640 con 0x7fec10072130 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.450+0000 7fec0f7fe640 1 -- 172.21.15.27:0/3461697545 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc02d040 con 0x7fec10072130 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.451+0000 7fec16eba640 1 -- 172.21.15.27:0/3461697545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec10072130 msgr2=0x7fec100c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.451+0000 7fec16eba640 1 --2- 172.21.15.27:0/3461697545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec10072130 0x7fec100c87f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fec100cb910 tx=0x7febfc02b8f0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.451+0000 7fec16eba640 1 -- 172.21.15.27:0/3461697545 shutdown_connections 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.451+0000 7fec16eba640 1 --2- 172.21.15.27:0/3461697545 >> v2:172.21.15.137:3300/0 conn(0x7fec100c8d30 0x7fec1015f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.451+0000 7fec16eba640 1 --2- 172.21.15.27:0/3461697545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec10072130 0x7fec100c87f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.451+0000 7fec16eba640 1 --2- 172.21.15.27:0/3461697545 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec1015ed10 0x7fec10073970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.451+0000 7fec16eba640 1 -- 172.21.15.27:0/3461697545 >> 172.21.15.27:0/3461697545 conn(0x7fec100ca8f0 msgr2=0x7fec100cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.451+0000 7fec16eba640 1 -- 172.21.15.27:0/3461697545 shutdown_connections 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.451+0000 7fec16eba640 1 -- 172.21.15.27:0/3461697545 wait complete. 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.452+0000 7fec16eba640 1 Processor -- start 2024-04-20T11:02:30.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.452+0000 7fec16eba640 1 -- start start 2024-04-20T11:02:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec16eba640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec10072130 0x7fec1014ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec16eba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec100c8d30 0x7fec10152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec16eba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec1015ed10 0x7fec10152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec16eba640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fec100c3850 con 0x7fec100c8d30 2024-04-20T11:02:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec16eba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fec100c3570 con 0x7fec1015ed10 2024-04-20T11:02:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec16eba640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fec1015f630 con 0x7fec10072130 2024-04-20T11:02:30.457 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec0ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec100c8d30 0x7fec10152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec0ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec100c8d30 0x7fec10152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38782/0 (socket says 172.21.15.27:38782) 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec0ffff640 1 -- 172.21.15.27:0/1116309223 learned_addr learned my addr 172.21.15.27:0/1116309223 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec15457640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec1015ed10 0x7fec10152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.453+0000 7fec14c56640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec10072130 0x7fec1014ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec0ffff640 1 -- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec10072130 msgr2=0x7fec1014ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec0ffff640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec10072130 0x7fec1014ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec0ffff640 1 -- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec1015ed10 msgr2=0x7fec10152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec0ffff640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec1015ed10 0x7fec10152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec0ffff640 1 -- 172.21.15.27:0/1116309223 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1015d2a0 con 0x7fec100c8d30 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec0ffff640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec100c8d30 0x7fec10152a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7febfc000c00 tx=0x7febfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec0d7fa640 1 -- 172.21.15.27:0/1116309223 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc040070 con 0x7fec100c8d30 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec0d7fa640 1 -- 172.21.15.27:0/1116309223 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febfc0399f0 con 0x7fec100c8d30 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec101f1b50 con 0x7fec100c8d30 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec0d7fa640 1 -- 172.21.15.27:0/1116309223 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc035040 con 0x7fec100c8d30 2024-04-20T11:02:30.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.454+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec10155e00 con 0x7fec100c8d30 2024-04-20T11:02:30.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.455+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1004fcf0 con 0x7fec100c8d30 2024-04-20T11:02:30.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.456+0000 7fec0d7fa640 1 -- 172.21.15.27:0/1116309223 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febfc04b060 con 0x7fec100c8d30 2024-04-20T11:02:30.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.456+0000 7fec0d7fa640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febf406c8f0 0x7febf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.456+0000 7fec0d7fa640 1 -- 172.21.15.27:0/1116309223 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7febfc0ad450 con 0x7fec100c8d30 2024-04-20T11:02:30.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.457+0000 7fec14c56640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febf406c8f0 0x7febf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.459 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.457+0000 7fec14c56640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febf406c8f0 0x7febf406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fec04009aa0 tx=0x7fec04009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.460 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.461+0000 7fec0d7fa640 1 -- 172.21.15.27:0/1116309223 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7febfc07b010 con 0x7fec100c8d30 2024-04-20T11:02:30.645 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.645+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fec1015daf0 con 0x7febf406c8f0 2024-04-20T11:02:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.646+0000 7fec0d7fa640 1 -- 172.21.15.27:0/1116309223 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fec1015daf0 con 0x7febf406c8f0 2024-04-20T11:02:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 2 to deep-scrub 2024-04-20T11:02:30.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febf406c8f0 msgr2=0x7febf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febf406c8f0 0x7febf406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fec04009aa0 tx=0x7fec04009190 comp rx=0 tx=0).stop 2024-04-20T11:02:30.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec100c8d30 msgr2=0x7fec10152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.648 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec100c8d30 0x7fec10152a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7febfc000c00 tx=0x7febfc0046c0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 shutdown_connections 2024-04-20T11:02:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7febf406c8f0 0x7febf406edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec1015ed10 0x7fec10152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec100c8d30 0x7fec10152a90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 --2- 172.21.15.27:0/1116309223 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fec10072130 0x7fec1014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 >> 172.21.15.27:0/1116309223 conn(0x7fec100ca8f0 msgr2=0x7fec100c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 shutdown_connections 2024-04-20T11:02:30.649 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.648+0000 7fec16eba640 1 -- 172.21.15.27:0/1116309223 wait complete. 2024-04-20T11:02:30.659 INFO:tasks.ceph:Scrubbing osd.3 2024-04-20T11:02:30.659 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:02:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.764+0000 7ff17d596640 1 Processor -- start 2024-04-20T11:02:30.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.764+0000 7ff17d596640 1 -- start start 2024-04-20T11:02:30.765 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff17d596640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff1780c9000 0x7ff1780c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff17d596640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff17815b640 con 0x7ff1780b6a30 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff17d596640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff17815b8e0 con 0x7ff178155640 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff17d596640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7ff17815c460 con 0x7ff1780c9000 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff1767fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff1780b6a30 0x7ff1780cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58676/0 (socket says 172.21.15.27:58676) 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff1767fc640 1 -- 172.21.15.27:0/1130151619 learned_addr learned my addr 172.21.15.27:0/1130151619 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff1777fe640 1 --2- 172.21.15.27:0/1130151619 >> v2:172.21.15.137:3300/0 conn(0x7ff1780c9000 0x7ff1780c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff175ffb640 1 -- 172.21.15.27:0/1130151619 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 620864250 0 0) 0x7ff17815b640 con 0x7ff1780b6a30 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff175ffb640 1 -- 172.21.15.27:0/1130151619 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff160003520 con 0x7ff1780b6a30 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff175ffb640 1 -- 172.21.15.27:0/1130151619 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 492011176 0 0) 0x7ff160003520 con 0x7ff1780b6a30 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff1777fe640 1 -- 172.21.15.27:0/1130151619 >> v1:172.21.15.137:6789/0 conn(0x7ff178155640 legacy=0x7ff1780cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff1777fe640 1 -- 172.21.15.27:0/1130151619 >> v1:172.21.15.27:6789/0 conn(0x7ff1780b6a30 legacy=0x7ff1780cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.766 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.765+0000 7ff1777fe640 1 -- 172.21.15.27:0/1130151619 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17815bbc0 con 0x7ff1780c9000 2024-04-20T11:02:30.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.766+0000 7ff1777fe640 1 --2- 172.21.15.27:0/1130151619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780c9000 0x7ff1780c94b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff170009d90 tx=0x7ff17002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a99d79af1ccc74c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.766+0000 7ff175ffb640 1 -- 172.21.15.27:0/1130151619 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170039070 con 0x7ff1780c9000 2024-04-20T11:02:30.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.767+0000 7ff175ffb640 1 -- 172.21.15.27:0/1130151619 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17002c940 con 0x7ff1780c9000 2024-04-20T11:02:30.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.767+0000 7ff175ffb640 1 -- 172.21.15.27:0/1130151619 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c001690 con 0x7ff1780c9000 2024-04-20T11:02:30.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.767+0000 7ff17d596640 1 -- 172.21.15.27:0/1130151619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780c9000 msgr2=0x7ff1780c94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.767+0000 7ff17d596640 1 --2- 172.21.15.27:0/1130151619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780c9000 0x7ff1780c94b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff170009d90 tx=0x7ff17002be10 comp rx=0 tx=0).stop 2024-04-20T11:02:30.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.767+0000 7ff17d596640 1 -- 172.21.15.27:0/1130151619 shutdown_connections 2024-04-20T11:02:30.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.767+0000 7ff17d596640 1 --2- 172.21.15.27:0/1130151619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780c9000 0x7ff1780c94b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.767+0000 7ff17d596640 1 -- 172.21.15.27:0/1130151619 >> 172.21.15.27:0/1130151619 conn(0x7ff1780d0c50 msgr2=0x7ff1780d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:30.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.767+0000 7ff17d596640 1 -- 172.21.15.27:0/1130151619 shutdown_connections 2024-04-20T11:02:30.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.767+0000 7ff17d596640 1 -- 172.21.15.27:0/1130151619 wait complete. 2024-04-20T11:02:30.769 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.768+0000 7ff17d596640 1 Processor -- start 2024-04-20T11:02:30.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.768+0000 7ff17d596640 1 -- start start 2024-04-20T11:02:30.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff17d596640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780b6a30 0x7ff178158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff17d596640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1780c9000 0x7ff178158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff17d596640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff178155640 0x7ff1781e5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff17d596640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff17815c180 con 0x7ff1780c9000 2024-04-20T11:02:30.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff17d596640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1780c99f0 con 0x7ff1780b6a30 2024-04-20T11:02:30.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff17d596640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7ff178155a20 con 0x7ff178155640 2024-04-20T11:02:30.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff1767fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1780c9000 0x7ff178158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff1767fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1780c9000 0x7ff178158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38798/0 (socket says 172.21.15.27:38798) 2024-04-20T11:02:30.774 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff1767fc640 1 -- 172.21.15.27:0/2012014945 learned_addr learned my addr 172.21.15.27:0/2012014945 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.769+0000 7ff176ffd640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780b6a30 0x7ff178158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff1777fe640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff178155640 0x7ff1781e5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff1767fc640 1 -- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff178155640 msgr2=0x7ff1781e5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff1767fc640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff178155640 0x7ff1781e5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff1767fc640 1 -- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780b6a30 msgr2=0x7ff178158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff1767fc640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1780b6a30 0x7ff178158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff1767fc640 1 -- 172.21.15.27:0/2012014945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1781f1580 con 0x7ff1780c9000 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff1767fc640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1780c9000 0x7ff178158a50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c7b0 tx=0x7ff16c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff15f7fe640 1 -- 172.21.15.27:0/2012014945 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c018070 con 0x7ff1780c9000 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff15f7fe640 1 -- 172.21.15.27:0/2012014945 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16c0072f0 con 0x7ff1780c9000 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1781f1860 con 0x7ff1780c9000 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff15f7fe640 1 -- 172.21.15.27:0/2012014945 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c0047e0 con 0x7ff1780c9000 2024-04-20T11:02:30.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.770+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1781f1e20 con 0x7ff1780c9000 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.771+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1781e5a00 con 0x7ff1780c9000 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.772+0000 7ff15f7fe640 1 -- 172.21.15.27:0/2012014945 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff16c004980 con 0x7ff1780c9000 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.772+0000 7ff15f7fe640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff16406c8f0 0x7ff16406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.772+0000 7ff15f7fe640 1 -- 172.21.15.27:0/2012014945 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7ff16c08b210 con 0x7ff1780c9000 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.772+0000 7ff15f7fe640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7ff164074f90 0x7ff164077410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.772+0000 7ff15f7fe640 1 -- 172.21.15.27:0/2012014945 --> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff164077ae0 con 0x7ff164074f90 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.772+0000 7ff15f7fe640 1 -- 172.21.15.27:0/2012014945 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff16c092050 con 0x7ff1780c9000 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.772+0000 7ff176ffd640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff16406c8f0 0x7ff16406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.772+0000 7ff1777fe640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7ff164074f90 0x7ff164077410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.773+0000 7ff1777fe640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7ff164074f90 0x7ff164077410 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.773+0000 7ff176ffd640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff16406c8f0 0x7ff16406edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff168000f60 tx=0x7ff168009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.776 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.773+0000 7ff15f7fe640 1 -- 172.21.15.27:0/2012014945 <== osd.3 v2:172.21.15.27:6820/574866103 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff164077ae0 con 0x7ff164074f90 2024-04-20T11:02:30.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 --> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff17815e0e0 con 0x7ff164074f90 2024-04-20T11:02:30.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff15f7fe640 1 -- 172.21.15.27:0/2012014945 <== osd.3 v2:172.21.15.27:6820/574866103 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff17003a040 con 0x7ff164074f90 2024-04-20T11:02:30.787 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T11:02:30.787 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:02:30.787 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7ff164074f90 msgr2=0x7ff164077410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.27:6820/574866103,v1:172.21.15.27:6822/574866103] conn(0x7ff164074f90 0x7ff164077410 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff16406c8f0 msgr2=0x7ff16406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7ff16406c8f0 0x7ff16406edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff168000f60 tx=0x7ff168009040 comp rx=0 tx=0).stop 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1780c9000 msgr2=0x7ff178158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 --2- 172.21.15.27:0/2012014945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1780c9000 0x7ff178158a50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00c7b0 tx=0x7ff16c00cc70 comp rx=0 tx=0).stop 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff1777fe640 1 -- 172.21.15.27:0/2012014945 reap_dead start 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 shutdown_connections 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 >> 172.21.15.27:0/2012014945 conn(0x7ff1780d0c50 msgr2=0x7ff1780cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 shutdown_connections 2024-04-20T11:02:30.788 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.787+0000 7ff17d596640 1 -- 172.21.15.27:0/2012014945 wait complete. 2024-04-20T11:02:30.797 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-20T11:02:30.901 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.902+0000 7fab07031640 1 Processor -- start 2024-04-20T11:02:30.902 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.902+0000 7fab07031640 1 -- start start 2024-04-20T11:02:30.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab07031640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fab0015f710 0x7fab0006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.903 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab07031640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fab0015c9d0 0x7fab00071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab07031640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab0015ce20 con 0x7fab001556f0 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab07031640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fab0006ead0 con 0x7fab0015c9d0 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab07031640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fab0015ec70 con 0x7fab0015f710 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab055ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fab0015c9d0 0x7fab00071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab055ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fab0015c9d0 0x7fab00071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38810/0 (socket says 172.21.15.27:38810) 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab055ce640 1 -- 172.21.15.27:0/1175686172 learned_addr learned my addr 172.21.15.27:0/1175686172 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7faafffff640 1 --2- 172.21.15.27:0/1175686172 >> v2:172.21.15.137:3300/0 conn(0x7fab0015f710 0x7fab0006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab055ce640 1 -- 172.21.15.27:0/1175686172 >> v2:172.21.15.137:3300/0 conn(0x7fab0015f710 msgr2=0x7fab0006ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab055ce640 1 --2- 172.21.15.27:0/1175686172 >> v2:172.21.15.137:3300/0 conn(0x7fab0015f710 0x7fab0006ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab055ce640 1 -- 172.21.15.27:0/1175686172 >> v1:172.21.15.137:6789/0 conn(0x7fab001556f0 legacy=0x7fab000cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.903+0000 7fab055ce640 1 -- 172.21.15.27:0/1175686172 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0006e510 con 0x7fab0015c9d0 2024-04-20T11:02:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7fab055ce640 1 --2- 172.21.15.27:0/1175686172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 0x7fab00071570 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fab00070160 tx=0x7faaf802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2776e8aec17139b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7faaff7fe640 1 -- 172.21.15.27:0/1175686172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf802cb20 con 0x7fab0015c9d0 2024-04-20T11:02:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7faaff7fe640 1 -- 172.21.15.27:0/1175686172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf802ccc0 con 0x7fab0015c9d0 2024-04-20T11:02:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7faaff7fe640 1 -- 172.21.15.27:0/1175686172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8034610 con 0x7fab0015c9d0 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7fab07031640 1 -- 172.21.15.27:0/1175686172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 msgr2=0x7fab00071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7fab07031640 1 --2- 172.21.15.27:0/1175686172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 0x7fab00071570 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fab00070160 tx=0x7faaf802bf80 comp rx=0 tx=0).stop 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7fab07031640 1 -- 172.21.15.27:0/1175686172 shutdown_connections 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7fab07031640 1 --2- 172.21.15.27:0/1175686172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 0x7fab00071570 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7fab07031640 1 --2- 172.21.15.27:0/1175686172 >> v2:172.21.15.137:3300/0 conn(0x7fab0015f710 0x7fab0006ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.904+0000 7fab07031640 1 -- 172.21.15.27:0/1175686172 >> 172.21.15.27:0/1175686172 conn(0x7fab00067ae0 msgr2=0x7fab00067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.905+0000 7fab07031640 1 -- 172.21.15.27:0/1175686172 shutdown_connections 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.905+0000 7fab07031640 1 -- 172.21.15.27:0/1175686172 wait complete. 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.906+0000 7fab07031640 1 Processor -- start 2024-04-20T11:02:30.909 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.906+0000 7fab07031640 1 -- start start 2024-04-20T11:02:30.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.906+0000 7fab07031640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fab001556f0 0x7fab00152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.906+0000 7fab07031640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 0x7fab0014db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.906+0000 7fab07031640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab0015f710 0x7fab0014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.906+0000 7fab07031640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fab0015f1f0 con 0x7fab0015c9d0 2024-04-20T11:02:30.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.906+0000 7fab07031640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fab0015ef10 con 0x7fab0015f710 2024-04-20T11:02:30.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.906+0000 7fab07031640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fab000c9800 con 0x7fab001556f0 2024-04-20T11:02:30.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 0x7fab0014db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 0x7fab0014db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38812/0 (socket says 172.21.15.27:38812) 2024-04-20T11:02:30.910 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafffff640 1 -- 172.21.15.27:0/2134930170 learned_addr learned my addr 172.21.15.27:0/2134930170 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7fab055ce640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab0015f710 0x7fab0014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7fab04dcd640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fab001556f0 0x7fab00152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafffff640 1 -- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fab001556f0 msgr2=0x7fab00152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafffff640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fab001556f0 0x7fab00152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafffff640 1 -- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab0015f710 msgr2=0x7fab0014e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafffff640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab0015f710 0x7fab0014e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafffff640 1 -- 172.21.15.27:0/2134930170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab00150880 con 0x7fab0015c9d0 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7fab04dcd640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fab001556f0 0x7fab00152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafffff640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 0x7fab0014db70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faaec00d5f0 tx=0x7faaec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafd7fa640 1 -- 172.21.15.27:0/2134930170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec0125f0 con 0x7fab0015c9d0 2024-04-20T11:02:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafd7fa640 1 -- 172.21.15.27:0/2134930170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaec019070 con 0x7fab0015c9d0 2024-04-20T11:02:30.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab000cb5a0 con 0x7fab0015c9d0 2024-04-20T11:02:30.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7faafd7fa640 1 -- 172.21.15.27:0/2134930170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec0129d0 con 0x7fab0015c9d0 2024-04-20T11:02:30.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.907+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab00150b20 con 0x7fab0015c9d0 2024-04-20T11:02:30.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.908+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab000688e0 con 0x7fab0015c9d0 2024-04-20T11:02:30.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.909+0000 7faafd7fa640 1 -- 172.21.15.27:0/2134930170 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faaec012b70 con 0x7fab0015c9d0 2024-04-20T11:02:30.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.909+0000 7faafd7fa640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7faae406c920 0x7faae406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:30.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.909+0000 7faafd7fa640 1 -- 172.21.15.27:0/2134930170 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7faaec08c240 con 0x7fab0015c9d0 2024-04-20T11:02:30.912 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.909+0000 7fab04dcd640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7faae406c920 0x7faae406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:30.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.910+0000 7fab04dcd640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7faae406c920 0x7faae406ede0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faaf4003eb0 tx=0x7faaf4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:30.913 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:30.914+0000 7faafd7fa640 1 -- 172.21.15.27:0/2134930170 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faaec05a3d0 con 0x7fab0015c9d0 2024-04-20T11:02:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.103+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fab0004fcf0 con 0x7faae406c920 2024-04-20T11:02:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.103+0000 7faafd7fa640 1 -- 172.21.15.27:0/2134930170 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fab0004fcf0 con 0x7faae406c920 2024-04-20T11:02:31.103 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 3 to deep-scrub 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.105+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7faae406c920 msgr2=0x7faae406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7faae406c920 0x7faae406ede0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faaf4003eb0 tx=0x7faaf4009040 comp rx=0 tx=0).stop 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 msgr2=0x7fab0014db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 0x7fab0014db70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faaec00d5f0 tx=0x7faaec00dab0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 shutdown_connections 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7faae406c920 0x7faae406ede0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab0015f710 0x7fab0014e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab0015c9d0 0x7fab0014db70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 --2- 172.21.15.27:0/2134930170 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fab001556f0 0x7fab00152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 >> 172.21.15.27:0/2134930170 conn(0x7fab00067ae0 msgr2=0x7fab00070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:31.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 shutdown_connections 2024-04-20T11:02:31.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.106+0000 7fab07031640 1 -- 172.21.15.27:0/2134930170 wait complete. 2024-04-20T11:02:31.117 INFO:tasks.ceph:Scrubbing osd.4 2024-04-20T11:02:31.117 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:02:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.222+0000 7f3700d7e640 1 Processor -- start 2024-04-20T11:02:31.222 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.222+0000 7f3700d7e640 1 -- start start 2024-04-20T11:02:31.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.223+0000 7f3700d7e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0cbdf0 0x7f36fc15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.223+0000 7f3700d7e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f36fc0c66b0 0x7f36fc0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.223+0000 7f3700d7e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36fc0c5d50 con 0x7f36fc0c6160 2024-04-20T11:02:31.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.223+0000 7f3700d7e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f36fc0ce5e0 con 0x7f36fc0c66b0 2024-04-20T11:02:31.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.223+0000 7f3700d7e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f36fc15be10 con 0x7f36fc0cbdf0 2024-04-20T11:02:31.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.223+0000 7f36fad9d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f36fc0c66b0 0x7f36fc0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.223+0000 7f36fa59c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0cbdf0 0x7f36fc15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.223+0000 7f36f3fff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f36fc0c6160 0x7f36fc0cd640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:59900/0 (socket says 172.21.15.27:59900) 2024-04-20T11:02:31.223 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.223+0000 7f36f3fff640 1 -- 172.21.15.27:0/3688690443 learned_addr learned my addr 172.21.15.27:0/3688690443 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:31.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.224+0000 7f36fa59c640 1 -- 172.21.15.27:0/3688690443 >> v2:172.21.15.137:3300/0 conn(0x7f36fc0c66b0 msgr2=0x7f36fc0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:31.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.224+0000 7f36fa59c640 1 --2- 172.21.15.27:0/3688690443 >> v2:172.21.15.137:3300/0 conn(0x7f36fc0c66b0 0x7f36fc0cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.224+0000 7f36fa59c640 1 -- 172.21.15.27:0/3688690443 >> v1:172.21.15.137:6789/0 conn(0x7f36fc0c6160 legacy=0x7f36fc0cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.224+0000 7f36fa59c640 1 -- 172.21.15.27:0/3688690443 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36fc0ce060 con 0x7f36fc0cbdf0 2024-04-20T11:02:31.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.224+0000 7f36fa59c640 1 --2- 172.21.15.27:0/3688690443 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0cbdf0 0x7f36fc15c2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0097a0 tx=0x7f36ec02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b913777675de5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f36f9d9b640 1 -- 172.21.15.27:0/3688690443 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec035020 con 0x7f36fc0cbdf0 2024-04-20T11:02:31.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f36f9d9b640 1 -- 172.21.15.27:0/3688690443 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36ec031830 con 0x7f36fc0cbdf0 2024-04-20T11:02:31.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f36f9d9b640 1 -- 172.21.15.27:0/3688690443 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec031c00 con 0x7f36fc0cbdf0 2024-04-20T11:02:31.225 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f3700d7e640 1 -- 172.21.15.27:0/3688690443 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0cbdf0 msgr2=0x7f36fc15c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f3700d7e640 1 --2- 172.21.15.27:0/3688690443 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0cbdf0 0x7f36fc15c2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0097a0 tx=0x7f36ec02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f3700d7e640 1 -- 172.21.15.27:0/3688690443 shutdown_connections 2024-04-20T11:02:31.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f3700d7e640 1 --2- 172.21.15.27:0/3688690443 >> v2:172.21.15.137:3300/0 conn(0x7f36fc0c66b0 0x7f36fc0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f3700d7e640 1 --2- 172.21.15.27:0/3688690443 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0cbdf0 0x7f36fc15c2a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f3700d7e640 1 -- 172.21.15.27:0/3688690443 >> 172.21.15.27:0/3688690443 conn(0x7f36fc0676c0 msgr2=0x7f36fc067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:31.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f3700d7e640 1 -- 172.21.15.27:0/3688690443 shutdown_connections 2024-04-20T11:02:31.226 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.225+0000 7f3700d7e640 1 -- 172.21.15.27:0/3688690443 wait complete. 2024-04-20T11:02:31.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.226+0000 7f3700d7e640 1 Processor -- start 2024-04-20T11:02:31.228 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.226+0000 7f3700d7e640 1 -- start start 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f3700d7e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0c6160 0x7f36fc0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f3700d7e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36fc0c66b0 0x7f36fc150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f3700d7e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36fc0cbdf0 0x7f36fc150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f3700d7e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc15b7c0 con 0x7f36fc0c66b0 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f3700d7e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc0ce300 con 0x7f36fc0cbdf0 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f3700d7e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f36fc15baa0 con 0x7f36fc0c6160 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f36f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36fc0c66b0 0x7f36fc150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f36f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36fc0c66b0 0x7f36fc150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38814/0 (socket says 172.21.15.27:38814) 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f36f3fff640 1 -- 172.21.15.27:0/550636270 learned_addr learned my addr 172.21.15.27:0/550636270 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f36fad9d640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36fc0cbdf0 0x7f36fc150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.227+0000 7f36fa59c640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0c6160 0x7f36fc0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36f3fff640 1 -- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0c6160 msgr2=0x7f36fc0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36f3fff640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0c6160 0x7f36fc0cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.229 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36f3fff640 1 -- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36fc0cbdf0 msgr2=0x7f36fc150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36f3fff640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36fc0cbdf0 0x7f36fc150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36f3fff640 1 -- 172.21.15.27:0/550636270 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36fc1f5d20 con 0x7f36fc0c66b0 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36fad9d640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36fc0cbdf0 0x7f36fc150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36fa59c640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f36fc0c6160 0x7f36fc0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36f3fff640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36fc0c66b0 0x7f36fc150130 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f36e400c430 tx=0x7f36e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36f2ffd640 1 -- 172.21.15.27:0/550636270 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4017070 con 0x7f36fc0c66b0 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36f2ffd640 1 -- 172.21.15.27:0/550636270 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e4005ce0 con 0x7f36fc0c66b0 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36fc0cf550 con 0x7f36fc0c66b0 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f36f2ffd640 1 -- 172.21.15.27:0/550636270 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e40072f0 con 0x7f36fc0c66b0 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.228+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36fc0d0680 con 0x7f36fc0c66b0 2024-04-20T11:02:31.230 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.229+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36fc0b6b00 con 0x7f36fc0c66b0 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.229+0000 7f36f2ffd640 1 -- 172.21.15.27:0/550636270 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36e401a070 con 0x7f36fc0c66b0 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.230+0000 7f36f2ffd640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f36dc06c8f0 0x7f36dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.230+0000 7f36f2ffd640 1 -- 172.21.15.27:0/550636270 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f36e408b220 con 0x7f36fc0c66b0 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.230+0000 7f36f2ffd640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f36dc074f90 0x7f36dc077410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.230+0000 7f36f2ffd640 1 -- 172.21.15.27:0/550636270 --> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36e4009e50 con 0x7f36dc074f90 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.230+0000 7f36f2ffd640 1 -- 172.21.15.27:0/550636270 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36e4092050 con 0x7f36fc0c66b0 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.230+0000 7f36fa59c640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f36dc06c8f0 0x7f36dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.230+0000 7f36fad9d640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f36dc074f90 0x7f36dc077410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.231+0000 7f36fa59c640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f36dc06c8f0 0x7f36dc06edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f36ec036000 tx=0x7f36ec03c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.232 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.231+0000 7f36fad9d640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f36dc074f90 0x7f36dc077410 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.232+0000 7f36f2ffd640 1 -- 172.21.15.27:0/550636270 <== osd.4 v2:172.21.15.137:6800/3079865776 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36e4009e50 con 0x7f36dc074f90 2024-04-20T11:02:31.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.245+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 --> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f36fc04fcf0 con 0x7f36dc074f90 2024-04-20T11:02:31.245 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f36f2ffd640 1 -- 172.21.15.27:0/550636270 <== osd.4 v2:172.21.15.137:6800/3079865776 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f36fc04fcf0 con 0x7f36dc074f90 2024-04-20T11:02:31.245 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T11:02:31.245 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:02:31.246 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T11:02:31.246 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f36dc074f90 msgr2=0x7f36dc077410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6800/3079865776,v1:172.21.15.137:6801/3079865776] conn(0x7f36dc074f90 0x7f36dc077410 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f36dc06c8f0 msgr2=0x7f36dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f36dc06c8f0 0x7f36dc06edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f36ec036000 tx=0x7f36ec03c040 comp rx=0 tx=0).stop 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36fc0c66b0 msgr2=0x7f36fc150130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 --2- 172.21.15.27:0/550636270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36fc0c66b0 0x7f36fc150130 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f36e400c430 tx=0x7f36e400c8f0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f36fad9d640 1 -- 172.21.15.27:0/550636270 reap_dead start 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 shutdown_connections 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 >> 172.21.15.27:0/550636270 conn(0x7f36fc0676c0 msgr2=0x7f36fc159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 shutdown_connections 2024-04-20T11:02:31.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.246+0000 7f3700d7e640 1 -- 172.21.15.27:0/550636270 wait complete. 2024-04-20T11:02:31.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-20T11:02:31.364 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.365+0000 7fe7c6806640 1 Processor -- start 2024-04-20T11:02:31.365 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.365+0000 7fe7c6806640 1 -- start start 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c6806640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe7c01537f0 0x7fe7c00c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c6806640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0159260 0x7fe7c00c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c6806640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7c00c7cd0 con 0x7fe7c0153410 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c6806640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fe7c006cad0 con 0x7fe7c01537f0 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c6806640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c00c7fb0 con 0x7fe7c0159260 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c4da3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0159260 0x7fe7c00c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7bf7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe7c01537f0 0x7fe7c00c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7bffff640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fe7c0153410 0x7fe7c015d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:59902/0 (socket says 172.21.15.27:59902) 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7bf7fe640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe7c01537f0 0x7fe7c00c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:36220/0 (socket says 172.21.15.27:36220) 2024-04-20T11:02:31.366 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c4da3640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0159260 0x7fe7c00c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:40306/0 (socket says 172.21.15.27:40306) 2024-04-20T11:02:31.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7bffff640 1 -- 172.21.15.27:0/1143753640 learned_addr learned my addr 172.21.15.27:0/1143753640 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:31.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c4da3640 1 -- 172.21.15.27:0/1143753640 >> v2:172.21.15.137:3300/0 conn(0x7fe7c01537f0 msgr2=0x7fe7c00c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:31.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c4da3640 1 --2- 172.21.15.27:0/1143753640 >> v2:172.21.15.137:3300/0 conn(0x7fe7c01537f0 0x7fe7c00c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7beffd640 1 -- 172.21.15.27:0/1143753640 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2523890694 0 0) 0x7fe7c00c7cd0 con 0x7fe7c0153410 2024-04-20T11:02:31.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c4da3640 1 -- 172.21.15.27:0/1143753640 >> v1:172.21.15.137:6789/0 conn(0x7fe7c0153410 legacy=0x7fe7c015d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.367 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.366+0000 7fe7c4da3640 1 -- 172.21.15.27:0/1143753640 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c00c8290 con 0x7fe7c0159260 2024-04-20T11:02:31.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.367+0000 7fe7c4da3640 1 --2- 172.21.15.27:0/1143753640 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0159260 0x7fe7c00c6c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009d90 tx=0x7fe7b802bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf8acfaee6696740 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.367+0000 7fe7beffd640 1 -- 172.21.15.27:0/1143753640 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8039070 con 0x7fe7c0159260 2024-04-20T11:02:31.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.367+0000 7fe7beffd640 1 -- 172.21.15.27:0/1143753640 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b802cce0 con 0x7fe7c0159260 2024-04-20T11:02:31.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.367+0000 7fe7beffd640 1 -- 172.21.15.27:0/1143753640 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8034660 con 0x7fe7c0159260 2024-04-20T11:02:31.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.368+0000 7fe7c6806640 1 -- 172.21.15.27:0/1143753640 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0159260 msgr2=0x7fe7c00c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.368+0000 7fe7c6806640 1 --2- 172.21.15.27:0/1143753640 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0159260 0x7fe7c00c6c70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009d90 tx=0x7fe7b802bf30 comp rx=0 tx=0).stop 2024-04-20T11:02:31.368 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.368+0000 7fe7c6806640 1 -- 172.21.15.27:0/1143753640 shutdown_connections 2024-04-20T11:02:31.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.368+0000 7fe7c6806640 1 --2- 172.21.15.27:0/1143753640 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0159260 0x7fe7c00c6c70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.368+0000 7fe7c6806640 1 --2- 172.21.15.27:0/1143753640 >> v2:172.21.15.137:3300/0 conn(0x7fe7c01537f0 0x7fe7c00c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.368+0000 7fe7c6806640 1 -- 172.21.15.27:0/1143753640 >> 172.21.15.27:0/1143753640 conn(0x7fe7c00ccbd0 msgr2=0x7fe7c00ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:31.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.368+0000 7fe7c6806640 1 -- 172.21.15.27:0/1143753640 shutdown_connections 2024-04-20T11:02:31.369 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.368+0000 7fe7c6806640 1 -- 172.21.15.27:0/1143753640 wait complete. 2024-04-20T11:02:31.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.369+0000 7fe7c6806640 1 Processor -- start 2024-04-20T11:02:31.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.369+0000 7fe7c6806640 1 -- start start 2024-04-20T11:02:31.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7c6806640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0153410 0x7fe7c0147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7c6806640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7c01537f0 0x7fe7c0143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7c6806640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c0159260 0x7fe7c0144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7c6806640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c00c75d0 con 0x7fe7c01537f0 2024-04-20T11:02:31.371 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7c6806640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c015cc50 con 0x7fe7c0159260 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7c6806640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c00c7890 con 0x7fe7c0153410 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7c01537f0 0x7fe7c0143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7c01537f0 0x7fe7c0143710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38830/0 (socket says 172.21.15.27:38830) 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bf7fe640 1 -- 172.21.15.27:0/2475471835 learned_addr learned my addr 172.21.15.27:0/2475471835 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7c4da3640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c0159260 0x7fe7c0144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bffff640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0153410 0x7fe7c0147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bf7fe640 1 -- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0153410 msgr2=0x7fe7c0147dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bf7fe640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0153410 0x7fe7c0147dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bf7fe640 1 -- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c0159260 msgr2=0x7fe7c0144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bf7fe640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c0159260 0x7fe7c0144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bf7fe640 1 -- 172.21.15.27:0/2475471835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c01465e0 con 0x7fe7c01537f0 2024-04-20T11:02:31.372 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.370+0000 7fe7bf7fe640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7c01537f0 0x7fe7c0143710 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e9e0 tx=0x7fe7b400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.371+0000 7fe7bcff9640 1 -- 172.21.15.27:0/2475471835 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4017070 con 0x7fe7c01537f0 2024-04-20T11:02:31.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.371+0000 7fe7bcff9640 1 -- 172.21.15.27:0/2475471835 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b400c9d0 con 0x7fe7c01537f0 2024-04-20T11:02:31.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.371+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c01488e0 con 0x7fe7c01537f0 2024-04-20T11:02:31.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.371+0000 7fe7bcff9640 1 -- 172.21.15.27:0/2475471835 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400cdc0 con 0x7fe7c01537f0 2024-04-20T11:02:31.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.371+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c0156e10 con 0x7fe7c01537f0 2024-04-20T11:02:31.373 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.372+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c004fcf0 con 0x7fe7c01537f0 2024-04-20T11:02:31.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.372+0000 7fe7bcff9640 1 -- 172.21.15.27:0/2475471835 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7b401a070 con 0x7fe7c01537f0 2024-04-20T11:02:31.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.373+0000 7fe7bcff9640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.373+0000 7fe7bcff9640 1 -- 172.21.15.27:0/2475471835 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fe7b408bcc0 con 0x7fe7c01537f0 2024-04-20T11:02:31.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.373+0000 7fe7bffff640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.374 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.373+0000 7fe7bffff640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007fd0 tx=0x7fe7b0007de0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.376 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.377+0000 7fe7bcff9640 1 -- 172.21.15.27:0/2475471835 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe7b4059880 con 0x7fe7c01537f0 2024-04-20T11:02:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.565+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe7c00c9050 con 0x7fe7ac06c8f0 2024-04-20T11:02:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.566+0000 7fe7bcff9640 1 -- 172.21.15.27:0/2475471835 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe7c00c9050 con 0x7fe7ac06c8f0 2024-04-20T11:02:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 4 to deep-scrub 2024-04-20T11:02:31.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe7ac06c8f0 msgr2=0x7fe7ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.568 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0007fd0 tx=0x7fe7b0007de0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7c01537f0 msgr2=0x7fe7c0143710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7c01537f0 0x7fe7c0143710 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e9e0 tx=0x7fe7b400eea0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 shutdown_connections 2024-04-20T11:02:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fe7ac06c8f0 0x7fe7ac06edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7c0159260 0x7fe7c0144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7c01537f0 0x7fe7c0143710 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 --2- 172.21.15.27:0/2475471835 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe7c0153410 0x7fe7c0147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 >> 172.21.15.27:0/2475471835 conn(0x7fe7c00ccbd0 msgr2=0x7fe7c0158310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 shutdown_connections 2024-04-20T11:02:31.570 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.568+0000 7fe7c6806640 1 -- 172.21.15.27:0/2475471835 wait complete. 2024-04-20T11:02:31.579 INFO:tasks.ceph:Scrubbing osd.5 2024-04-20T11:02:31.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:02:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.684+0000 7f1678bbd640 1 Processor -- start 2024-04-20T11:02:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.684+0000 7f1678bbd640 1 -- start start 2024-04-20T11:02:31.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.685+0000 7f1678bbd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 0x7f16740ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.685 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.685+0000 7f1678bbd640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f16740b3e60 0x7f16740ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.685+0000 7f1678bbd640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f167415f7f0 con 0x7f16740caf10 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.685+0000 7f1678bbd640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f16740cd600 con 0x7f16740b3e60 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.685+0000 7f1678bbd640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f16740b4310 con 0x7f16740cbff0 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.685+0000 7f1672d9d640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f16740b3e60 0x7f16740ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.685+0000 7f1671d9b640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 0x7f16740ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.685+0000 7f167259c640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f16740caf10 0x7f16740d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:59910/0 (socket says 172.21.15.27:59910) 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.686+0000 7f167259c640 1 -- 172.21.15.27:0/2550632431 learned_addr learned my addr 172.21.15.27:0/2550632431 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.686+0000 7f1672d9d640 1 --2- 172.21.15.27:0/2550632431 >> v2:172.21.15.137:3300/0 conn(0x7f16740b3e60 0x7f16740ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.27:36234/0 (socket says 172.21.15.27:36234) 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.686+0000 7f1672d9d640 1 -- 172.21.15.27:0/2550632431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 msgr2=0x7f16740ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.686+0000 7f1672d9d640 1 --2- 172.21.15.27:0/2550632431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 0x7f16740ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.686+0000 7f1672d9d640 1 -- 172.21.15.27:0/2550632431 >> v1:172.21.15.137:6789/0 conn(0x7f16740caf10 legacy=0x7f16740d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.686+0000 7f1672d9d640 1 -- 172.21.15.27:0/2550632431 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167415fab0 con 0x7f16740b3e60 2024-04-20T11:02:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.686+0000 7f1671d9b640 1 --2- 172.21.15.27:0/2550632431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 0x7f16740ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:02:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.687+0000 7f1672d9d640 1 --2- 172.21.15.27:0/2550632431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16740b3e60 0x7f16740ccac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f166c0097a0 tx=0x7f166c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5db5813ef7d0d39 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.687+0000 7f167159a640 1 -- 172.21.15.27:0/2550632431 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c035020 con 0x7f16740b3e60 2024-04-20T11:02:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.687+0000 7f167159a640 1 -- 172.21.15.27:0/2550632431 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c031830 con 0x7f16740b3e60 2024-04-20T11:02:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.687+0000 7f167159a640 1 -- 172.21.15.27:0/2550632431 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c031c00 con 0x7f16740b3e60 2024-04-20T11:02:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.687+0000 7f1678bbd640 1 -- 172.21.15.27:0/2550632431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16740b3e60 msgr2=0x7f16740ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.687+0000 7f1678bbd640 1 --2- 172.21.15.27:0/2550632431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16740b3e60 0x7f16740ccac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f166c0097a0 tx=0x7f166c02b8d0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.688+0000 7f1678bbd640 1 -- 172.21.15.27:0/2550632431 shutdown_connections 2024-04-20T11:02:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.688+0000 7f1678bbd640 1 --2- 172.21.15.27:0/2550632431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16740b3e60 0x7f16740ccac0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.688+0000 7f1678bbd640 1 --2- 172.21.15.27:0/2550632431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 0x7f16740ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.688+0000 7f1678bbd640 1 -- 172.21.15.27:0/2550632431 >> 172.21.15.27:0/2550632431 conn(0x7f167406dde0 msgr2=0x7f167406e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:31.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.688+0000 7f1678bbd640 1 -- 172.21.15.27:0/2550632431 shutdown_connections 2024-04-20T11:02:31.689 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.688+0000 7f1678bbd640 1 -- 172.21.15.27:0/2550632431 wait complete. 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.689+0000 7f1678bbd640 1 Processor -- start 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.689+0000 7f1678bbd640 1 -- start start 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.689+0000 7f1678bbd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16740b3e60 0x7f167415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.689+0000 7f1678bbd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16740caf10 0x7f16741f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.689+0000 7f1678bbd640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 0x7f16741f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1678bbd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f16740cde10 con 0x7f16740caf10 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1678bbd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16740cdb30 con 0x7f16740b3e60 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1678bbd640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f16740d1a60 con 0x7f16740cbff0 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1671d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16740caf10 0x7f16741f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1671d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16740caf10 0x7f16741f4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38846/0 (socket says 172.21.15.27:38846) 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1671d9b640 1 -- 172.21.15.27:0/2680878528 learned_addr learned my addr 172.21.15.27:0/2680878528 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1672d9d640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 0x7f16741f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f167259c640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16740b3e60 0x7f167415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1671d9b640 1 -- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 msgr2=0x7f16741f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1671d9b640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f16740cbff0 0x7f16741f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1671d9b640 1 -- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16740b3e60 msgr2=0x7f167415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1671d9b640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16740b3e60 0x7f167415de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1671d9b640 1 -- 172.21.15.27:0/2680878528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167415d660 con 0x7f16740caf10 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1671d9b640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16740caf10 0x7f16741f4680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f165c00ebe0 tx=0x7f165c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f166affd640 1 -- 172.21.15.27:0/2680878528 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c017070 con 0x7f16740caf10 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f166affd640 1 -- 172.21.15.27:0/2680878528 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c00cc30 con 0x7f16740caf10 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1674157500 con 0x7f16740caf10 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f166affd640 1 -- 172.21.15.27:0/2680878528 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c013470 con 0x7f16740caf10 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.690+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1674158c10 con 0x7f16740caf10 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.691+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f167404fcf0 con 0x7f16740caf10 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.692+0000 7f166affd640 1 -- 172.21.15.27:0/2680878528 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f165c01a070 con 0x7f16740caf10 2024-04-20T11:02:31.694 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.692+0000 7f166affd640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f165406c8f0 0x7f165406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.692+0000 7f166affd640 1 -- 172.21.15.27:0/2680878528 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f165c08bc30 con 0x7f16740caf10 2024-04-20T11:02:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.692+0000 7f166affd640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7f1654074f90 0x7f1654077410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.692+0000 7f166affd640 1 -- 172.21.15.27:0/2680878528 --> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f165c0093e0 con 0x7f1654074f90 2024-04-20T11:02:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.692+0000 7f166affd640 1 -- 172.21.15.27:0/2680878528 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f165c08e330 con 0x7f16740caf10 2024-04-20T11:02:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.692+0000 7f167259c640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f165406c8f0 0x7f165406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.693+0000 7f1672d9d640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7f1654074f90 0x7f1654077410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.693+0000 7f167259c640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f165406c8f0 0x7f165406edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f16640012e0 tx=0x7f1664001a30 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.693+0000 7f1672d9d640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7f1654074f90 0x7f1654077410 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.694+0000 7f166affd640 1 -- 172.21.15.27:0/2680878528 <== osd.5 v2:172.21.15.137:6824/673751184 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f165c0093e0 con 0x7f1654074f90 2024-04-20T11:02:31.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.707+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 --> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f16740cc8a0 con 0x7f1654074f90 2024-04-20T11:02:31.707 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f166affd640 1 -- 172.21.15.27:0/2680878528 <== osd.5 v2:172.21.15.137:6824/673751184 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f16740cc8a0 con 0x7f1654074f90 2024-04-20T11:02:31.707 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T11:02:31.708 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:02:31.708 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T11:02:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7f1654074f90 msgr2=0x7f1654077410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6824/673751184,v1:172.21.15.137:6825/673751184] conn(0x7f1654074f90 0x7f1654077410 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f165406c8f0 msgr2=0x7f165406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f165406c8f0 0x7f165406edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f16640012e0 tx=0x7f1664001a30 comp rx=0 tx=0).stop 2024-04-20T11:02:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16740caf10 msgr2=0x7f16741f4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 --2- 172.21.15.27:0/2680878528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16740caf10 0x7f16741f4680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f165c00ebe0 tx=0x7f165c00c520 comp rx=0 tx=0).stop 2024-04-20T11:02:31.709 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1672d9d640 1 -- 172.21.15.27:0/2680878528 reap_dead start 2024-04-20T11:02:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 shutdown_connections 2024-04-20T11:02:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 >> 172.21.15.27:0/2680878528 conn(0x7f167406dde0 msgr2=0x7f16740d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 shutdown_connections 2024-04-20T11:02:31.710 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.708+0000 7f1678bbd640 1 -- 172.21.15.27:0/2680878528 wait complete. 2024-04-20T11:02:31.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-20T11:02:31.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.821+0000 7f1f17ec1640 1 Processor -- start 2024-04-20T11:02:31.821 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.821+0000 7f1f17ec1640 1 -- start start 2024-04-20T11:02:31.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.822+0000 7f1f17ec1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f10155950 0x7f1f100cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.822+0000 7f1f17ec1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f100cc330 0x7f1f100b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.822+0000 7f1f17ec1640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f100b5ea0 con 0x7f1f100b5a30 2024-04-20T11:02:31.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.822+0000 7f1f17ec1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f1f1015b9c0 con 0x7f1f10155950 2024-04-20T11:02:31.824 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.822+0000 7f1f17ec1640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1015bca0 con 0x7f1f100cc330 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.822+0000 7f1f15c5d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f10155950 0x7f1f100cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.822+0000 7f1f15c5d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f10155950 0x7f1f100cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38858/0 (socket says 172.21.15.27:38858) 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.822+0000 7f1f15c5d640 1 -- 172.21.15.27:0/3149570824 learned_addr learned my addr 172.21.15.27:0/3149570824 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.823+0000 7f1f1545c640 1 --2- 172.21.15.27:0/3149570824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f100cc330 0x7f1f100b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.823+0000 7f1f15c5d640 1 -- 172.21.15.27:0/3149570824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f100cc330 msgr2=0x7f1f100b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.823+0000 7f1f15c5d640 1 --2- 172.21.15.27:0/3149570824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f100cc330 0x7f1f100b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.823+0000 7f1f15c5d640 1 -- 172.21.15.27:0/3149570824 >> v1:172.21.15.137:6789/0 conn(0x7f1f100b5a30 legacy=0x7f1f100d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.823+0000 7f1f15c5d640 1 -- 172.21.15.27:0/3149570824 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1015bf80 con 0x7f1f10155950 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.823+0000 7f1f1545c640 1 --2- 172.21.15.27:0/3149570824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f100cc330 0x7f1f100b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:02:31.825 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.823+0000 7f1f15c5d640 1 --2- 172.21.15.27:0/3149570824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f10155950 0x7f1f100cbdf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009870 tx=0x7f1f0802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a509ca6bb7cadd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f14c5b640 1 -- 172.21.15.27:0/3149570824 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0802f830 con 0x7f1f10155950 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f14c5b640 1 -- 172.21.15.27:0/3149570824 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0802f9d0 con 0x7f1f10155950 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f14c5b640 1 -- 172.21.15.27:0/3149570824 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0802fda0 con 0x7f1f10155950 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f17ec1640 1 -- 172.21.15.27:0/3149570824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f10155950 msgr2=0x7f1f100cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f17ec1640 1 --2- 172.21.15.27:0/3149570824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f10155950 0x7f1f100cbdf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009870 tx=0x7f1f0802ba10 comp rx=0 tx=0).stop 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f17ec1640 1 -- 172.21.15.27:0/3149570824 shutdown_connections 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f17ec1640 1 --2- 172.21.15.27:0/3149570824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f100cc330 0x7f1f100b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f17ec1640 1 --2- 172.21.15.27:0/3149570824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f10155950 0x7f1f100cbdf0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f17ec1640 1 -- 172.21.15.27:0/3149570824 >> 172.21.15.27:0/3149570824 conn(0x7f1f100676c0 msgr2=0x7f1f10067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f17ec1640 1 -- 172.21.15.27:0/3149570824 shutdown_connections 2024-04-20T11:02:31.826 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.824+0000 7f1f17ec1640 1 -- 172.21.15.27:0/3149570824 wait complete. 2024-04-20T11:02:31.828 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.825+0000 7f1f17ec1640 1 Processor -- start 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.825+0000 7f1f17ec1640 1 -- start start 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f17ec1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f100b5a30 0x7f1f100c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f17ec1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f100cc330 0x7f1f101f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f17ec1640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f10155950 0x7f1f101f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f17ec1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1015fb90 con 0x7f1f100b5a30 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f17ec1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1015c540 con 0x7f1f100cc330 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f17ec1640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1015fd10 con 0x7f1f10155950 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f15c5d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f100b5a30 0x7f1f100c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f15c5d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f100b5a30 0x7f1f100c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38872/0 (socket says 172.21.15.27:38872) 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f15c5d640 1 -- 172.21.15.27:0/332669139 learned_addr learned my addr 172.21.15.27:0/332669139 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f1545c640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f100cc330 0x7f1f101f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.826+0000 7f1f1645e640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f10155950 0x7f1f101f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f15c5d640 1 -- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f10155950 msgr2=0x7f1f101f19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.829 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f15c5d640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f10155950 0x7f1f101f19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f15c5d640 1 -- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f100cc330 msgr2=0x7f1f101f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f15c5d640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f100cc330 0x7f1f101f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f15c5d640 1 -- 172.21.15.27:0/332669139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f100c7b60 con 0x7f1f100b5a30 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f15c5d640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f100b5a30 0x7f1f100c7620 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1f08000c00 tx=0x7f1f08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f0e7fc640 1 -- 172.21.15.27:0/332669139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0802f800 con 0x7f1f100b5a30 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f0e7fc640 1 -- 172.21.15.27:0/332669139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08046070 con 0x7f1f100b5a30 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1015b4a0 con 0x7f1f100b5a30 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f0e7fc640 1 -- 172.21.15.27:0/332669139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08041660 con 0x7f1f100b5a30 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.827+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f101f4e90 con 0x7f1f100b5a30 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.828+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f100686d0 con 0x7f1f100b5a30 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.828+0000 7f1f0e7fc640 1 -- 172.21.15.27:0/332669139 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f08040070 con 0x7f1f100b5a30 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.829+0000 7f1f0e7fc640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1efc06c8f0 0x7f1efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:31.830 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.829+0000 7f1f0e7fc640 1 -- 172.21.15.27:0/332669139 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f1f080ae550 con 0x7f1f100b5a30 2024-04-20T11:02:31.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.829+0000 7f1f1545c640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1efc06c8f0 0x7f1efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:31.831 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.830+0000 7f1f1545c640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1efc06c8f0 0x7f1efc06edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1f101f25d0 tx=0x7f1f04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:31.833 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:31.833+0000 7f1f0e7fc640 1 -- 172.21.15.27:0/332669139 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1f100686d0 con 0x7f1f100b5a30 2024-04-20T11:02:32.022 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.022+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f1f1004fcf0 con 0x7f1efc06c8f0 2024-04-20T11:02:32.023 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.023+0000 7f1f0e7fc640 1 -- 172.21.15.27:0/332669139 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1f1004fcf0 con 0x7f1efc06c8f0 2024-04-20T11:02:32.023 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 5 to deep-scrub 2024-04-20T11:02:32.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.025+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1efc06c8f0 msgr2=0x7f1efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.025+0000 7f1f17ec1640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1efc06c8f0 0x7f1efc06edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1f101f25d0 tx=0x7f1f04008040 comp rx=0 tx=0).stop 2024-04-20T11:02:32.025 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.025+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f100b5a30 msgr2=0x7f1f100c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.025+0000 7f1f17ec1640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f100b5a30 0x7f1f100c7620 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1f08000c00 tx=0x7f1f08004730 comp rx=0 tx=0).stop 2024-04-20T11:02:32.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.025+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 shutdown_connections 2024-04-20T11:02:32.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.026+0000 7f1f17ec1640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f1efc06c8f0 0x7f1efc06edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.026+0000 7f1f17ec1640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f1f10155950 0x7f1f101f19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.026+0000 7f1f17ec1640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f100cc330 0x7f1f101f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.026+0000 7f1f17ec1640 1 --2- 172.21.15.27:0/332669139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f100b5a30 0x7f1f100c7620 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.026+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 >> 172.21.15.27:0/332669139 conn(0x7f1f100676c0 msgr2=0x7f1f10067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:32.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.026+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 shutdown_connections 2024-04-20T11:02:32.026 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.026+0000 7f1f17ec1640 1 -- 172.21.15.27:0/332669139 wait complete. 2024-04-20T11:02:32.036 INFO:tasks.ceph:Scrubbing osd.6 2024-04-20T11:02:32.037 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:02:32.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.140+0000 7f2712281640 1 Processor -- start 2024-04-20T11:02:32.141 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.140+0000 7f2712281640 1 -- start start 2024-04-20T11:02:32.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.141+0000 7f2712281640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c155610 0x7f270c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.142 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.141+0000 7f2712281640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f270c0cb940 0x7f270c06df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.141+0000 7f2712281640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f270c0ce710 con 0x7f270c0cd6e0 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.141+0000 7f2712281640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f270c0c60a0 con 0x7f270c0cb940 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.141+0000 7f2712281640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f270c0c6380 con 0x7f270c155610 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.141+0000 7f271081e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f270c0cb940 0x7f270c06df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.141+0000 7f270bfff640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c155610 0x7f270c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.141+0000 7f270b7fe640 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f270c0cd6e0 0x7f270c15b410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.27:59924/0 (socket says 172.21.15.27:59924) 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.141+0000 7f270b7fe640 1 -- 172.21.15.27:0/3346347701 learned_addr learned my addr 172.21.15.27:0/3346347701 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.142+0000 7f270bfff640 1 -- 172.21.15.27:0/3346347701 >> v2:172.21.15.137:3300/0 conn(0x7f270c0cb940 msgr2=0x7f270c06df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.142+0000 7f270bfff640 1 --2- 172.21.15.27:0/3346347701 >> v2:172.21.15.137:3300/0 conn(0x7f270c0cb940 0x7f270c06df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.142+0000 7f270bfff640 1 -- 172.21.15.27:0/3346347701 >> v1:172.21.15.137:6789/0 conn(0x7f270c0cd6e0 legacy=0x7f270c15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.142+0000 7f270bfff640 1 -- 172.21.15.27:0/3346347701 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c0c6660 con 0x7f270c155610 2024-04-20T11:02:32.143 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.142+0000 7f271081e640 1 --2- 172.21.15.27:0/3346347701 >> v2:172.21.15.137:3300/0 conn(0x7f270c0cb940 0x7f270c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:02:32.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.142+0000 7f270bfff640 1 --2- 172.21.15.27:0/3346347701 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c155610 0x7f270c06da50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0097a0 tx=0x7f26fc02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77a4b838dfe42e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f270affd640 1 -- 172.21.15.27:0/3346347701 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc034070 con 0x7f270c155610 2024-04-20T11:02:32.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f270affd640 1 -- 172.21.15.27:0/3346347701 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26fc030800 con 0x7f270c155610 2024-04-20T11:02:32.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f270affd640 1 -- 172.21.15.27:0/3346347701 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc030bd0 con 0x7f270c155610 2024-04-20T11:02:32.145 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f2712281640 1 -- 172.21.15.27:0/3346347701 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c155610 msgr2=0x7f270c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f2712281640 1 --2- 172.21.15.27:0/3346347701 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c155610 0x7f270c06da50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0097a0 tx=0x7f26fc02b940 comp rx=0 tx=0).stop 2024-04-20T11:02:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f2712281640 1 -- 172.21.15.27:0/3346347701 shutdown_connections 2024-04-20T11:02:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f2712281640 1 --2- 172.21.15.27:0/3346347701 >> v2:172.21.15.137:3300/0 conn(0x7f270c0cb940 0x7f270c06df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f2712281640 1 --2- 172.21.15.27:0/3346347701 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c155610 0x7f270c06da50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f2712281640 1 -- 172.21.15.27:0/3346347701 >> 172.21.15.27:0/3346347701 conn(0x7f270c06c310 msgr2=0x7f270c06cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f2712281640 1 -- 172.21.15.27:0/3346347701 shutdown_connections 2024-04-20T11:02:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.143+0000 7f2712281640 1 -- 172.21.15.27:0/3346347701 wait complete. 2024-04-20T11:02:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.144+0000 7f2712281640 1 Processor -- start 2024-04-20T11:02:32.146 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.144+0000 7f2712281640 1 -- start start 2024-04-20T11:02:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f2712281640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c0cb940 0x7f270c14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f2712281640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f270c0cd6e0 0x7f270c1494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f2712281640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f270c155610 0x7f270c149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f2712281640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f270c155b70 con 0x7f270c0cd6e0 2024-04-20T11:02:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f2712281640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f270c1559f0 con 0x7f270c155610 2024-04-20T11:02:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f2712281640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f270c0cc590 con 0x7f270c0cb940 2024-04-20T11:02:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f270b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f270c0cd6e0 0x7f270c1494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f270b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f270c0cd6e0 0x7f270c1494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38878/0 (socket says 172.21.15.27:38878) 2024-04-20T11:02:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f270b7fe640 1 -- 172.21.15.27:0/1505447368 learned_addr learned my addr 172.21.15.27:0/1505447368 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f270bfff640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c0cb940 0x7f270c14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.145+0000 7f271081e640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f270c155610 0x7f270c149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.150 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f270b7fe640 1 -- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c0cb940 msgr2=0x7f270c14e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f270b7fe640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f270c0cb940 0x7f270c14e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f270b7fe640 1 -- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f270c155610 msgr2=0x7f270c149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f270b7fe640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f270c155610 0x7f270c149a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f270b7fe640 1 -- 172.21.15.27:0/1505447368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c0cfc60 con 0x7f270c0cd6e0 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f270b7fe640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f270c0cd6e0 0x7f270c1494d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f270000c400 tx=0x7f270000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f2708ff9640 1 -- 172.21.15.27:0/1505447368 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700017070 con 0x7f270c0cd6e0 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270c1f12c0 con 0x7f270c0cd6e0 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f2708ff9640 1 -- 172.21.15.27:0/1505447368 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2700005d30 con 0x7f270c0cd6e0 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f2708ff9640 1 -- 172.21.15.27:0/1505447368 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27000072f0 con 0x7f270c0cd6e0 2024-04-20T11:02:32.151 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.146+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270c1f1860 con 0x7f270c0cd6e0 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.147+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f270c04fcf0 con 0x7f270c0cd6e0 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.147+0000 7f2708ff9640 1 -- 172.21.15.27:0/1505447368 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f270001a070 con 0x7f270c0cd6e0 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.148+0000 7f2708ff9640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f26f806c8f0 0x7f26f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.148+0000 7f2708ff9640 1 -- 172.21.15.27:0/1505447368 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f270008b210 con 0x7f270c0cd6e0 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.148+0000 7f2708ff9640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7f26f8074f90 0x7f26f8077410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.148+0000 7f2708ff9640 1 -- 172.21.15.27:0/1505447368 --> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2700013e50 con 0x7f26f8074f90 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.148+0000 7f2708ff9640 1 -- 172.21.15.27:0/1505447368 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2700092050 con 0x7f270c0cd6e0 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.148+0000 7f270bfff640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f26f806c8f0 0x7f26f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.148+0000 7f271081e640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7f26f8074f90 0x7f26f8077410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.149+0000 7f270bfff640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f26f806c8f0 0x7f26f806edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26fc036000 tx=0x7f26fc03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.149+0000 7f271081e640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7f26f8074f90 0x7f26f8077410 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.153 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.150+0000 7f2708ff9640 1 -- 172.21.15.27:0/1505447368 <== osd.6 v2:172.21.15.137:6808/1373372098 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2700013e50 con 0x7f26f8074f90 2024-04-20T11:02:32.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 --> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f270c0cd4c0 con 0x7f26f8074f90 2024-04-20T11:02:32.163 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2708ff9640 1 -- 172.21.15.27:0/1505447368 <== osd.6 v2:172.21.15.137:6808/1373372098 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f270c0cd4c0 con 0x7f26f8074f90 2024-04-20T11:02:32.164 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T11:02:32.164 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:02:32.164 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T11:02:32.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7f26f8074f90 msgr2=0x7f26f8077410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2712281640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6808/1373372098,v1:172.21.15.137:6809/1373372098] conn(0x7f26f8074f90 0x7f26f8077410 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f26f806c8f0 msgr2=0x7f26f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.165 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2712281640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f26f806c8f0 0x7f26f806edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26fc036000 tx=0x7f26fc03b040 comp rx=0 tx=0).stop 2024-04-20T11:02:32.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f270c0cd6e0 msgr2=0x7f270c1494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2712281640 1 --2- 172.21.15.27:0/1505447368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f270c0cd6e0 0x7f270c1494d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f270000c400 tx=0x7f270000c8c0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f271081e640 1 -- 172.21.15.27:0/1505447368 reap_dead start 2024-04-20T11:02:32.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 shutdown_connections 2024-04-20T11:02:32.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.163+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 >> 172.21.15.27:0/1505447368 conn(0x7f270c06c310 msgr2=0x7f270c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:32.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.164+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 shutdown_connections 2024-04-20T11:02:32.166 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.164+0000 7f2712281640 1 -- 172.21.15.27:0/1505447368 wait complete. 2024-04-20T11:02:32.175 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-20T11:02:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.275+0000 7fbbaae0f640 1 Processor -- start 2024-04-20T11:02:32.276 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.275+0000 7fbbaae0f640 1 -- start start 2024-04-20T11:02:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbbaae0f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba4152260 0x7fbba40719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbbaae0f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbba415ba80 con 0x7fbba40cd6f0 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbbaae0f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbba415bd60 con 0x7fbba4152640 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbbaae0f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbba415c040 con 0x7fbba4152260 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbba93ac640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbba40cd6f0 0x7fbba4158760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58678/0 (socket says 172.21.15.27:58678) 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbba93ac640 1 -- 172.21.15.27:0/1126107026 learned_addr learned my addr 172.21.15.27:0/1126107026 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbba8bab640 1 --2- 172.21.15.27:0/1126107026 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba4152260 0x7fbba40719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbb9b7fe640 1 -- 172.21.15.27:0/1126107026 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967875084 0 0) 0x7fbba415ba80 con 0x7fbba40cd6f0 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbb9b7fe640 1 -- 172.21.15.27:0/1126107026 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb8c003520 con 0x7fbba40cd6f0 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbb9b7fe640 1 -- 172.21.15.27:0/1126107026 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1998929453 0 0) 0x7fbba415bd60 con 0x7fbba4152640 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.276+0000 7fbb9b7fe640 1 -- 172.21.15.27:0/1126107026 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbba415ba80 con 0x7fbba4152640 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.277+0000 7fbba8bab640 1 -- 172.21.15.27:0/1126107026 >> v1:172.21.15.137:6789/0 conn(0x7fbba4152640 legacy=0x7fbba40c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.277+0000 7fbb9b7fe640 1 -- 172.21.15.27:0/1126107026 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 592167323 0 0) 0x7fbb8c003520 con 0x7fbba40cd6f0 2024-04-20T11:02:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.277+0000 7fbba8bab640 1 -- 172.21.15.27:0/1126107026 >> v1:172.21.15.27:6789/0 conn(0x7fbba40cd6f0 legacy=0x7fbba4158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.279 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.277+0000 7fbba8bab640 1 -- 172.21.15.27:0/1126107026 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba415c320 con 0x7fbba4152260 2024-04-20T11:02:32.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.277+0000 7fbba8bab640 1 --2- 172.21.15.27:0/1126107026 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba4152260 0x7fbba40719c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbba00097d0 tx=0x7fbba002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db693811c41edcc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbb9b7fe640 1 -- 172.21.15.27:0/1126107026 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0019f0 con 0x7fbba4152260 2024-04-20T11:02:32.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbb9b7fe640 1 -- 172.21.15.27:0/1126107026 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba0037070 con 0x7fbba4152260 2024-04-20T11:02:32.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbb9b7fe640 1 -- 172.21.15.27:0/1126107026 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba00317a0 con 0x7fbba4152260 2024-04-20T11:02:32.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbbaae0f640 1 -- 172.21.15.27:0/1126107026 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba4152260 msgr2=0x7fbba40719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbbaae0f640 1 --2- 172.21.15.27:0/1126107026 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba4152260 0x7fbba40719c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbba00097d0 tx=0x7fbba002b8a0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.280 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbbaae0f640 1 -- 172.21.15.27:0/1126107026 shutdown_connections 2024-04-20T11:02:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbbaae0f640 1 --2- 172.21.15.27:0/1126107026 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba4152260 0x7fbba40719c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbbaae0f640 1 -- 172.21.15.27:0/1126107026 >> 172.21.15.27:0/1126107026 conn(0x7fbba40ccbd0 msgr2=0x7fbba40ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbbaae0f640 1 -- 172.21.15.27:0/1126107026 shutdown_connections 2024-04-20T11:02:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.278+0000 7fbbaae0f640 1 -- 172.21.15.27:0/1126107026 wait complete. 2024-04-20T11:02:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.279+0000 7fbbaae0f640 1 Processor -- start 2024-04-20T11:02:32.281 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.279+0000 7fbbaae0f640 1 -- start start 2024-04-20T11:02:32.283 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbbaae0f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba40cd6f0 0x7fbba41478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbbaae0f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbba4152260 0x7fbba41f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbbaae0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbba4152640 0x7fbba41f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbbaae0f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbba415c600 con 0x7fbba4152260 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbbaae0f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbba4074260 con 0x7fbba4152640 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbbaae0f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fbba415b720 con 0x7fbba40cd6f0 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbba4152260 0x7fbba41f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbba4152260 0x7fbba41f4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38888/0 (socket says 172.21.15.27:38888) 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbb9bfff640 1 -- 172.21.15.27:0/2972013166 learned_addr learned my addr 172.21.15.27:0/2972013166 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.280+0000 7fbba8bab640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba40cd6f0 0x7fbba41478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbba93ac640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbba4152640 0x7fbba41f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbb9bfff640 1 -- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba40cd6f0 msgr2=0x7fbba41478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.284 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbb9bfff640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba40cd6f0 0x7fbba41478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbb9bfff640 1 -- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbba4152640 msgr2=0x7fbba41f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbb9bfff640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbba4152640 0x7fbba41f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbb9bfff640 1 -- 172.21.15.27:0/2972013166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba4157b00 con 0x7fbba4152260 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbb9bfff640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbba4152260 0x7fbba41f4240 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400c4c0 tx=0x7fbb9400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbb997fa640 1 -- 172.21.15.27:0/2972013166 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb940052d0 con 0x7fbba4152260 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba40c7260 con 0x7fbba4152260 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbb997fa640 1 -- 172.21.15.27:0/2972013166 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb94005470 con 0x7fbba4152260 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbb997fa640 1 -- 172.21.15.27:0/2972013166 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94012870 con 0x7fbba4152260 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.281+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba40c7520 con 0x7fbba4152260 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.282+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba404fcf0 con 0x7fbba4152260 2024-04-20T11:02:32.285 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.283+0000 7fbb997fa640 1 -- 172.21.15.27:0/2972013166 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb94012a10 con 0x7fbba4152260 2024-04-20T11:02:32.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.283+0000 7fbb997fa640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbb9006c920 0x7fbb9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.283+0000 7fbb997fa640 1 -- 172.21.15.27:0/2972013166 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fbb9408ba10 con 0x7fbba4152260 2024-04-20T11:02:32.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.283+0000 7fbba8bab640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbb9006c920 0x7fbb9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.286 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.284+0000 7fbba8bab640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbb9006c920 0x7fbb9006ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbba0036000 tx=0x7fbba003d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.288 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.288+0000 7fbb997fa640 1 -- 172.21.15.27:0/2972013166 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbb94059c30 con 0x7fbba4152260 2024-04-20T11:02:32.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.480+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fbba4157fe0 con 0x7fbb9006c920 2024-04-20T11:02:32.481 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.480+0000 7fbb997fa640 1 -- 172.21.15.27:0/2972013166 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbba0037040 con 0x7fbb9006c920 2024-04-20T11:02:32.481 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 6 to deep-scrub 2024-04-20T11:02:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.482+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbb9006c920 msgr2=0x7fbb9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.483 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.482+0000 7fbbaae0f640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbb9006c920 0x7fbb9006ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbba0036000 tx=0x7fbba003d040 comp rx=0 tx=0).stop 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.482+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbba4152260 msgr2=0x7fbba41f4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.482+0000 7fbbaae0f640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbba4152260 0x7fbba41f4240 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400c4c0 tx=0x7fbb9400c980 comp rx=0 tx=0).stop 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.483+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 shutdown_connections 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.483+0000 7fbbaae0f640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fbb9006c920 0x7fbb9006ede0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.483+0000 7fbbaae0f640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbba4152640 0x7fbba41f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.483+0000 7fbbaae0f640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbba4152260 0x7fbba41f4240 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.483+0000 7fbbaae0f640 1 --2- 172.21.15.27:0/2972013166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fbba40cd6f0 0x7fbba41478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.483+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 >> 172.21.15.27:0/2972013166 conn(0x7fbba40ccbd0 msgr2=0x7fbba40c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.483+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 shutdown_connections 2024-04-20T11:02:32.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.483+0000 7fbbaae0f640 1 -- 172.21.15.27:0/2972013166 wait complete. 2024-04-20T11:02:32.494 INFO:tasks.ceph:Scrubbing osd.7 2024-04-20T11:02:32.495 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-20T11:02:32.602 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.601+0000 7efc90b92640 1 Processor -- start 2024-04-20T11:02:32.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.601+0000 7efc90b92640 1 -- start start 2024-04-20T11:02:32.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.602+0000 7efc90b92640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efc8c0cd6e0 0x7efc8c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.602+0000 7efc90b92640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7efc8c0cb940 0x7efc8c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.605 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.602+0000 7efc90b92640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8c0ce710 con 0x7efc8c155610 2024-04-20T11:02:32.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.602+0000 7efc90b92640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7efc8c15be90 con 0x7efc8c0cd6e0 2024-04-20T11:02:32.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.602+0000 7efc90b92640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7efc8c0c60a0 con 0x7efc8c0cb940 2024-04-20T11:02:32.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.602+0000 7efc83fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efc8c0cd6e0 0x7efc8c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.602+0000 7efc83fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efc8c0cd6e0 0x7efc8c15b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38890/0 (socket says 172.21.15.27:38890) 2024-04-20T11:02:32.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.602+0000 7efc83fff640 1 -- 172.21.15.27:0/3789394735 learned_addr learned my addr 172.21.15.27:0/3789394735 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:32.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.602+0000 7efc8a59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efc8c155610 0x7efc8c06da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58682/0 (socket says 172.21.15.27:58682) 2024-04-20T11:02:32.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.603+0000 7efc8ad9d640 1 --2- 172.21.15.27:0/3789394735 >> v2:172.21.15.137:3300/0 conn(0x7efc8c0cb940 0x7efc8c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.603+0000 7efc83fff640 1 -- 172.21.15.27:0/3789394735 >> v2:172.21.15.137:3300/0 conn(0x7efc8c0cb940 msgr2=0x7efc8c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:32.606 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.603+0000 7efc83fff640 1 --2- 172.21.15.27:0/3789394735 >> v2:172.21.15.137:3300/0 conn(0x7efc8c0cb940 0x7efc8c15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.603+0000 7efc83fff640 1 -- 172.21.15.27:0/3789394735 >> v1:172.21.15.27:6789/0 conn(0x7efc8c155610 legacy=0x7efc8c06da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.603+0000 7efc83fff640 1 -- 172.21.15.27:0/3789394735 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c0c6380 con 0x7efc8c0cd6e0 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.603+0000 7efc89d9b640 1 -- 172.21.15.27:0/3789394735 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819539566 0 0) 0x7efc8c0ce710 con 0x7efc8c155610 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.603+0000 7efc83fff640 1 --2- 172.21.15.27:0/3789394735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c0cd6e0 0x7efc8c15b410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0b3a40 tx=0x7efc7402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95783d23bc34b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc89d9b640 1 -- 172.21.15.27:0/3789394735 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74004230 con 0x7efc8c0cd6e0 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc89d9b640 1 -- 172.21.15.27:0/3789394735 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc740043d0 con 0x7efc8c0cd6e0 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc89d9b640 1 -- 172.21.15.27:0/3789394735 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7402f840 con 0x7efc8c0cd6e0 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc90b92640 1 -- 172.21.15.27:0/3789394735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c0cd6e0 msgr2=0x7efc8c15b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc90b92640 1 --2- 172.21.15.27:0/3789394735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c0cd6e0 0x7efc8c15b410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0b3a40 tx=0x7efc7402ba20 comp rx=0 tx=0).stop 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc90b92640 1 -- 172.21.15.27:0/3789394735 shutdown_connections 2024-04-20T11:02:32.607 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc90b92640 1 --2- 172.21.15.27:0/3789394735 >> v2:172.21.15.137:3300/0 conn(0x7efc8c0cb940 0x7efc8c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc90b92640 1 --2- 172.21.15.27:0/3789394735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c0cd6e0 0x7efc8c15b410 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc90b92640 1 -- 172.21.15.27:0/3789394735 >> 172.21.15.27:0/3789394735 conn(0x7efc8c06c310 msgr2=0x7efc8c06cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:32.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc90b92640 1 -- 172.21.15.27:0/3789394735 shutdown_connections 2024-04-20T11:02:32.608 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.604+0000 7efc90b92640 1 -- 172.21.15.27:0/3789394735 wait complete. 2024-04-20T11:02:32.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.605+0000 7efc90b92640 1 Processor -- start 2024-04-20T11:02:32.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc90b92640 1 -- start start 2024-04-20T11:02:32.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc90b92640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc8c0cb940 0x7efc8c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc90b92640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efc8c0cd6e0 0x7efc8c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc90b92640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c155610 0x7efc8c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc90b92640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c0ce4c0 con 0x7efc8c155610 2024-04-20T11:02:32.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc90b92640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c0ce340 con 0x7efc8c0cb940 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc90b92640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7efc8c06e160 con 0x7efc8c0cd6e0 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc8ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c155610 0x7efc8c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc8ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c155610 0x7efc8c0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38892/0 (socket says 172.21.15.27:38892) 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.606+0000 7efc8ad9d640 1 -- 172.21.15.27:0/1281132246 learned_addr learned my addr 172.21.15.27:0/1281132246 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc83fff640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efc8c0cd6e0 0x7efc8c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc8a59c640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc8c0cb940 0x7efc8c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc8ad9d640 1 -- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efc8c0cd6e0 msgr2=0x7efc8c155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc8ad9d640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7efc8c0cd6e0 0x7efc8c155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc8ad9d640 1 -- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc8c0cb940 msgr2=0x7efc8c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc8ad9d640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc8c0cb940 0x7efc8c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc8ad9d640 1 -- 172.21.15.27:0/1281132246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c158750 con 0x7efc8c155610 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc8ad9d640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c155610 0x7efc8c0d1100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efc8400cc30 tx=0x7efc8400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc82ffd640 1 -- 172.21.15.27:0/1281132246 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8400ce40 con 0x7efc8c155610 2024-04-20T11:02:32.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc82ffd640 1 -- 172.21.15.27:0/1281132246 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc84003990 con 0x7efc8c155610 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc82ffd640 1 -- 172.21.15.27:0/1281132246 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84003d40 con 0x7efc8c155610 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8c158a30 con 0x7efc8c155610 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.607+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8c0d0900 con 0x7efc8c155610 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.608+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efc8c158500 con 0x7efc8c155610 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.608+0000 7efc82ffd640 1 -- 172.21.15.27:0/1281132246 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc84003ee0 con 0x7efc8c155610 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.609+0000 7efc82ffd640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efc6c06c8f0 0x7efc6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.609+0000 7efc82ffd640 1 -- 172.21.15.27:0/1281132246 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7efc8408c1c0 con 0x7efc8c155610 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.609+0000 7efc8a59c640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efc6c06c8f0 0x7efc6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.609+0000 7efc82ffd640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7efc6c074fa0 0x7efc6c077420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.609+0000 7efc82ffd640 1 -- 172.21.15.27:0/1281132246 --> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc8400de50 con 0x7efc6c074fa0 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.609+0000 7efc82ffd640 1 -- 172.21.15.27:0/1281132246 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc84093050 con 0x7efc8c155610 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.609+0000 7efc83fff640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7efc6c074fa0 0x7efc6c077420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.610+0000 7efc8a59c640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efc6c06c8f0 0x7efc6c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efc7c000f60 tx=0x7efc7c007b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.614 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.610+0000 7efc83fff640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7efc6c074fa0 0x7efc6c077420 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.615 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.611+0000 7efc82ffd640 1 -- 172.21.15.27:0/1281132246 <== osd.7 v2:172.21.15.137:6816/460941503 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efc8400de50 con 0x7efc6c074fa0 2024-04-20T11:02:32.624 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.624+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 --> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efc8c15c020 con 0x7efc6c074fa0 2024-04-20T11:02:32.625 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.624+0000 7efc82ffd640 1 -- 172.21.15.27:0/1281132246 <== osd.7 v2:172.21.15.137:6816/460941503 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efc8c15c020 con 0x7efc6c074fa0 2024-04-20T11:02:32.625 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-04-20T11:02:32.625 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T11:02:32.625 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-04-20T11:02:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.624+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7efc6c074fa0 msgr2=0x7efc6c077420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.624+0000 7efc90b92640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6816/460941503,v1:172.21.15.137:6817/460941503] conn(0x7efc6c074fa0 0x7efc6c077420 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.625+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efc6c06c8f0 msgr2=0x7efc6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.625+0000 7efc90b92640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7efc6c06c8f0 0x7efc6c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efc7c000f60 tx=0x7efc7c007b50 comp rx=0 tx=0).stop 2024-04-20T11:02:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.625+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c155610 msgr2=0x7efc8c0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.625+0000 7efc90b92640 1 --2- 172.21.15.27:0/1281132246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc8c155610 0x7efc8c0d1100 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7efc8400cc30 tx=0x7efc8400a5c0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.626 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.625+0000 7efc8ad9d640 1 -- 172.21.15.27:0/1281132246 reap_dead start 2024-04-20T11:02:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.625+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 shutdown_connections 2024-04-20T11:02:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.625+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 >> 172.21.15.27:0/1281132246 conn(0x7efc8c06c310 msgr2=0x7efc8c0cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.625+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 shutdown_connections 2024-04-20T11:02:32.627 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.625+0000 7efc90b92640 1 -- 172.21.15.27:0/1281132246 wait complete. 2024-04-20T11:02:32.636 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-20T11:02:32.739 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.737+0000 7f398ae5c640 1 Processor -- start 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.737+0000 7f398ae5c640 1 -- start start 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.738+0000 7f398ae5c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f3984155950 0x7f39840cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.738+0000 7f398ae5c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39840b5c00 0x7f39840d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.738+0000 7f398ae5c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398415b9c0 con 0x7f39840cc330 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.738+0000 7f398ae5c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f398415bca0 con 0x7f39840b5c00 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.738+0000 7f398ae5c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f398415bf80 con 0x7f3984155950 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.738+0000 7f39893f9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39840b5c00 0x7f39840d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.738+0000 7f39893f9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39840b5c00 0x7f39840d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38902/0 (socket says 172.21.15.27:38902) 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.739+0000 7f39893f9640 1 -- 172.21.15.27:0/2113311411 learned_addr learned my addr 172.21.15.27:0/2113311411 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.739+0000 7f3988bf8640 1 --2- 172.21.15.27:0/2113311411 >> v2:172.21.15.137:3300/0 conn(0x7f3984155950 0x7f39840cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.739+0000 7f39893f9640 1 -- 172.21.15.27:0/2113311411 >> v2:172.21.15.137:3300/0 conn(0x7f3984155950 msgr2=0x7f39840cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:32.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.739+0000 7f39893f9640 1 --2- 172.21.15.27:0/2113311411 >> v2:172.21.15.137:3300/0 conn(0x7f3984155950 0x7f39840cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.739+0000 7f39893f9640 1 -- 172.21.15.27:0/2113311411 >> v1:172.21.15.137:6789/0 conn(0x7f39840cc330 legacy=0x7f39840b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.741 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.739+0000 7f39893f9640 1 -- 172.21.15.27:0/2113311411 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398415c260 con 0x7f39840b5c00 2024-04-20T11:02:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f39893f9640 1 --2- 172.21.15.27:0/2113311411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39840b5c00 0x7f39840d0c90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f39840ce7c0 tx=0x7f397402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8725e8d2f71b9c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f39837fe640 1 -- 172.21.15.27:0/2113311411 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397402f850 con 0x7f39840b5c00 2024-04-20T11:02:32.742 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f39837fe640 1 -- 172.21.15.27:0/2113311411 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397402f9f0 con 0x7f39840b5c00 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f39837fe640 1 -- 172.21.15.27:0/2113311411 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397402fdc0 con 0x7f39840b5c00 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f398ae5c640 1 -- 172.21.15.27:0/2113311411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39840b5c00 msgr2=0x7f39840d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f398ae5c640 1 --2- 172.21.15.27:0/2113311411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39840b5c00 0x7f39840d0c90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f39840ce7c0 tx=0x7f397402ba30 comp rx=0 tx=0).stop 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f398ae5c640 1 -- 172.21.15.27:0/2113311411 shutdown_connections 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f398ae5c640 1 --2- 172.21.15.27:0/2113311411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39840b5c00 0x7f39840d0c90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f398ae5c640 1 --2- 172.21.15.27:0/2113311411 >> v2:172.21.15.137:3300/0 conn(0x7f3984155950 0x7f39840cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f398ae5c640 1 -- 172.21.15.27:0/2113311411 >> 172.21.15.27:0/2113311411 conn(0x7f39840676c0 msgr2=0x7f3984067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f398ae5c640 1 -- 172.21.15.27:0/2113311411 shutdown_connections 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.740+0000 7f398ae5c640 1 -- 172.21.15.27:0/2113311411 wait complete. 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.742+0000 7f398ae5c640 1 Processor -- start 2024-04-20T11:02:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.742+0000 7f398ae5c640 1 -- start start 2024-04-20T11:02:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.742+0000 7f398ae5c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39840b5c00 0x7f3984150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.742+0000 7f398ae5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39840cc330 0x7f3984150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.742+0000 7f398ae5c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984155950 0x7f398414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.742+0000 7f398ae5c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f398415fd10 con 0x7f3984155950 2024-04-20T11:02:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.742+0000 7f398ae5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f398415fb90 con 0x7f39840cc330 2024-04-20T11:02:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.742+0000 7f398ae5c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f3984155d30 con 0x7f39840b5c00 2024-04-20T11:02:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39893f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984155950 0x7f398414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39893f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984155950 0x7f398414e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38908/0 (socket says 172.21.15.27:38908) 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39893f9640 1 -- 172.21.15.27:0/109612008 learned_addr learned my addr 172.21.15.27:0/109612008 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f3983fff640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39840cc330 0x7f3984150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f3988bf8640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39840b5c00 0x7f3984150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39893f9640 1 -- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39840b5c00 msgr2=0x7f3984150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39893f9640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39840b5c00 0x7f3984150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39893f9640 1 -- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39840cc330 msgr2=0x7f3984150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39893f9640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39840cc330 0x7f3984150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39893f9640 1 -- 172.21.15.27:0/109612008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398415e250 con 0x7f3984155950 2024-04-20T11:02:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39893f9640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984155950 0x7f398414e600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f397402ba70 tx=0x7f3974004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39817fa640 1 -- 172.21.15.27:0/109612008 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974037c40 con 0x7f3984155950 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39817fa640 1 -- 172.21.15.27:0/109612008 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3974046070 con 0x7f3984155950 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398415dc90 con 0x7f3984155950 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f39817fa640 1 -- 172.21.15.27:0/109612008 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974041400 con 0x7f3984155950 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.743+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398414f550 con 0x7f3984155950 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.744+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398404fcf0 con 0x7f3984155950 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.745+0000 7f39817fa640 1 -- 172.21.15.27:0/109612008 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3974040070 con 0x7f3984155950 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.745+0000 7f39817fa640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f397006c920 0x7f397006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.745+0000 7f39817fa640 1 -- 172.21.15.27:0/109612008 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f39740ae120 con 0x7f3984155950 2024-04-20T11:02:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.745+0000 7f3988bf8640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f397006c920 0x7f397006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:32.749 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.746+0000 7f3988bf8640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f397006c920 0x7f397006ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f397c0113e0 tx=0x7f397c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.750+0000 7f39817fa640 1 -- 172.21.15.27:0/109612008 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3974033030 con 0x7f3984155950 2024-04-20T11:02:32.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.935+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f39840cd0a0 con 0x7f397006c920 2024-04-20T11:02:32.936 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.936+0000 7f39817fa640 1 -- 172.21.15.27:0/109612008 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f39840cd0a0 con 0x7f397006c920 2024-04-20T11:02:32.937 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 7 to deep-scrub 2024-04-20T11:02:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f397006c920 msgr2=0x7f397006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f397006c920 0x7f397006ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f397c0113e0 tx=0x7f397c00b040 comp rx=0 tx=0).stop 2024-04-20T11:02:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984155950 msgr2=0x7f398414e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984155950 0x7f398414e600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f397402ba70 tx=0x7f3974004730 comp rx=0 tx=0).stop 2024-04-20T11:02:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 shutdown_connections 2024-04-20T11:02:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f397006c920 0x7f397006ede0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3984155950 0x7f398414e600 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39840cc330 0x7f3984150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 --2- 172.21.15.27:0/109612008 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f39840b5c00 0x7f3984150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 >> 172.21.15.27:0/109612008 conn(0x7f39840676c0 msgr2=0x7f39840d0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 shutdown_connections 2024-04-20T11:02:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:32.938+0000 7f398ae5c640 1 -- 172.21.15.27:0/109612008 wait complete. 2024-04-20T11:02:32.950 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:02:33.050 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.050+0000 7fdc56552640 1 Processor -- start 2024-04-20T11:02:33.051 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.050+0000 7fdc56552640 1 -- start start 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.050+0000 7fdc56552640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdc50152260 0x7fdc500719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.050+0000 7fdc56552640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 0x7fdc500c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.050+0000 7fdc56552640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5015ba80 con 0x7fdc500cd6f0 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.050+0000 7fdc56552640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5015bd60 con 0x7fdc50152260 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.050+0000 7fdc56552640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5015c040 con 0x7fdc50152640 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.051+0000 7fdc4ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdc50152260 0x7fdc500719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.051+0000 7fdc4ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdc50152260 0x7fdc500719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38916/0 (socket says 172.21.15.27:38916) 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.051+0000 7fdc4ffff640 1 -- 172.21.15.27:0/2602113128 learned_addr learned my addr 172.21.15.27:0/2602113128 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.051+0000 7fdc4ffff640 1 -- 172.21.15.27:0/2602113128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 msgr2=0x7fdc500c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.051+0000 7fdc4f7fe640 1 --2- 172.21.15.27:0/2602113128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 0x7fdc500c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.051+0000 7fdc4ffff640 1 --2- 172.21.15.27:0/2602113128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 0x7fdc500c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.051+0000 7fdc4ffff640 1 -- 172.21.15.27:0/2602113128 >> v1:172.21.15.137:6789/0 conn(0x7fdc500cd6f0 legacy=0x7fdc50158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.051+0000 7fdc4ffff640 1 -- 172.21.15.27:0/2602113128 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5015c320 con 0x7fdc50152260 2024-04-20T11:02:33.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.052+0000 7fdc4ffff640 1 --2- 172.21.15.27:0/2602113128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc50152260 0x7fdc500719c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009870 tx=0x7fdc4002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a9f8ffe58f632a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:33.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.052+0000 7fdc4effd640 1 -- 172.21.15.27:0/2602113128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40034070 con 0x7fdc50152260 2024-04-20T11:02:33.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.052+0000 7fdc4effd640 1 -- 172.21.15.27:0/2602113128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc40030840 con 0x7fdc50152260 2024-04-20T11:02:33.054 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.052+0000 7fdc4effd640 1 -- 172.21.15.27:0/2602113128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40030c10 con 0x7fdc50152260 2024-04-20T11:02:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.052+0000 7fdc56552640 1 -- 172.21.15.27:0/2602113128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc50152260 msgr2=0x7fdc500719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.052+0000 7fdc56552640 1 --2- 172.21.15.27:0/2602113128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc50152260 0x7fdc500719c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009870 tx=0x7fdc4002b980 comp rx=0 tx=0).stop 2024-04-20T11:02:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.052+0000 7fdc56552640 1 -- 172.21.15.27:0/2602113128 shutdown_connections 2024-04-20T11:02:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.052+0000 7fdc56552640 1 --2- 172.21.15.27:0/2602113128 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 0x7fdc500c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.052+0000 7fdc56552640 1 --2- 172.21.15.27:0/2602113128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc50152260 0x7fdc500719c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.053+0000 7fdc56552640 1 -- 172.21.15.27:0/2602113128 >> 172.21.15.27:0/2602113128 conn(0x7fdc500ccbd0 msgr2=0x7fdc500ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.053+0000 7fdc56552640 1 -- 172.21.15.27:0/2602113128 shutdown_connections 2024-04-20T11:02:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.053+0000 7fdc56552640 1 -- 172.21.15.27:0/2602113128 wait complete. 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.054+0000 7fdc56552640 1 Processor -- start 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.054+0000 7fdc56552640 1 -- start start 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.054+0000 7fdc56552640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500cd6f0 0x7fdc5015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.054+0000 7fdc56552640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50152260 0x7fdc50156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.054+0000 7fdc56552640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 0x7fdc5006b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc56552640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5015c600 con 0x7fdc500cd6f0 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc56552640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50074260 con 0x7fdc50152260 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc56552640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5015b720 con 0x7fdc50152640 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500cd6f0 0x7fdc5015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500cd6f0 0x7fdc5015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38930/0 (socket says 172.21.15.27:38930) 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4ffff640 1 -- 172.21.15.27:0/2536454105 learned_addr learned my addr 172.21.15.27:0/2536454105 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4f7fe640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50152260 0x7fdc50156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4ffff640 1 -- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 msgr2=0x7fdc5006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc54aef640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 0x7fdc5006b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4ffff640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 0x7fdc5006b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4ffff640 1 -- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50152260 msgr2=0x7fdc50156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4ffff640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50152260 0x7fdc50156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4ffff640 1 -- 172.21.15.27:0/2536454105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5014be70 con 0x7fdc500cd6f0 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4f7fe640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50152260 0x7fdc50156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4ffff640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500cd6f0 0x7fdc5015a8f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdc40000c00 tx=0x7fdc40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4cff9640 1 -- 172.21.15.27:0/2536454105 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40048070 con 0x7fdc500cd6f0 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4cff9640 1 -- 172.21.15.27:0/2536454105 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc40030640 con 0x7fdc500cd6f0 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc50151400 con 0x7fdc500cd6f0 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc4cff9640 1 -- 172.21.15.27:0/2536454105 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40034040 con 0x7fdc500cd6f0 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.055+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc501519c0 con 0x7fdc500cd6f0 2024-04-20T11:02:33.059 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.056+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5014c2e0 con 0x7fdc500cd6f0 2024-04-20T11:02:33.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.057+0000 7fdc4cff9640 1 -- 172.21.15.27:0/2536454105 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc4004b070 con 0x7fdc500cd6f0 2024-04-20T11:02:33.060 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.057+0000 7fdc4cff9640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.058+0000 7fdc4cff9640 1 -- 172.21.15.27:0/2536454105 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fdc400adda0 con 0x7fdc500cd6f0 2024-04-20T11:02:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.058+0000 7fdc4f7fe640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:33.061 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.058+0000 7fdc4f7fe640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc50146af0 tx=0x7fdc44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:33.062 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.062+0000 7fdc4cff9640 1 -- 172.21.15.27:0/2536454105 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdc5014c2e0 con 0x7fdc500cd6f0 2024-04-20T11:02:33.247 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.246+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc500c4620 con 0x7fdc3c06c8f0 2024-04-20T11:02:33.253 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.253+0000 7fdc4cff9640 1 -- 172.21.15.27:0/2536454105 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253470 (secure 0 0 0) 0x7fdc44008580 con 0x7fdc3c06c8f0 2024-04-20T11:02:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:33.258 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdc3c06c8f0 msgr2=0x7fdc3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc50146af0 tx=0x7fdc44008040 comp rx=0 tx=0).stop 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500cd6f0 msgr2=0x7fdc5015a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500cd6f0 0x7fdc5015a8f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdc40000c00 tx=0x7fdc40004730 comp rx=0 tx=0).stop 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 shutdown_connections 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fdc3c06c8f0 0x7fdc3c06edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fdc50152640 0x7fdc5006b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc50152260 0x7fdc50156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 --2- 172.21.15.27:0/2536454105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500cd6f0 0x7fdc5015a8f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 >> 172.21.15.27:0/2536454105 conn(0x7fdc500ccbd0 msgr2=0x7fdc500c5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 shutdown_connections 2024-04-20T11:02:33.260 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:33.256+0000 7fdc56552640 1 -- 172.21.15.27:0/2536454105 wait complete. 2024-04-20T11:02:33.268 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-04-20T11:02:31.794262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":152723592,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":1845,"num_read_kb":3292270,"num_write":30950,"num_write_kb":4353509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34983,"ondisk_log_size":34983,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":659848,"kb_used_data":300808,"kb_used_omap":12,"kb_used_meta":358963,"kb_avail":754314872,"statfs":{"total":773094113280,"available":772418428928,"internally_reserved":0,"allocated":308027392,"data_stored":306366936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":367578673},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2726,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003351"},"pg_stats":[{"pgid":"3.3f","version":"27'425","reported_seq":468,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.656501+0000","last_change":"2024-04-20T10:55:42.802900+0000","last_active":"2024-04-20T11:02:26.656501+0000","last_peered":"2024-04-20T11:02:26.656501+0000","last_clean":"2024-04-20T11:02:26.656501+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T11:02:26.656501+0000","last_undegraded":"2024-04-20T11:02:26.656501+0000","last_fullsized":"2024-04-20T11:02:26.656501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:25:30.882912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":31614,"num_write":375,"num_write_kb":59119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"27'429","reported_seq":474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.635145+0000","last_change":"2024-04-20T10:55:42.794956+0000","last_active":"2024-04-20T11:02:26.635145+0000","last_peered":"2024-04-20T11:02:26.635145+0000","last_clean":"2024-04-20T11:02:26.635145+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T11:02:26.635145+0000","last_undegraded":"2024-04-20T11:02:26.635145+0000","last_fullsized":"2024-04-20T11:02:26.635145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:47:03.798136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":40656,"num_write":384,"num_write_kb":54681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"27'406","reported_seq":455,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.647261+0000","last_change":"2024-04-20T10:55:42.798492+0000","last_active":"2024-04-20T11:02:26.647261+0000","last_peered":"2024-04-20T11:02:26.647261+0000","last_clean":"2024-04-20T11:02:26.647261+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T11:02:26.647261+0000","last_undegraded":"2024-04-20T11:02:26.647261+0000","last_fullsized":"2024-04-20T11:02:26.647261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":42823,"num_write":366,"num_write_kb":47539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'392","reported_seq":434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.655198+0000","last_change":"2024-04-20T10:55:42.796800+0000","last_active":"2024-04-20T11:02:26.655198+0000","last_peered":"2024-04-20T11:02:26.655198+0000","last_clean":"2024-04-20T11:02:26.655198+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T11:02:26.655198+0000","last_undegraded":"2024-04-20T11:02:26.655198+0000","last_fullsized":"2024-04-20T11:02:26.655198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:16:57.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":22108,"num_write":353,"num_write_kb":40048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'430","reported_seq":476,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.650261+0000","last_change":"2024-04-20T10:55:42.799984+0000","last_active":"2024-04-20T11:02:26.650261+0000","last_peered":"2024-04-20T11:02:26.650261+0000","last_clean":"2024-04-20T11:02:26.650261+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T11:02:26.650261+0000","last_undegraded":"2024-04-20T11:02:26.650261+0000","last_fullsized":"2024-04-20T11:02:26.650261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:23:01.311298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":63832,"num_write":387,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'371","reported_seq":405,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.655513+0000","last_change":"2024-04-20T10:55:42.793746+0000","last_active":"2024-04-20T11:02:26.655513+0000","last_peered":"2024-04-20T11:02:26.655513+0000","last_clean":"2024-04-20T11:02:26.655513+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T11:02:26.655513+0000","last_undegraded":"2024-04-20T11:02:26.655513+0000","last_fullsized":"2024-04-20T11:02:26.655513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:21:10.346457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":22263,"num_write":332,"num_write_kb":22043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'436","reported_seq":483,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.648107+0000","last_change":"2024-04-20T10:55:42.805623+0000","last_active":"2024-04-20T11:02:26.648107+0000","last_peered":"2024-04-20T11:02:26.648107+0000","last_clean":"2024-04-20T11:02:26.648107+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T11:02:26.648107+0000","last_undegraded":"2024-04-20T11:02:26.648107+0000","last_fullsized":"2024-04-20T11:02:26.648107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:02:19.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":74379,"num_write":399,"num_write_kb":92171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'369","reported_seq":409,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.655646+0000","last_change":"2024-04-20T10:55:42.794365+0000","last_active":"2024-04-20T11:02:26.655646+0000","last_peered":"2024-04-20T11:02:26.655646+0000","last_clean":"2024-04-20T11:02:26.655646+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T11:02:26.655646+0000","last_undegraded":"2024-04-20T11:02:26.655646+0000","last_fullsized":"2024-04-20T11:02:26.655646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:57:03.024638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":32797,"num_write":314,"num_write_kb":38008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"27'433","reported_seq":479,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.659884+0000","last_change":"2024-04-20T10:55:42.806996+0000","last_active":"2024-04-20T11:02:26.659884+0000","last_peered":"2024-04-20T11:02:26.659884+0000","last_clean":"2024-04-20T11:02:26.659884+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T11:02:26.659884+0000","last_undegraded":"2024-04-20T11:02:26.659884+0000","last_fullsized":"2024-04-20T11:02:26.659884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:04:03.426584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":52787,"num_write":396,"num_write_kb":59707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"27'425","reported_seq":481,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.649215+0000","last_change":"2024-04-20T10:56:22.913401+0000","last_active":"2024-04-20T11:02:26.649215+0000","last_peered":"2024-04-20T11:02:26.649215+0000","last_clean":"2024-04-20T11:02:26.649215+0000","last_became_active":"2024-04-20T10:56:22.913113+0000","last_became_peered":"2024-04-20T10:56:22.913113+0000","last_unstale":"2024-04-20T11:02:26.649215+0000","last_undegraded":"2024-04-20T11:02:26.649215+0000","last_fullsized":"2024-04-20T11:02:26.649215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:17.248882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":77299,"num_write":373,"num_write_kb":63557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"27'387","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.641786+0000","last_change":"2024-04-20T10:55:42.802767+0000","last_active":"2024-04-20T11:02:26.641786+0000","last_peered":"2024-04-20T11:02:26.641786+0000","last_clean":"2024-04-20T11:02:26.641786+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T11:02:26.641786+0000","last_undegraded":"2024-04-20T11:02:26.641786+0000","last_fullsized":"2024-04-20T11:02:26.641786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:33:16.913612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":50462,"num_write":358,"num_write_kb":46678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'394","reported_seq":441,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.644814+0000","last_change":"2024-04-20T10:55:42.800071+0000","last_active":"2024-04-20T11:02:26.644814+0000","last_peered":"2024-04-20T11:02:26.644814+0000","last_clean":"2024-04-20T11:02:26.644814+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T11:02:26.644814+0000","last_undegraded":"2024-04-20T11:02:26.644814+0000","last_fullsized":"2024-04-20T11:02:26.644814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:42:08.629612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":64677,"num_write":357,"num_write_kb":71584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"27'415","reported_seq":464,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.642019+0000","last_change":"2024-04-20T10:55:42.803705+0000","last_active":"2024-04-20T11:02:26.642019+0000","last_peered":"2024-04-20T11:02:26.642019+0000","last_clean":"2024-04-20T11:02:26.642019+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T11:02:26.642019+0000","last_undegraded":"2024-04-20T11:02:26.642019+0000","last_fullsized":"2024-04-20T11:02:26.642019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:27.208806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":59589,"num_write":367,"num_write_kb":71789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"27'421","reported_seq":459,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.646741+0000","last_change":"2024-04-20T10:55:42.802397+0000","last_active":"2024-04-20T11:02:26.646741+0000","last_peered":"2024-04-20T11:02:26.646741+0000","last_clean":"2024-04-20T11:02:26.646741+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T11:02:26.646741+0000","last_undegraded":"2024-04-20T11:02:26.646741+0000","last_fullsized":"2024-04-20T11:02:26.646741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":30435,"num_write":362,"num_write_kb":42694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"27'419","reported_seq":460,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.643799+0000","last_change":"2024-04-20T10:55:42.800854+0000","last_active":"2024-04-20T11:02:26.643799+0000","last_peered":"2024-04-20T11:02:26.643799+0000","last_clean":"2024-04-20T11:02:26.643799+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T11:02:26.643799+0000","last_undegraded":"2024-04-20T11:02:26.643799+0000","last_fullsized":"2024-04-20T11:02:26.643799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:17.426820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":33945,"num_write":377,"num_write_kb":75986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"27'451","reported_seq":504,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.655478+0000","last_change":"2024-04-20T10:55:42.796473+0000","last_active":"2024-04-20T11:02:26.655478+0000","last_peered":"2024-04-20T11:02:26.655478+0000","last_clean":"2024-04-20T11:02:26.655478+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T11:02:26.655478+0000","last_undegraded":"2024-04-20T11:02:26.655478+0000","last_fullsized":"2024-04-20T11:02:26.655478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.540115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":54533,"num_write":414,"num_write_kb":39250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"27'427","reported_seq":477,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.657238+0000","last_change":"2024-04-20T10:55:42.801257+0000","last_active":"2024-04-20T11:02:26.657238+0000","last_peered":"2024-04-20T11:02:26.657238+0000","last_clean":"2024-04-20T11:02:26.657238+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T11:02:26.657238+0000","last_undegraded":"2024-04-20T11:02:26.657238+0000","last_fullsized":"2024-04-20T11:02:26.657238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:21:36.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":68755,"num_write":396,"num_write_kb":79570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"27'429","reported_seq":476,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.646014+0000","last_change":"2024-04-20T10:55:42.803134+0000","last_active":"2024-04-20T11:02:26.646014+0000","last_peered":"2024-04-20T11:02:26.646014+0000","last_clean":"2024-04-20T11:02:26.646014+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T11:02:26.646014+0000","last_undegraded":"2024-04-20T11:02:26.646014+0000","last_fullsized":"2024-04-20T11:02:26.646014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:29.996414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53186,"num_write":383,"num_write_kb":88017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"27'439","reported_seq":492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.650020+0000","last_change":"2024-04-20T10:55:42.801267+0000","last_active":"2024-04-20T11:02:26.650020+0000","last_peered":"2024-04-20T11:02:26.650020+0000","last_clean":"2024-04-20T11:02:26.650020+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T11:02:26.650020+0000","last_undegraded":"2024-04-20T11:02:26.650020+0000","last_fullsized":"2024-04-20T11:02:26.650020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:45.760791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":86853,"num_write":391,"num_write_kb":112571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"27'455","reported_seq":497,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.645517+0000","last_change":"2024-04-20T10:55:42.795366+0000","last_active":"2024-04-20T11:02:26.645517+0000","last_peered":"2024-04-20T11:02:26.645517+0000","last_clean":"2024-04-20T11:02:26.645517+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T11:02:26.645517+0000","last_undegraded":"2024-04-20T11:02:26.645517+0000","last_fullsized":"2024-04-20T11:02:26.645517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:41:35.429512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36592,"num_write":417,"num_write_kb":59154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"27'423","reported_seq":474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:29.737603+0000","last_change":"2024-04-20T10:55:42.804883+0000","last_active":"2024-04-20T11:02:29.737603+0000","last_peered":"2024-04-20T11:02:29.737603+0000","last_clean":"2024-04-20T11:02:29.737603+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T11:02:29.737603+0000","last_undegraded":"2024-04-20T11:02:29.737603+0000","last_fullsized":"2024-04-20T11:02:29.737603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":55674,"num_write":374,"num_write_kb":59056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"27'445","reported_seq":491,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.657284+0000","last_change":"2024-04-20T10:55:42.800961+0000","last_active":"2024-04-20T11:02:26.657284+0000","last_peered":"2024-04-20T11:02:26.657284+0000","last_clean":"2024-04-20T11:02:26.657284+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T11:02:26.657284+0000","last_undegraded":"2024-04-20T11:02:26.657284+0000","last_fullsized":"2024-04-20T11:02:26.657284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:16.721120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":66341,"num_write":402,"num_write_kb":96389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'409","reported_seq":446,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.635706+0000","last_change":"2024-04-20T10:55:42.798762+0000","last_active":"2024-04-20T11:02:26.635706+0000","last_peered":"2024-04-20T11:02:26.635706+0000","last_clean":"2024-04-20T11:02:26.635706+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T11:02:26.635706+0000","last_undegraded":"2024-04-20T11:02:26.635706+0000","last_fullsized":"2024-04-20T11:02:26.635706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:19.824039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":25926,"num_write":360,"num_write_kb":50488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'472","reported_seq":525,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.655407+0000","last_change":"2024-04-20T10:55:42.790229+0000","last_active":"2024-04-20T11:02:26.655407+0000","last_peered":"2024-04-20T11:02:26.655407+0000","last_clean":"2024-04-20T11:02:26.655407+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T11:02:26.655407+0000","last_undegraded":"2024-04-20T11:02:26.655407+0000","last_fullsized":"2024-04-20T11:02:26.655407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:08.954665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":61071,"num_write":433,"num_write_kb":92498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"27'405","reported_seq":440,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.622930+0000","last_change":"2024-04-20T10:55:42.796464+0000","last_active":"2024-04-20T11:02:26.622930+0000","last_peered":"2024-04-20T11:02:26.622930+0000","last_clean":"2024-04-20T11:02:26.622930+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T11:02:26.622930+0000","last_undegraded":"2024-04-20T11:02:26.622930+0000","last_fullsized":"2024-04-20T11:02:26.622930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:53:49.773552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":21468,"num_write":380,"num_write_kb":33760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"27'460","reported_seq":492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.640122+0000","last_change":"2024-04-20T10:55:42.798462+0000","last_active":"2024-04-20T11:02:26.640122+0000","last_peered":"2024-04-20T11:02:26.640122+0000","last_clean":"2024-04-20T11:02:26.640122+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T11:02:26.640122+0000","last_undegraded":"2024-04-20T11:02:26.640122+0000","last_fullsized":"2024-04-20T11:02:26.640122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:15.321129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8561,"num_write":404,"num_write_kb":47121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'432","reported_seq":487,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.646407+0000","last_change":"2024-04-20T10:55:42.798522+0000","last_active":"2024-04-20T11:02:26.646407+0000","last_peered":"2024-04-20T11:02:26.646407+0000","last_clean":"2024-04-20T11:02:26.646407+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T11:02:26.646407+0000","last_undegraded":"2024-04-20T11:02:26.646407+0000","last_fullsized":"2024-04-20T11:02:26.646407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:28.208926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":82311,"num_write":397,"num_write_kb":100756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"27'406","reported_seq":455,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.644819+0000","last_change":"2024-04-20T10:55:42.795776+0000","last_active":"2024-04-20T11:02:26.644819+0000","last_peered":"2024-04-20T11:02:26.644819+0000","last_clean":"2024-04-20T11:02:26.644819+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T11:02:26.644819+0000","last_undegraded":"2024-04-20T11:02:26.644819+0000","last_fullsized":"2024-04-20T11:02:26.644819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:02.978549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":40362,"num_write":368,"num_write_kb":83934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'437","reported_seq":486,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.647068+0000","last_change":"2024-04-20T10:55:42.799266+0000","last_active":"2024-04-20T11:02:26.647068+0000","last_peered":"2024-04-20T11:02:26.647068+0000","last_clean":"2024-04-20T11:02:26.647068+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T11:02:26.647068+0000","last_undegraded":"2024-04-20T11:02:26.647068+0000","last_fullsized":"2024-04-20T11:02:26.647068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":51907,"num_write":395,"num_write_kb":76367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"27'479","reported_seq":531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.654519+0000","last_change":"2024-04-20T10:55:42.796324+0000","last_active":"2024-04-20T11:02:26.654519+0000","last_peered":"2024-04-20T11:02:26.654519+0000","last_clean":"2024-04-20T11:02:26.654519+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T11:02:26.654519+0000","last_undegraded":"2024-04-20T11:02:26.654519+0000","last_fullsized":"2024-04-20T11:02:26.654519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:09:37.993493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":45044,"num_write":427,"num_write_kb":43197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"27'425","reported_seq":462,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.655843+0000","last_change":"2024-04-20T10:55:42.807092+0000","last_active":"2024-04-20T11:02:26.655843+0000","last_peered":"2024-04-20T11:02:26.655843+0000","last_clean":"2024-04-20T11:02:26.655843+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T11:02:26.655843+0000","last_undegraded":"2024-04-20T11:02:26.655843+0000","last_fullsized":"2024-04-20T11:02:26.655843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:52.017801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":30452,"num_write":380,"num_write_kb":32013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"27'454","reported_seq":490,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.632245+0000","last_change":"2024-04-20T10:55:42.805642+0000","last_active":"2024-04-20T11:02:26.632245+0000","last_peered":"2024-04-20T11:02:26.632245+0000","last_clean":"2024-04-20T11:02:26.632245+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T11:02:26.632245+0000","last_undegraded":"2024-04-20T11:02:26.632245+0000","last_fullsized":"2024-04-20T11:02:26.632245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:47.760609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":23674,"num_write":409,"num_write_kb":56716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"27'397","reported_seq":438,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.642584+0000","last_change":"2024-04-20T10:55:42.795826+0000","last_active":"2024-04-20T11:02:26.642584+0000","last_peered":"2024-04-20T11:02:26.642584+0000","last_clean":"2024-04-20T11:02:26.642584+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T11:02:26.642584+0000","last_undegraded":"2024-04-20T11:02:26.642584+0000","last_fullsized":"2024-04-20T11:02:26.642584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:10.456248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36784,"num_write":356,"num_write_kb":38078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'161","reported_seq":194,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:12.934610+0000","last_change":"2024-04-20T10:55:41.793477+0000","last_active":"2024-04-20T11:02:12.934610+0000","last_peered":"2024-04-20T11:02:12.934610+0000","last_clean":"2024-04-20T11:02:12.934610+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T11:02:12.934610+0000","last_undegraded":"2024-04-20T11:02:12.934610+0000","last_fullsized":"2024-04-20T11:02:12.934610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195439,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":112,"num_write_kb":4163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'403","reported_seq":440,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.652277+0000","last_change":"2024-04-20T10:55:42.796857+0000","last_active":"2024-04-20T11:02:26.652277+0000","last_peered":"2024-04-20T11:02:26.652277+0000","last_clean":"2024-04-20T11:02:26.652277+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T11:02:26.652277+0000","last_undegraded":"2024-04-20T11:02:26.652277+0000","last_fullsized":"2024-04-20T11:02:26.652277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":26712,"num_write":367,"num_write_kb":67448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"27'322","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.494755+0000","last_change":"2024-04-20T10:55:41.795436+0000","last_active":"2024-04-20T11:02:11.494755+0000","last_peered":"2024-04-20T11:02:11.494755+0000","last_clean":"2024-04-20T11:02:11.494755+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T11:02:11.494755+0000","last_undegraded":"2024-04-20T11:02:11.494755+0000","last_fullsized":"2024-04-20T11:02:11.494755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:51:28.987127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":260,"num_write_kb":12413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"27'95","reported_seq":129,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:05.455443+0000","last_change":"2024-04-20T10:55:41.795469+0000","last_active":"2024-04-20T11:02:05.455443+0000","last_peered":"2024-04-20T11:02:05.455443+0000","last_clean":"2024-04-20T11:02:05.455443+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T11:02:05.455443+0000","last_undegraded":"2024-04-20T11:02:05.455443+0000","last_fullsized":"2024-04-20T11:02:05.455443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:49:28.965381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":20,"num_write":51,"num_write_kb":4136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'449","reported_seq":486,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.635949+0000","last_change":"2024-04-20T10:55:42.797501+0000","last_active":"2024-04-20T11:02:26.635949+0000","last_peered":"2024-04-20T11:02:26.635949+0000","last_clean":"2024-04-20T11:02:26.635949+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T11:02:26.635949+0000","last_undegraded":"2024-04-20T11:02:26.635949+0000","last_fullsized":"2024-04-20T11:02:26.635949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:50.080059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":16783,"num_write":403,"num_write_kb":42765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"27'797","reported_seq":828,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.485457+0000","last_change":"2024-04-20T10:55:41.793345+0000","last_active":"2024-04-20T11:02:16.485457+0000","last_peered":"2024-04-20T11:02:16.485457+0000","last_clean":"2024-04-20T11:02:16.485457+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T11:02:16.485457+0000","last_undegraded":"2024-04-20T11:02:16.485457+0000","last_fullsized":"2024-04-20T11:02:16.485457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:09:38.442904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":724,"num_write_kb":8631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"27'457","reported_seq":520,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.646922+0000","last_change":"2024-04-20T10:55:42.798934+0000","last_active":"2024-04-20T11:02:26.646922+0000","last_peered":"2024-04-20T11:02:26.646922+0000","last_clean":"2024-04-20T11:02:26.646922+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T11:02:26.646922+0000","last_undegraded":"2024-04-20T11:02:26.646922+0000","last_fullsized":"2024-04-20T11:02:26.646922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:39.136858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":131398,"num_write":411,"num_write_kb":92477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"27'73","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.943331+0000","last_change":"2024-04-20T10:55:41.799690+0000","last_active":"2024-04-20T11:02:17.943331+0000","last_peered":"2024-04-20T11:02:17.943331+0000","last_clean":"2024-04-20T11:02:17.943331+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T11:02:17.943331+0000","last_undegraded":"2024-04-20T11:02:17.943331+0000","last_fullsized":"2024-04-20T11:02:17.943331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:51.008515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":22,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"27'455","reported_seq":508,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.655415+0000","last_change":"2024-04-20T10:55:42.798865+0000","last_active":"2024-04-20T11:02:26.655415+0000","last_peered":"2024-04-20T11:02:26.655415+0000","last_clean":"2024-04-20T11:02:26.655415+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T11:02:26.655415+0000","last_undegraded":"2024-04-20T11:02:26.655415+0000","last_fullsized":"2024-04-20T11:02:26.655415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:38:18.689576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":75132,"num_write":413,"num_write_kb":100522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'95","reported_seq":131,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.944720+0000","last_change":"2024-04-20T10:55:41.798842+0000","last_active":"2024-04-20T11:02:17.944720+0000","last_peered":"2024-04-20T11:02:17.944720+0000","last_clean":"2024-04-20T11:02:17.944720+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T11:02:17.944720+0000","last_undegraded":"2024-04-20T11:02:17.944720+0000","last_fullsized":"2024-04-20T11:02:17.944720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:47:57.968981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":51,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"27'450","reported_seq":533,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.656062+0000","last_change":"2024-04-20T10:55:42.803448+0000","last_active":"2024-04-20T11:02:26.656062+0000","last_peered":"2024-04-20T11:02:26.656062+0000","last_clean":"2024-04-20T11:02:26.656062+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T11:02:26.656062+0000","last_undegraded":"2024-04-20T11:02:26.656062+0000","last_fullsized":"2024-04-20T11:02:26.656062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":176347,"num_write":418,"num_write_kb":171699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"27'268","reported_seq":300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.162494+0000","last_change":"2024-04-20T10:55:41.798565+0000","last_active":"2024-04-20T11:02:17.162494+0000","last_peered":"2024-04-20T11:02:17.162494+0000","last_clean":"2024-04-20T11:02:17.162494+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T11:02:17.162494+0000","last_undegraded":"2024-04-20T11:02:17.162494+0000","last_fullsized":"2024-04-20T11:02:17.162494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:58:05.791682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":225,"num_write_kb":4349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'408","reported_seq":455,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.645446+0000","last_change":"2024-04-20T10:55:42.799222+0000","last_active":"2024-04-20T11:02:26.645446+0000","last_peered":"2024-04-20T11:02:26.645446+0000","last_clean":"2024-04-20T11:02:26.645446+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T11:02:26.645446+0000","last_undegraded":"2024-04-20T11:02:26.645446+0000","last_fullsized":"2024-04-20T11:02:26.645446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:39:19.720233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53530,"num_write":359,"num_write_kb":71690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"27'396","reported_seq":432,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.951080+0000","last_change":"2024-04-20T10:55:41.795944+0000","last_active":"2024-04-20T11:02:17.951080+0000","last_peered":"2024-04-20T11:02:17.951080+0000","last_clean":"2024-04-20T11:02:17.951080+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T11:02:17.951080+0000","last_undegraded":"2024-04-20T11:02:17.951080+0000","last_fullsized":"2024-04-20T11:02:17.951080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:45.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":365,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"27'429","reported_seq":480,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.652500+0000","last_change":"2024-04-20T10:55:42.801818+0000","last_active":"2024-04-20T11:02:26.652500+0000","last_peered":"2024-04-20T11:02:26.652500+0000","last_clean":"2024-04-20T11:02:26.652500+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T11:02:26.652500+0000","last_undegraded":"2024-04-20T11:02:26.652500+0000","last_fullsized":"2024-04-20T11:02:26.652500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:06.091158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":72220,"num_write":383,"num_write_kb":72034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"27'357","reported_seq":397,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.951826+0000","last_change":"2024-04-20T10:55:41.793424+0000","last_active":"2024-04-20T11:02:17.951826+0000","last_peered":"2024-04-20T11:02:17.951826+0000","last_clean":"2024-04-20T11:02:17.951826+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T11:02:17.951826+0000","last_undegraded":"2024-04-20T11:02:17.951826+0000","last_fullsized":"2024-04-20T11:02:17.951826+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":299,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"27'406","reported_seq":444,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.650790+0000","last_change":"2024-04-20T10:55:42.804921+0000","last_active":"2024-04-20T11:02:26.650790+0000","last_peered":"2024-04-20T11:02:26.650790+0000","last_clean":"2024-04-20T11:02:26.650790+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T11:02:26.650790+0000","last_undegraded":"2024-04-20T11:02:26.650790+0000","last_fullsized":"2024-04-20T11:02:26.650790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":32812,"num_write":361,"num_write_kb":30179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"27'73","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.943894+0000","last_change":"2024-04-20T10:55:41.796131+0000","last_active":"2024-04-20T11:02:17.943894+0000","last_peered":"2024-04-20T11:02:17.943894+0000","last_clean":"2024-04-20T11:02:17.943894+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T11:02:17.943894+0000","last_undegraded":"2024-04-20T11:02:17.943894+0000","last_fullsized":"2024-04-20T11:02:17.943894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:15.519607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":17,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'418","reported_seq":461,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.651306+0000","last_change":"2024-04-20T10:56:22.904413+0000","last_active":"2024-04-20T11:02:26.651306+0000","last_peered":"2024-04-20T11:02:26.651306+0000","last_clean":"2024-04-20T11:02:26.651306+0000","last_became_active":"2024-04-20T10:56:22.904118+0000","last_became_peered":"2024-04-20T10:56:22.904118+0000","last_unstale":"2024-04-20T11:02:26.651306+0000","last_undegraded":"2024-04-20T11:02:26.651306+0000","last_fullsized":"2024-04-20T11:02:26.651306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:14:06.204840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":63538,"num_write":372,"num_write_kb":113421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"27'78","reported_seq":113,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:01.406304+0000","last_change":"2024-04-20T10:55:41.794988+0000","last_active":"2024-04-20T11:02:01.406304+0000","last_peered":"2024-04-20T11:02:01.406304+0000","last_clean":"2024-04-20T11:02:01.406304+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T11:02:01.406304+0000","last_undegraded":"2024-04-20T11:02:01.406304+0000","last_fullsized":"2024-04-20T11:02:01.406304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:20.745924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":10,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"27'444","reported_seq":492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.626332+0000","last_change":"2024-04-20T10:55:42.798732+0000","last_active":"2024-04-20T11:02:26.626332+0000","last_peered":"2024-04-20T11:02:26.626332+0000","last_clean":"2024-04-20T11:02:26.626332+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T11:02:26.626332+0000","last_undegraded":"2024-04-20T11:02:26.626332+0000","last_fullsized":"2024-04-20T11:02:26.626332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:31:53.845515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":68877,"num_write":411,"num_write_kb":59157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"27'361","reported_seq":397,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.948421+0000","last_change":"2024-04-20T10:55:41.790624+0000","last_active":"2024-04-20T11:02:17.948421+0000","last_peered":"2024-04-20T11:02:17.948421+0000","last_clean":"2024-04-20T11:02:17.948421+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T11:02:17.948421+0000","last_undegraded":"2024-04-20T11:02:17.948421+0000","last_fullsized":"2024-04-20T11:02:17.948421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:48.155677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":307,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"27'477","reported_seq":531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.650928+0000","last_change":"2024-04-20T10:55:42.804278+0000","last_active":"2024-04-20T11:02:26.650928+0000","last_peered":"2024-04-20T11:02:26.650928+0000","last_clean":"2024-04-20T11:02:26.650928+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T11:02:26.650928+0000","last_undegraded":"2024-04-20T11:02:26.650928+0000","last_fullsized":"2024-04-20T11:02:26.650928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:59.122352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":59233,"num_write":434,"num_write_kb":71822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'75","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.455397+0000","last_change":"2024-04-20T10:55:41.795843+0000","last_active":"2024-04-20T11:02:16.455397+0000","last_peered":"2024-04-20T11:02:16.455397+0000","last_clean":"2024-04-20T11:02:16.455397+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T11:02:16.455397+0000","last_undegraded":"2024-04-20T11:02:16.455397+0000","last_fullsized":"2024-04-20T11:02:16.455397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:54.517493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"27'413","reported_seq":475,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.651159+0000","last_change":"2024-04-20T10:55:42.804857+0000","last_active":"2024-04-20T11:02:26.651159+0000","last_peered":"2024-04-20T11:02:26.651159+0000","last_clean":"2024-04-20T11:02:26.651159+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T11:02:26.651159+0000","last_undegraded":"2024-04-20T11:02:26.651159+0000","last_fullsized":"2024-04-20T11:02:26.651159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:06:55.622038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":108675,"num_write":383,"num_write_kb":121193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'262","reported_seq":297,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:05.438503+0000","last_change":"2024-04-20T10:55:41.791181+0000","last_active":"2024-04-20T11:02:05.438503+0000","last_peered":"2024-04-20T11:02:05.438503+0000","last_clean":"2024-04-20T11:02:05.438503+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T11:02:05.438503+0000","last_undegraded":"2024-04-20T11:02:05.438503+0000","last_fullsized":"2024-04-20T11:02:05.438503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:07:52.576212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":218,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"27'393","reported_seq":435,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.660383+0000","last_change":"2024-04-20T10:55:42.804678+0000","last_active":"2024-04-20T11:02:26.660383+0000","last_peered":"2024-04-20T11:02:26.660383+0000","last_clean":"2024-04-20T11:02:26.660383+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T11:02:26.660383+0000","last_undegraded":"2024-04-20T11:02:26.660383+0000","last_fullsized":"2024-04-20T11:02:26.660383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:19:39.836283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":38645,"num_write":351,"num_write_kb":38372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"27'464","reported_seq":495,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.453716+0000","last_change":"2024-04-20T10:55:41.787976+0000","last_active":"2024-04-20T11:02:16.453716+0000","last_peered":"2024-04-20T11:02:16.453716+0000","last_clean":"2024-04-20T11:02:16.453716+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T11:02:16.453716+0000","last_undegraded":"2024-04-20T11:02:16.453716+0000","last_fullsized":"2024-04-20T11:02:16.453716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:19:12.885641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"27'466","reported_seq":507,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.654843+0000","last_change":"2024-04-20T10:55:42.796281+0000","last_active":"2024-04-20T11:02:26.654843+0000","last_peered":"2024-04-20T11:02:26.654843+0000","last_clean":"2024-04-20T11:02:26.654843+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T11:02:26.654843+0000","last_undegraded":"2024-04-20T11:02:26.654843+0000","last_fullsized":"2024-04-20T11:02:26.654843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:02.604473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":42813,"num_write":427,"num_write_kb":83769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"27'70","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.488270+0000","last_change":"2024-04-20T10:55:41.798650+0000","last_active":"2024-04-20T11:02:11.488270+0000","last_peered":"2024-04-20T11:02:11.488270+0000","last_clean":"2024-04-20T11:02:11.488270+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T11:02:11.488270+0000","last_undegraded":"2024-04-20T11:02:11.488270+0000","last_fullsized":"2024-04-20T11:02:11.488270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:18.914116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":8,"num_write":11,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"27'404","reported_seq":436,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:27.300409+0000","last_change":"2024-04-20T10:55:42.806998+0000","last_active":"2024-04-20T11:02:27.300409+0000","last_peered":"2024-04-20T11:02:27.300409+0000","last_clean":"2024-04-20T11:02:27.300409+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T11:02:27.300409+0000","last_undegraded":"2024-04-20T11:02:27.300409+0000","last_fullsized":"2024-04-20T11:02:27.300409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:49:07.363549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6180,"num_write":356,"num_write_kb":9278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:01:52.405499+0000","last_change":"2024-04-20T10:55:30.924053+0000","last_active":"2024-04-20T11:01:52.405499+0000","last_peered":"2024-04-20T11:01:52.405499+0000","last_clean":"2024-04-20T11:01:52.405499+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T11:01:52.405499+0000","last_undegraded":"2024-04-20T11:01:52.405499+0000","last_fullsized":"2024-04-20T11:01:52.405499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:29.772827+0000","last_clean_scrub_stamp":"2024-04-20T10:55:29.772827+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:21.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"27'77","reported_seq":115,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.490479+0000","last_change":"2024-04-20T10:55:41.793574+0000","last_active":"2024-04-20T11:02:16.490479+0000","last_peered":"2024-04-20T11:02:16.490479+0000","last_clean":"2024-04-20T11:02:16.490479+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T11:02:16.490479+0000","last_undegraded":"2024-04-20T11:02:16.490479+0000","last_fullsized":"2024-04-20T11:02:16.490479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:39:45.985740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":18,"num_write":19,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'360","reported_seq":395,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.642256+0000","last_change":"2024-04-20T10:55:42.793663+0000","last_active":"2024-04-20T11:02:26.642256+0000","last_peered":"2024-04-20T11:02:26.642256+0000","last_clean":"2024-04-20T11:02:26.642256+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T11:02:26.642256+0000","last_undegraded":"2024-04-20T11:02:26.642256+0000","last_fullsized":"2024-04-20T11:02:26.642256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:55.573925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":17282,"num_write":327,"num_write_kb":42097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"27'122","reported_seq":155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:02.416875+0000","last_change":"2024-04-20T10:55:41.792518+0000","last_active":"2024-04-20T11:02:02.416875+0000","last_peered":"2024-04-20T11:02:02.416875+0000","last_clean":"2024-04-20T11:02:02.416875+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T11:02:02.416875+0000","last_undegraded":"2024-04-20T11:02:02.416875+0000","last_fullsized":"2024-04-20T11:02:02.416875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:05.053080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":64,"num_write_kb":8241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"27'385","reported_seq":427,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.650936+0000","last_change":"2024-04-20T10:55:42.792214+0000","last_active":"2024-04-20T11:02:26.650936+0000","last_peered":"2024-04-20T11:02:26.650936+0000","last_clean":"2024-04-20T11:02:26.650936+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T11:02:26.650936+0000","last_undegraded":"2024-04-20T11:02:26.650936+0000","last_fullsized":"2024-04-20T11:02:26.650936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":18713,"num_write":347,"num_write_kb":51198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"27'326","reported_seq":611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:18.437066+0000","last_change":"2024-04-20T10:55:41.791886+0000","last_active":"2024-04-20T11:02:18.437066+0000","last_peered":"2024-04-20T11:02:18.437066+0000","last_clean":"2024-04-20T11:02:18.437066+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T11:02:18.437066+0000","last_undegraded":"2024-04-20T11:02:18.437066+0000","last_fullsized":"2024-04-20T11:02:18.437066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.395301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462167,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":258,"num_read_kb":1597,"num_write":262,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"27'418","reported_seq":465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.632885+0000","last_change":"2024-04-20T10:55:42.797620+0000","last_active":"2024-04-20T11:02:26.632885+0000","last_peered":"2024-04-20T11:02:26.632885+0000","last_clean":"2024-04-20T11:02:26.632885+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T11:02:26.632885+0000","last_undegraded":"2024-04-20T11:02:26.632885+0000","last_fullsized":"2024-04-20T11:02:26.632885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:39:27.986957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":47086,"num_write":383,"num_write_kb":51294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"27'799","reported_seq":838,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:01:59.759462+0000","last_change":"2024-04-20T10:55:41.798905+0000","last_active":"2024-04-20T11:01:59.759462+0000","last_peered":"2024-04-20T11:01:59.759462+0000","last_clean":"2024-04-20T11:01:59.759462+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T11:01:59.759462+0000","last_undegraded":"2024-04-20T11:01:59.759462+0000","last_fullsized":"2024-04-20T11:01:59.759462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:08.951715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":731,"num_write_kb":8599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"27'428","reported_seq":478,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.648804+0000","last_change":"2024-04-20T10:55:42.793946+0000","last_active":"2024-04-20T11:02:26.648804+0000","last_peered":"2024-04-20T11:02:26.648804+0000","last_clean":"2024-04-20T11:02:26.648804+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T11:02:26.648804+0000","last_undegraded":"2024-04-20T11:02:26.648804+0000","last_fullsized":"2024-04-20T11:02:26.648804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:59.154202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":65250,"num_write":382,"num_write_kb":83821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"27'75","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.455107+0000","last_change":"2024-04-20T10:55:41.797424+0000","last_active":"2024-04-20T11:02:16.455107+0000","last_peered":"2024-04-20T11:02:16.455107+0000","last_clean":"2024-04-20T11:02:16.455107+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T11:02:16.455107+0000","last_undegraded":"2024-04-20T11:02:16.455107+0000","last_fullsized":"2024-04-20T11:02:16.455107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:03:13.914726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"27'426","reported_seq":459,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.654508+0000","last_change":"2024-04-20T10:55:42.795728+0000","last_active":"2024-04-20T11:02:26.654508+0000","last_peered":"2024-04-20T11:02:26.654508+0000","last_clean":"2024-04-20T11:02:26.654508+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T11:02:26.654508+0000","last_undegraded":"2024-04-20T11:02:26.654508+0000","last_fullsized":"2024-04-20T11:02:26.654508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:34.624133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":375,"num_write_kb":62820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"27'85","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:16.454748+0000","last_change":"2024-04-20T10:55:41.797633+0000","last_active":"2024-04-20T11:02:16.454748+0000","last_peered":"2024-04-20T11:02:16.454748+0000","last_clean":"2024-04-20T11:02:16.454748+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T11:02:16.454748+0000","last_undegraded":"2024-04-20T11:02:16.454748+0000","last_fullsized":"2024-04-20T11:02:16.454748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:40.684023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":29,"num_write_kb":4110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"27'391","reported_seq":430,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.633828+0000","last_change":"2024-04-20T10:55:42.797112+0000","last_active":"2024-04-20T11:02:26.633828+0000","last_peered":"2024-04-20T11:02:26.633828+0000","last_clean":"2024-04-20T11:02:26.633828+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T11:02:26.633828+0000","last_undegraded":"2024-04-20T11:02:26.633828+0000","last_fullsized":"2024-04-20T11:02:26.633828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:58.646784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":25761,"num_write":337,"num_write_kb":50347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"27'346","reported_seq":383,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:18.958286+0000","last_change":"2024-04-20T10:55:41.798496+0000","last_active":"2024-04-20T11:02:18.958286+0000","last_peered":"2024-04-20T11:02:18.958286+0000","last_clean":"2024-04-20T11:02:18.958286+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T11:02:18.958286+0000","last_undegraded":"2024-04-20T11:02:18.958286+0000","last_fullsized":"2024-04-20T11:02:18.958286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:35.212194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":669,"num_write_kb":1875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"27'345","reported_seq":381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.644935+0000","last_change":"2024-04-20T10:55:42.800333+0000","last_active":"2024-04-20T11:02:26.644935+0000","last_peered":"2024-04-20T11:02:26.644935+0000","last_clean":"2024-04-20T11:02:26.644935+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T11:02:26.644935+0000","last_undegraded":"2024-04-20T11:02:26.644935+0000","last_fullsized":"2024-04-20T11:02:26.644935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:56.948399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15872,"num_write":302,"num_write_kb":13258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"27'211","reported_seq":249,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.602857+0000","last_change":"2024-04-20T10:55:41.796496+0000","last_active":"2024-04-20T11:02:26.602857+0000","last_peered":"2024-04-20T11:02:26.602857+0000","last_clean":"2024-04-20T11:02:26.602857+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T11:02:26.602857+0000","last_undegraded":"2024-04-20T11:02:26.602857+0000","last_fullsized":"2024-04-20T11:02:26.602857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:58:38.846247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3862944,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":149,"num_write_kb":3853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'458","reported_seq":529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.645350+0000","last_change":"2024-04-20T10:55:42.801709+0000","last_active":"2024-04-20T11:02:26.645350+0000","last_peered":"2024-04-20T11:02:26.645350+0000","last_clean":"2024-04-20T11:02:26.645350+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T11:02:26.645350+0000","last_undegraded":"2024-04-20T11:02:26.645350+0000","last_fullsized":"2024-04-20T11:02:26.645350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":106048,"num_write":421,"num_write_kb":76099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"27'559","reported_seq":592,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.947780+0000","last_change":"2024-04-20T10:55:41.797965+0000","last_active":"2024-04-20T11:02:17.947780+0000","last_peered":"2024-04-20T11:02:17.947780+0000","last_clean":"2024-04-20T11:02:17.947780+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T11:02:17.947780+0000","last_undegraded":"2024-04-20T11:02:17.947780+0000","last_fullsized":"2024-04-20T11:02:17.947780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:52:59.202258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":504,"num_write_kb":4393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'461","reported_seq":506,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.611658+0000","last_change":"2024-04-20T10:55:42.797201+0000","last_active":"2024-04-20T11:02:26.611658+0000","last_peered":"2024-04-20T11:02:26.611658+0000","last_clean":"2024-04-20T11:02:26.611658+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T11:02:26.611658+0000","last_undegraded":"2024-04-20T11:02:26.611658+0000","last_fullsized":"2024-04-20T11:02:26.611658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:21:30.846060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":55402,"num_write":430,"num_write_kb":80382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'246","reported_seq":286,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.947314+0000","last_change":"2024-04-20T10:55:41.790576+0000","last_active":"2024-04-20T11:02:17.947314+0000","last_peered":"2024-04-20T11:02:17.947314+0000","last_clean":"2024-04-20T11:02:17.947314+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T11:02:17.947314+0000","last_undegraded":"2024-04-20T11:02:17.947314+0000","last_fullsized":"2024-04-20T11:02:17.947314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:12.302700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":197,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'410","reported_seq":451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.644588+0000","last_change":"2024-04-20T10:55:42.795226+0000","last_active":"2024-04-20T11:02:26.644588+0000","last_peered":"2024-04-20T11:02:26.644588+0000","last_clean":"2024-04-20T11:02:26.644588+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T11:02:26.644588+0000","last_undegraded":"2024-04-20T11:02:26.644588+0000","last_fullsized":"2024-04-20T11:02:26.644588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:29.290003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":45668,"num_write":367,"num_write_kb":46153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'230","reported_seq":263,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:14.768794+0000","last_change":"2024-04-20T10:55:41.795297+0000","last_active":"2024-04-20T11:02:14.768794+0000","last_peered":"2024-04-20T11:02:14.768794+0000","last_clean":"2024-04-20T11:02:14.768794+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T11:02:14.768794+0000","last_undegraded":"2024-04-20T11:02:14.768794+0000","last_fullsized":"2024-04-20T11:02:14.768794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:22.458940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":165,"num_write_kb":4183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"27'472","reported_seq":536,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.645182+0000","last_change":"2024-04-20T10:55:42.797879+0000","last_active":"2024-04-20T11:02:26.645182+0000","last_peered":"2024-04-20T11:02:26.645182+0000","last_clean":"2024-04-20T11:02:26.645182+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T11:02:26.645182+0000","last_undegraded":"2024-04-20T11:02:26.645182+0000","last_fullsized":"2024-04-20T11:02:26.645182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:09:54.218670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":121848,"num_write":422,"num_write_kb":125716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'140","reported_seq":173,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:17.949268+0000","last_change":"2024-04-20T10:55:41.790036+0000","last_active":"2024-04-20T11:02:17.949268+0000","last_peered":"2024-04-20T11:02:17.949268+0000","last_clean":"2024-04-20T11:02:17.949268+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T11:02:17.949268+0000","last_undegraded":"2024-04-20T11:02:17.949268+0000","last_fullsized":"2024-04-20T11:02:17.949268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:42:58.780934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":89,"num_write_kb":8247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"27'440","reported_seq":487,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.650357+0000","last_change":"2024-04-20T10:55:42.798259+0000","last_active":"2024-04-20T11:02:26.650357+0000","last_peered":"2024-04-20T11:02:26.650357+0000","last_clean":"2024-04-20T11:02:26.650357+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T11:02:26.650357+0000","last_undegraded":"2024-04-20T11:02:26.650357+0000","last_fullsized":"2024-04-20T11:02:26.650357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:28:13.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":46429,"num_write":408,"num_write_kb":79692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"27'393","reported_seq":422,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.650142+0000","last_change":"2024-04-20T10:56:22.904797+0000","last_active":"2024-04-20T11:02:26.650142+0000","last_peered":"2024-04-20T11:02:26.650142+0000","last_clean":"2024-04-20T11:02:26.650142+0000","last_became_active":"2024-04-20T10:56:22.904460+0000","last_became_peered":"2024-04-20T10:56:22.904460+0000","last_unstale":"2024-04-20T11:02:26.650142+0000","last_undegraded":"2024-04-20T11:02:26.650142+0000","last_fullsized":"2024-04-20T11:02:26.650142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:55.531408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15562,"num_write":337,"num_write_kb":42622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'264","reported_seq":300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.428494+0000","last_change":"2024-04-20T10:55:41.799053+0000","last_active":"2024-04-20T11:02:07.428494+0000","last_peered":"2024-04-20T11:02:07.428494+0000","last_clean":"2024-04-20T11:02:07.428494+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T11:02:07.428494+0000","last_undegraded":"2024-04-20T11:02:07.428494+0000","last_fullsized":"2024-04-20T11:02:07.428494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:19.595058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":212,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'389","reported_seq":430,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.655064+0000","last_change":"2024-04-20T10:55:42.791793+0000","last_active":"2024-04-20T11:02:26.655064+0000","last_peered":"2024-04-20T11:02:26.655064+0000","last_clean":"2024-04-20T11:02:26.655064+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T11:02:26.655064+0000","last_undegraded":"2024-04-20T11:02:26.655064+0000","last_fullsized":"2024-04-20T11:02:26.655064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:57:01.181969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":31020,"num_write":340,"num_write_kb":46796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"27'219","reported_seq":263,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:07.427078+0000","last_change":"2024-04-20T10:55:41.798350+0000","last_active":"2024-04-20T11:02:07.427078+0000","last_peered":"2024-04-20T11:02:07.427078+0000","last_clean":"2024-04-20T11:02:07.427078+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T11:02:07.427078+0000","last_undegraded":"2024-04-20T11:02:07.427078+0000","last_fullsized":"2024-04-20T11:02:07.427078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:11.913329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":39,"num_read_kb":43,"num_write":158,"num_write_kb":12367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"27'383","reported_seq":437,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.656173+0000","last_change":"2024-04-20T10:55:42.803668+0000","last_active":"2024-04-20T11:02:26.656173+0000","last_peered":"2024-04-20T11:02:26.656173+0000","last_clean":"2024-04-20T11:02:26.656173+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T11:02:26.656173+0000","last_undegraded":"2024-04-20T11:02:26.656173+0000","last_fullsized":"2024-04-20T11:02:26.656173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":72309,"num_write":351,"num_write_kb":75679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'88","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.474013+0000","last_change":"2024-04-20T10:55:41.792770+0000","last_active":"2024-04-20T11:02:11.474013+0000","last_peered":"2024-04-20T11:02:11.474013+0000","last_clean":"2024-04-20T11:02:11.474013+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T11:02:11.474013+0000","last_undegraded":"2024-04-20T11:02:11.474013+0000","last_fullsized":"2024-04-20T11:02:11.474013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:27.885885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'434","reported_seq":479,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:26.631568+0000","last_change":"2024-04-20T10:56:22.913007+0000","last_active":"2024-04-20T11:02:26.631568+0000","last_peered":"2024-04-20T11:02:26.631568+0000","last_clean":"2024-04-20T11:02:26.631568+0000","last_became_active":"2024-04-20T10:56:22.912718+0000","last_became_peered":"2024-04-20T10:56:22.912718+0000","last_unstale":"2024-04-20T11:02:26.631568+0000","last_undegraded":"2024-04-20T11:02:26.631568+0000","last_fullsized":"2024-04-20T11:02:26.631568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:41.771498+0000","last_clean_scrub_stamp":"2024-04-20T10:55:41.771498+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:24:35.954727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":67508,"num_write":400,"num_write_kb":96285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1f","version":"27'70","reported_seq":110,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:11.436331+0000","last_change":"2024-04-20T10:55:41.794108+0000","last_active":"2024-04-20T11:02:11.436331+0000","last_peered":"2024-04-20T11:02:11.436331+0000","last_clean":"2024-04-20T11:02:11.436331+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T11:02:11.436331+0000","last_undegraded":"2024-04-20T11:02:11.436331+0000","last_fullsized":"2024-04-20T11:02:11.436331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:55:40.762212+0000","last_clean_scrub_stamp":"2024-04-20T10:55:40.762212+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":25,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1110,"num_read_kb":3290197,"num_write":24364,"num_write_kb":4196200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27062,"ondisk_log_size":27062,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":152133224,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":619,"num_read_kb":1973,"num_write":6427,"num_write_kb":154893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":304513024,"data_stored":304266448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7804,"ondisk_log_size":7804,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574937,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115468,"kb_used_data":57924,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94256372,"statfs":{"total":96636764160,"available":96518524928,"internally_reserved":0,"allocated":59314176,"data_stored":59096824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574937,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80316,"kb_used_data":49140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94291524,"statfs":{"total":96636764160,"available":96554520576,"internally_reserved":0,"allocated":50319360,"data_stored":50116697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574937,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86548,"kb_used_data":29004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94285292,"statfs":{"total":96636764160,"available":96548139008,"internally_reserved":0,"allocated":29700096,"data_stored":29479029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102908,"kb_used_data":45364,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94268932,"statfs":{"total":96636764160,"available":96531386368,"internally_reserved":0,"allocated":46452736,"data_stored":46253643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74004,"kb_used_data":34508,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94297836,"statfs":{"total":96636764160,"available":96560984064,"internally_reserved":0,"allocated":35336192,"data_stored":35133465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574936,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41048,"kb_used_data":14032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330792,"statfs":{"total":96636764160,"available":96594731008,"internally_reserved":0,"allocated":14368768,"data_stored":14161398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94656,"kb_used_data":41272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94277184,"statfs":{"total":96636764160,"available":96539836416,"internally_reserved":0,"allocated":42262528,"data_stored":42059541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574938,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64900,"kb_used_data":29564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94306940,"statfs":{"total":96636764160,"available":96570306560,"internally_reserved":0,"allocated":30273536,"data_stored":30066339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29388800,"data_stored":29361002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41971712,"data_stored":41944572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14077952,"data_stored":14046429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35045376,"data_stored":35018496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46161920,"data_stored":46138674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29409280,"data_stored":29364060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50028544,"data_stored":50001728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":58429440,"data_stored":58391487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:02:33.275 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.275 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.275 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.275 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.275 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.276 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.276 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.276 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.276 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.276 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.276 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.276 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.276 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.276 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.277 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.278 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.278 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.278 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.278 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.278 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.278 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.278 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.278 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.278 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.279 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.280 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.281 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.282 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.282 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-20T10:55:29.772827+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=29, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.282 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.282 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.282 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.282 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.282 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.282 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.282 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.283 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.283 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.283 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.283 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.283 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.283 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.283 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.283 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.283 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.284 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.284 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.284 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.284 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.284 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.284 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.284 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.284 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.284 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.285 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.285 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.285 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.285 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.285 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.285 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-20T10:55:41.771498+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.285 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-04-20T10:55:40.762212+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=10, tm_min=55, tm_sec=40, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=11, tm_min=2, tm_sec=28, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T11:02:33.285 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-20T11:02:43.261 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:02:43.264 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:02:53.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T11:02:53.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.400+0000 7f0858ef2640 1 Processor -- start 2024-04-20T11:02:53.400 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.400+0000 7f0858ef2640 1 -- start start 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.401+0000 7f0858ef2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540d0c90 0x7f08540d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.401+0000 7f0858ef2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085415b9c0 con 0x7f0854155950 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.401+0000 7f0858ef2640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f085415bca0 con 0x7f08540b54f0 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.401+0000 7f0858ef2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f085415bf80 con 0x7f08540d0c90 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.401+0000 7f085259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0854155950 0x7f08540cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48852/0 (socket says 172.21.15.27:48852) 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.402+0000 7f085259c640 1 -- 172.21.15.27:0/2323882386 learned_addr learned my addr 172.21.15.27:0/2323882386 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.402+0000 7f0852d9d640 1 --2- 172.21.15.27:0/2323882386 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540d0c90 0x7f08540d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.402+0000 7f085159a640 1 -- 172.21.15.27:0/2323882386 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 853815262 0 0) 0x7f085415b9c0 con 0x7f0854155950 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.402+0000 7f085159a640 1 -- 172.21.15.27:0/2323882386 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0840003520 con 0x7f0854155950 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.402+0000 7f085159a640 1 -- 172.21.15.27:0/2323882386 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3161267551 0 0) 0x7f085415bca0 con 0x7f08540b54f0 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.402+0000 7f085159a640 1 -- 172.21.15.27:0/2323882386 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f085415b9c0 con 0x7f08540b54f0 2024-04-20T11:02:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.402+0000 7f0852d9d640 1 -- 172.21.15.27:0/2323882386 >> v1:172.21.15.137:6789/0 conn(0x7f08540b54f0 legacy=0x7f08540b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.402+0000 7f0852d9d640 1 -- 172.21.15.27:0/2323882386 >> v1:172.21.15.27:6789/0 conn(0x7f0854155950 legacy=0x7f08540cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.402+0000 7f0852d9d640 1 -- 172.21.15.27:0/2323882386 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085415c260 con 0x7f08540d0c90 2024-04-20T11:02:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.403+0000 7f0852d9d640 1 --2- 172.21.15.27:0/2323882386 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540d0c90 0x7f08540d10e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0844009dc0 tx=0x7f084402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77dc2412cc19e9af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.403+0000 7f085159a640 1 -- 172.21.15.27:0/2323882386 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844039070 con 0x7f08540d0c90 2024-04-20T11:02:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.403+0000 7f085159a640 1 -- 172.21.15.27:0/2323882386 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084402cd50 con 0x7f08540d0c90 2024-04-20T11:02:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.403+0000 7f085159a640 1 -- 172.21.15.27:0/2323882386 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844034640 con 0x7f08540d0c90 2024-04-20T11:02:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.404+0000 7f0858ef2640 1 -- 172.21.15.27:0/2323882386 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540d0c90 msgr2=0x7f08540d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.404+0000 7f0858ef2640 1 --2- 172.21.15.27:0/2323882386 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540d0c90 0x7f08540d10e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0844009dc0 tx=0x7f084402bfa0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.404+0000 7f0858ef2640 1 -- 172.21.15.27:0/2323882386 shutdown_connections 2024-04-20T11:02:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.404+0000 7f0858ef2640 1 --2- 172.21.15.27:0/2323882386 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540d0c90 0x7f08540d10e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.404+0000 7f0858ef2640 1 -- 172.21.15.27:0/2323882386 >> 172.21.15.27:0/2323882386 conn(0x7f08540676c0 msgr2=0x7f0854067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:53.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.404+0000 7f0858ef2640 1 -- 172.21.15.27:0/2323882386 shutdown_connections 2024-04-20T11:02:53.405 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.404+0000 7f0858ef2640 1 -- 172.21.15.27:0/2323882386 wait complete. 2024-04-20T11:02:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.405+0000 7f0858ef2640 1 Processor -- start 2024-04-20T11:02:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.405+0000 7f0858ef2640 1 -- start start 2024-04-20T11:02:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0858ef2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540b54f0 0x7f0854159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0858ef2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08540d0c90 0x7f08541f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0858ef2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0854155950 0x7f0854148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0858ef2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f085415fb90 con 0x7f0854155950 2024-04-20T11:02:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0858ef2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08540d1620 con 0x7f08540d0c90 2024-04-20T11:02:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0858ef2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f085415fd10 con 0x7f08540b54f0 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0852d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0854155950 0x7f0854148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0852d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0854155950 0x7f0854148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42342/0 (socket says 172.21.15.27:42342) 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0852d9d640 1 -- 172.21.15.27:0/941495627 learned_addr learned my addr 172.21.15.27:0/941495627 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f085259c640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540b54f0 0x7f0854159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0851d9b640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08540d0c90 0x7f08541f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0852d9d640 1 -- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540b54f0 msgr2=0x7f0854159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0852d9d640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540b54f0 0x7f0854159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0852d9d640 1 -- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08540d0c90 msgr2=0x7f08541f5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0852d9d640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08540d0c90 0x7f08541f5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f0852d9d640 1 -- 172.21.15.27:0/941495627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085415ca10 con 0x7f0854155950 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.406+0000 7f085259c640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540b54f0 0x7f0854159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.407+0000 7f0852d9d640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0854155950 0x7f0854148860 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f084402c5b0 tx=0x7f0844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.407+0000 7f084affd640 1 -- 172.21.15.27:0/941495627 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844039050 con 0x7f0854155950 2024-04-20T11:02:53.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.407+0000 7f084affd640 1 -- 172.21.15.27:0/941495627 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0844045070 con 0x7f0854155950 2024-04-20T11:02:53.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.407+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08540cf710 con 0x7f0854155950 2024-04-20T11:02:53.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.407+0000 7f084affd640 1 -- 172.21.15.27:0/941495627 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844009450 con 0x7f0854155950 2024-04-20T11:02:53.409 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.407+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085415b6c0 con 0x7f0854155950 2024-04-20T11:02:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.408+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08540686d0 con 0x7f0854155950 2024-04-20T11:02:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.408+0000 7f084affd640 1 -- 172.21.15.27:0/941495627 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084403e850 con 0x7f0854155950 2024-04-20T11:02:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.409+0000 7f084affd640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f083406c920 0x7f083406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.409+0000 7f084affd640 1 -- 172.21.15.27:0/941495627 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f08440ae350 con 0x7f0854155950 2024-04-20T11:02:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.409+0000 7f085259c640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f083406c920 0x7f083406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.410+0000 7f085259c640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f083406c920 0x7f083406ede0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f084c007fd0 tx=0x7f084c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:53.413 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.413+0000 7f084affd640 1 -- 172.21.15.27:0/941495627 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f08540686d0 con 0x7f0854155950 2024-04-20T11:02:53.596 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.597+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 --> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f085404fcf0 con 0x7f083406c920 2024-04-20T11:02:53.603 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.604+0000 7f084affd640 1 -- 172.21.15.27:0/941495627 <== mgr.4106 v2:172.21.15.137:6832/2036904965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254411 (secure 0 0 0) 0x7f085404fcf0 con 0x7f083406c920 2024-04-20T11:02:53.606 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:53.608 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2024-04-20T11:02:53.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f083406c920 msgr2=0x7f083406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f083406c920 0x7f083406ede0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f084c007fd0 tx=0x7f084c009040 comp rx=0 tx=0).stop 2024-04-20T11:02:53.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0854155950 msgr2=0x7f0854148860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0854155950 0x7f0854148860 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f084402c5b0 tx=0x7f0844004a40 comp rx=0 tx=0).stop 2024-04-20T11:02:53.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 shutdown_connections 2024-04-20T11:02:53.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f083406c920 0x7f083406ede0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0854155950 0x7f0854148860 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08540d0c90 0x7f08541f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 --2- 172.21.15.27:0/941495627 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f08540b54f0 0x7f0854159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 >> 172.21.15.27:0/941495627 conn(0x7f08540676c0 msgr2=0x7f0854067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:53.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 shutdown_connections 2024-04-20T11:02:53.611 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.606+0000 7f0858ef2640 1 -- 172.21.15.27:0/941495627 wait complete. 2024-04-20T11:02:53.617 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-04-20T11:02:51.799305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":152723592,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":1845,"num_read_kb":3292270,"num_write":30950,"num_write_kb":4353509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35111,"ondisk_log_size":35111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":659848,"kb_used_data":300808,"kb_used_omap":12,"kb_used_meta":358963,"kb_avail":754314872,"statfs":{"total":773094113280,"available":772418428928,"internally_reserved":0,"allocated":308027392,"data_stored":306366936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":367578673},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003104"},"pg_stats":[{"pgid":"3.3f","version":"27'425","reported_seq":477,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:43.160042+0000","last_change":"2024-04-20T11:02:43.160105+0000","last_active":"2024-04-20T11:02:43.160042+0000","last_peered":"2024-04-20T11:02:43.160042+0000","last_clean":"2024-04-20T11:02:43.160042+0000","last_became_active":"2024-04-20T10:55:42.802590+0000","last_became_peered":"2024-04-20T10:55:42.802590+0000","last_unstale":"2024-04-20T11:02:43.160042+0000","last_undegraded":"2024-04-20T11:02:43.160042+0000","last_fullsized":"2024-04-20T11:02:43.160042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'425","last_scrub_stamp":"2024-04-20T11:02:43.159933+0000","last_deep_scrub":"27'425","last_deep_scrub_stamp":"2024-04-20T11:02:43.159933+0000","last_clean_scrub_stamp":"2024-04-20T11:02:43.159933+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:29:17.893910+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":31614,"num_write":375,"num_write_kb":59119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"27'429","reported_seq":483,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:36.481989+0000","last_change":"2024-04-20T11:02:36.482017+0000","last_active":"2024-04-20T11:02:36.481989+0000","last_peered":"2024-04-20T11:02:36.481989+0000","last_clean":"2024-04-20T11:02:36.481989+0000","last_became_active":"2024-04-20T10:55:42.794710+0000","last_became_peered":"2024-04-20T10:55:42.794710+0000","last_unstale":"2024-04-20T11:02:36.481989+0000","last_undegraded":"2024-04-20T11:02:36.481989+0000","last_fullsized":"2024-04-20T11:02:36.481989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'429","last_scrub_stamp":"2024-04-20T11:02:36.481895+0000","last_deep_scrub":"27'429","last_deep_scrub_stamp":"2024-04-20T11:02:36.481895+0000","last_clean_scrub_stamp":"2024-04-20T11:02:36.481895+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:20.471021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":40656,"num_write":384,"num_write_kb":54681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"27'406","reported_seq":463,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:33.778530+0000","last_change":"2024-04-20T11:02:33.778564+0000","last_active":"2024-04-20T11:02:33.778530+0000","last_peered":"2024-04-20T11:02:33.778530+0000","last_clean":"2024-04-20T11:02:33.778530+0000","last_became_active":"2024-04-20T10:55:42.798159+0000","last_became_peered":"2024-04-20T10:55:42.798159+0000","last_unstale":"2024-04-20T11:02:33.778530+0000","last_undegraded":"2024-04-20T11:02:33.778530+0000","last_fullsized":"2024-04-20T11:02:33.778530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'406","last_scrub_stamp":"2024-04-20T11:02:33.778449+0000","last_deep_scrub":"27'406","last_deep_scrub_stamp":"2024-04-20T11:02:33.778449+0000","last_clean_scrub_stamp":"2024-04-20T11:02:33.778449+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:58:17.631799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":42823,"num_write":366,"num_write_kb":47539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"27'392","reported_seq":443,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:44.225012+0000","last_change":"2024-04-20T11:02:44.225042+0000","last_active":"2024-04-20T11:02:44.225012+0000","last_peered":"2024-04-20T11:02:44.225012+0000","last_clean":"2024-04-20T11:02:44.225012+0000","last_became_active":"2024-04-20T10:55:42.796516+0000","last_became_peered":"2024-04-20T10:55:42.796516+0000","last_unstale":"2024-04-20T11:02:44.225012+0000","last_undegraded":"2024-04-20T11:02:44.225012+0000","last_fullsized":"2024-04-20T11:02:44.225012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'392","last_scrub_stamp":"2024-04-20T11:02:44.224914+0000","last_deep_scrub":"27'392","last_deep_scrub_stamp":"2024-04-20T11:02:44.224914+0000","last_clean_scrub_stamp":"2024-04-20T11:02:44.224914+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:41:36.571384+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":22108,"num_write":353,"num_write_kb":40048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'430","reported_seq":485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:41.167653+0000","last_change":"2024-04-20T11:02:41.167681+0000","last_active":"2024-04-20T11:02:41.167653+0000","last_peered":"2024-04-20T11:02:41.167653+0000","last_clean":"2024-04-20T11:02:41.167653+0000","last_became_active":"2024-04-20T10:55:42.799736+0000","last_became_peered":"2024-04-20T10:55:42.799736+0000","last_unstale":"2024-04-20T11:02:41.167653+0000","last_undegraded":"2024-04-20T11:02:41.167653+0000","last_fullsized":"2024-04-20T11:02:41.167653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'430","last_scrub_stamp":"2024-04-20T11:02:41.167566+0000","last_deep_scrub":"27'430","last_deep_scrub_stamp":"2024-04-20T11:02:41.167566+0000","last_clean_scrub_stamp":"2024-04-20T11:02:41.167566+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:47:10.882673+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":63832,"num_write":387,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'371","reported_seq":414,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:39.238672+0000","last_change":"2024-04-20T11:02:39.238700+0000","last_active":"2024-04-20T11:02:39.238672+0000","last_peered":"2024-04-20T11:02:39.238672+0000","last_clean":"2024-04-20T11:02:39.238672+0000","last_became_active":"2024-04-20T10:55:42.793423+0000","last_became_peered":"2024-04-20T10:55:42.793423+0000","last_unstale":"2024-04-20T11:02:39.238672+0000","last_undegraded":"2024-04-20T11:02:39.238672+0000","last_fullsized":"2024-04-20T11:02:39.238672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'371","last_scrub_stamp":"2024-04-20T11:02:39.238587+0000","last_deep_scrub":"27'371","last_deep_scrub_stamp":"2024-04-20T11:02:39.238587+0000","last_clean_scrub_stamp":"2024-04-20T11:02:39.238587+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:52:35.211239+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":22263,"num_write":332,"num_write_kb":22043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"27'436","reported_seq":492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:47.180345+0000","last_change":"2024-04-20T11:02:47.180393+0000","last_active":"2024-04-20T11:02:47.180345+0000","last_peered":"2024-04-20T11:02:47.180345+0000","last_clean":"2024-04-20T11:02:47.180345+0000","last_became_active":"2024-04-20T10:55:42.805311+0000","last_became_peered":"2024-04-20T10:55:42.805311+0000","last_unstale":"2024-04-20T11:02:47.180345+0000","last_undegraded":"2024-04-20T11:02:47.180345+0000","last_fullsized":"2024-04-20T11:02:47.180345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'436","last_scrub_stamp":"2024-04-20T11:02:47.180236+0000","last_deep_scrub":"27'436","last_deep_scrub_stamp":"2024-04-20T11:02:47.180236+0000","last_clean_scrub_stamp":"2024-04-20T11:02:47.180236+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:36:32.121530+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":74379,"num_write":399,"num_write_kb":92171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'369","reported_seq":418,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:35.680587+0000","last_change":"2024-04-20T11:02:35.680611+0000","last_active":"2024-04-20T11:02:35.680587+0000","last_peered":"2024-04-20T11:02:35.680587+0000","last_clean":"2024-04-20T11:02:35.680587+0000","last_became_active":"2024-04-20T10:55:42.794096+0000","last_became_peered":"2024-04-20T10:55:42.794096+0000","last_unstale":"2024-04-20T11:02:35.680587+0000","last_undegraded":"2024-04-20T11:02:35.680587+0000","last_fullsized":"2024-04-20T11:02:35.680587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'369","last_scrub_stamp":"2024-04-20T11:02:35.680490+0000","last_deep_scrub":"27'369","last_deep_scrub_stamp":"2024-04-20T11:02:35.680490+0000","last_clean_scrub_stamp":"2024-04-20T11:02:35.680490+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:34.640637+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":32797,"num_write":314,"num_write_kb":38008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"27'433","reported_seq":488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:48.206769+0000","last_change":"2024-04-20T11:02:48.206833+0000","last_active":"2024-04-20T11:02:48.206769+0000","last_peered":"2024-04-20T11:02:48.206769+0000","last_clean":"2024-04-20T11:02:48.206769+0000","last_became_active":"2024-04-20T10:55:42.806707+0000","last_became_peered":"2024-04-20T10:55:42.806707+0000","last_unstale":"2024-04-20T11:02:48.206769+0000","last_undegraded":"2024-04-20T11:02:48.206769+0000","last_fullsized":"2024-04-20T11:02:48.206769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'433","last_scrub_stamp":"2024-04-20T11:02:48.206659+0000","last_deep_scrub":"27'433","last_deep_scrub_stamp":"2024-04-20T11:02:48.206659+0000","last_clean_scrub_stamp":"2024-04-20T11:02:48.206659+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:43:09.942725+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":52787,"num_write":396,"num_write_kb":59707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"27'425","reported_seq":490,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:44.529985+0000","last_change":"2024-04-20T11:02:44.530012+0000","last_active":"2024-04-20T11:02:44.529985+0000","last_peered":"2024-04-20T11:02:44.529985+0000","last_clean":"2024-04-20T11:02:44.529985+0000","last_became_active":"2024-04-20T10:56:22.913113+0000","last_became_peered":"2024-04-20T10:56:22.913113+0000","last_unstale":"2024-04-20T11:02:44.529985+0000","last_undegraded":"2024-04-20T11:02:44.529985+0000","last_fullsized":"2024-04-20T11:02:44.529985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'425","last_scrub_stamp":"2024-04-20T11:02:44.529894+0000","last_deep_scrub":"27'425","last_deep_scrub_stamp":"2024-04-20T11:02:44.529894+0000","last_clean_scrub_stamp":"2024-04-20T11:02:44.529894+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:39:20.765802+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":77299,"num_write":373,"num_write_kb":63557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"27'387","reported_seq":440,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:42.565598+0000","last_change":"2024-04-20T11:02:42.565624+0000","last_active":"2024-04-20T11:02:42.565598+0000","last_peered":"2024-04-20T11:02:42.565598+0000","last_clean":"2024-04-20T11:02:42.565598+0000","last_became_active":"2024-04-20T10:55:42.802450+0000","last_became_peered":"2024-04-20T10:55:42.802450+0000","last_unstale":"2024-04-20T11:02:42.565598+0000","last_undegraded":"2024-04-20T11:02:42.565598+0000","last_fullsized":"2024-04-20T11:02:42.565598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'387","last_scrub_stamp":"2024-04-20T11:02:42.565493+0000","last_deep_scrub":"27'387","last_deep_scrub_stamp":"2024-04-20T11:02:42.565493+0000","last_clean_scrub_stamp":"2024-04-20T11:02:42.565493+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:26:27.817937+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":50462,"num_write":358,"num_write_kb":46678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"27'394","reported_seq":450,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:36.779484+0000","last_change":"2024-04-20T11:02:36.779509+0000","last_active":"2024-04-20T11:02:36.779484+0000","last_peered":"2024-04-20T11:02:36.779484+0000","last_clean":"2024-04-20T11:02:36.779484+0000","last_became_active":"2024-04-20T10:55:42.799811+0000","last_became_peered":"2024-04-20T10:55:42.799811+0000","last_unstale":"2024-04-20T11:02:36.779484+0000","last_undegraded":"2024-04-20T11:02:36.779484+0000","last_fullsized":"2024-04-20T11:02:36.779484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'394","last_scrub_stamp":"2024-04-20T11:02:36.779393+0000","last_deep_scrub":"27'394","last_deep_scrub_stamp":"2024-04-20T11:02:36.779393+0000","last_clean_scrub_stamp":"2024-04-20T11:02:36.779393+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:44.085377+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":64677,"num_write":357,"num_write_kb":71584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"27'415","reported_seq":473,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:43.695630+0000","last_change":"2024-04-20T11:02:43.695659+0000","last_active":"2024-04-20T11:02:43.695630+0000","last_peered":"2024-04-20T11:02:43.695630+0000","last_clean":"2024-04-20T11:02:43.695630+0000","last_became_active":"2024-04-20T10:55:42.803366+0000","last_became_peered":"2024-04-20T10:55:42.803366+0000","last_unstale":"2024-04-20T11:02:43.695630+0000","last_undegraded":"2024-04-20T11:02:43.695630+0000","last_fullsized":"2024-04-20T11:02:43.695630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'415","last_scrub_stamp":"2024-04-20T11:02:43.695521+0000","last_deep_scrub":"27'415","last_deep_scrub_stamp":"2024-04-20T11:02:43.695521+0000","last_clean_scrub_stamp":"2024-04-20T11:02:43.695521+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:57.291025+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":59589,"num_write":367,"num_write_kb":71789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"27'421","reported_seq":468,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:35.749515+0000","last_change":"2024-04-20T11:02:35.749515+0000","last_active":"2024-04-20T11:02:35.749515+0000","last_peered":"2024-04-20T11:02:35.749515+0000","last_clean":"2024-04-20T11:02:35.749515+0000","last_became_active":"2024-04-20T10:55:42.802118+0000","last_became_peered":"2024-04-20T10:55:42.802118+0000","last_unstale":"2024-04-20T11:02:35.749515+0000","last_undegraded":"2024-04-20T11:02:35.749515+0000","last_fullsized":"2024-04-20T11:02:35.749515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'421","last_scrub_stamp":"2024-04-20T11:02:35.749379+0000","last_deep_scrub":"27'421","last_deep_scrub_stamp":"2024-04-20T11:02:35.749379+0000","last_clean_scrub_stamp":"2024-04-20T11:02:35.749379+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:47:23.526015+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":30435,"num_write":362,"num_write_kb":42694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"27'419","reported_seq":470,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:41.600846+0000","last_change":"2024-04-20T11:02:41.600846+0000","last_active":"2024-04-20T11:02:41.600846+0000","last_peered":"2024-04-20T11:02:41.600846+0000","last_clean":"2024-04-20T11:02:41.600846+0000","last_became_active":"2024-04-20T10:55:42.800593+0000","last_became_peered":"2024-04-20T10:55:42.800593+0000","last_unstale":"2024-04-20T11:02:41.600846+0000","last_undegraded":"2024-04-20T11:02:41.600846+0000","last_fullsized":"2024-04-20T11:02:41.600846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'419","last_scrub_stamp":"2024-04-20T11:02:41.600707+0000","last_deep_scrub":"27'419","last_deep_scrub_stamp":"2024-04-20T11:02:41.600707+0000","last_clean_scrub_stamp":"2024-04-20T11:02:41.600707+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:44:32.818353+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":33945,"num_write":377,"num_write_kb":75986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"27'451","reported_seq":514,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:43.271958+0000","last_change":"2024-04-20T11:02:43.271958+0000","last_active":"2024-04-20T11:02:43.271958+0000","last_peered":"2024-04-20T11:02:43.271958+0000","last_clean":"2024-04-20T11:02:43.271958+0000","last_became_active":"2024-04-20T10:55:42.796184+0000","last_became_peered":"2024-04-20T10:55:42.796184+0000","last_unstale":"2024-04-20T11:02:43.271958+0000","last_undegraded":"2024-04-20T11:02:43.271958+0000","last_fullsized":"2024-04-20T11:02:43.271958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'451","last_scrub_stamp":"2024-04-20T11:02:43.271829+0000","last_deep_scrub":"27'451","last_deep_scrub_stamp":"2024-04-20T11:02:43.271829+0000","last_clean_scrub_stamp":"2024-04-20T11:02:43.271829+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:08:10.700050+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":54533,"num_write":414,"num_write_kb":39250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"27'427","reported_seq":487,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:47.184811+0000","last_change":"2024-04-20T11:02:47.184841+0000","last_active":"2024-04-20T11:02:47.184811+0000","last_peered":"2024-04-20T11:02:47.184811+0000","last_clean":"2024-04-20T11:02:47.184811+0000","last_became_active":"2024-04-20T10:55:42.800987+0000","last_became_peered":"2024-04-20T10:55:42.800987+0000","last_unstale":"2024-04-20T11:02:47.184811+0000","last_undegraded":"2024-04-20T11:02:47.184811+0000","last_fullsized":"2024-04-20T11:02:47.184811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'427","last_scrub_stamp":"2024-04-20T11:02:47.184712+0000","last_deep_scrub":"27'427","last_deep_scrub_stamp":"2024-04-20T11:02:47.184712+0000","last_clean_scrub_stamp":"2024-04-20T11:02:47.184712+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:05:22.510875+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":68755,"num_write":396,"num_write_kb":79570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"27'429","reported_seq":486,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:35.305737+0000","last_change":"2024-04-20T11:02:35.305737+0000","last_active":"2024-04-20T11:02:35.305737+0000","last_peered":"2024-04-20T11:02:35.305737+0000","last_clean":"2024-04-20T11:02:35.305737+0000","last_became_active":"2024-04-20T10:55:42.802817+0000","last_became_peered":"2024-04-20T10:55:42.802817+0000","last_unstale":"2024-04-20T11:02:35.305737+0000","last_undegraded":"2024-04-20T11:02:35.305737+0000","last_fullsized":"2024-04-20T11:02:35.305737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'429","last_scrub_stamp":"2024-04-20T11:02:35.305618+0000","last_deep_scrub":"27'429","last_deep_scrub_stamp":"2024-04-20T11:02:35.305618+0000","last_clean_scrub_stamp":"2024-04-20T11:02:35.305618+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:05:45.999781+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53186,"num_write":383,"num_write_kb":88017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"27'439","reported_seq":501,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:33.241061+0000","last_change":"2024-04-20T11:02:33.241087+0000","last_active":"2024-04-20T11:02:33.241061+0000","last_peered":"2024-04-20T11:02:33.241061+0000","last_clean":"2024-04-20T11:02:33.241061+0000","last_became_active":"2024-04-20T10:55:42.800372+0000","last_became_peered":"2024-04-20T10:55:42.800372+0000","last_unstale":"2024-04-20T11:02:33.241061+0000","last_undegraded":"2024-04-20T11:02:33.241061+0000","last_fullsized":"2024-04-20T11:02:33.241061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'439","last_scrub_stamp":"2024-04-20T11:02:33.240973+0000","last_deep_scrub":"27'439","last_deep_scrub_stamp":"2024-04-20T11:02:33.240973+0000","last_clean_scrub_stamp":"2024-04-20T11:02:33.240973+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:00.343707+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":86853,"num_write":391,"num_write_kb":112571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"27'455","reported_seq":506,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:38.558945+0000","last_change":"2024-04-20T11:02:38.558973+0000","last_active":"2024-04-20T11:02:38.558945+0000","last_peered":"2024-04-20T11:02:38.558945+0000","last_clean":"2024-04-20T11:02:38.558945+0000","last_became_active":"2024-04-20T10:55:42.795099+0000","last_became_peered":"2024-04-20T10:55:42.795099+0000","last_unstale":"2024-04-20T11:02:38.558945+0000","last_undegraded":"2024-04-20T11:02:38.558945+0000","last_fullsized":"2024-04-20T11:02:38.558945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'455","last_scrub_stamp":"2024-04-20T11:02:38.558852+0000","last_deep_scrub":"27'455","last_deep_scrub_stamp":"2024-04-20T11:02:38.558852+0000","last_clean_scrub_stamp":"2024-04-20T11:02:38.558852+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:13:17.210582+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36592,"num_write":417,"num_write_kb":59154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"27'423","reported_seq":482,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:39.706092+0000","last_change":"2024-04-20T11:02:39.706120+0000","last_active":"2024-04-20T11:02:39.706092+0000","last_peered":"2024-04-20T11:02:39.706092+0000","last_clean":"2024-04-20T11:02:39.706092+0000","last_became_active":"2024-04-20T10:55:42.804602+0000","last_became_peered":"2024-04-20T10:55:42.804602+0000","last_unstale":"2024-04-20T11:02:39.706092+0000","last_undegraded":"2024-04-20T11:02:39.706092+0000","last_fullsized":"2024-04-20T11:02:39.706092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'423","last_scrub_stamp":"2024-04-20T11:02:39.706004+0000","last_deep_scrub":"27'423","last_deep_scrub_stamp":"2024-04-20T11:02:39.706004+0000","last_clean_scrub_stamp":"2024-04-20T11:02:39.706004+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:22:33.665687+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":55674,"num_write":374,"num_write_kb":59056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"27'445","reported_seq":500,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:46.195731+0000","last_change":"2024-04-20T11:02:46.195758+0000","last_active":"2024-04-20T11:02:46.195731+0000","last_peered":"2024-04-20T11:02:46.195731+0000","last_clean":"2024-04-20T11:02:46.195731+0000","last_became_active":"2024-04-20T10:55:42.800679+0000","last_became_peered":"2024-04-20T10:55:42.800679+0000","last_unstale":"2024-04-20T11:02:46.195731+0000","last_undegraded":"2024-04-20T11:02:46.195731+0000","last_fullsized":"2024-04-20T11:02:46.195731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'445","last_scrub_stamp":"2024-04-20T11:02:46.195644+0000","last_deep_scrub":"27'445","last_deep_scrub_stamp":"2024-04-20T11:02:46.195644+0000","last_clean_scrub_stamp":"2024-04-20T11:02:46.195644+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:23:30.235405+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":66341,"num_write":402,"num_write_kb":96389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'409","reported_seq":455,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:41.682458+0000","last_change":"2024-04-20T11:02:41.682485+0000","last_active":"2024-04-20T11:02:41.682458+0000","last_peered":"2024-04-20T11:02:41.682458+0000","last_clean":"2024-04-20T11:02:41.682458+0000","last_became_active":"2024-04-20T10:55:42.797411+0000","last_became_peered":"2024-04-20T10:55:42.797411+0000","last_unstale":"2024-04-20T11:02:41.682458+0000","last_undegraded":"2024-04-20T11:02:41.682458+0000","last_fullsized":"2024-04-20T11:02:41.682458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'409","last_scrub_stamp":"2024-04-20T11:02:41.682364+0000","last_deep_scrub":"27'409","last_deep_scrub_stamp":"2024-04-20T11:02:41.682364+0000","last_clean_scrub_stamp":"2024-04-20T11:02:41.682364+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:50.976590+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":25926,"num_write":360,"num_write_kb":50488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'472","reported_seq":534,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:37.296057+0000","last_change":"2024-04-20T11:02:37.296108+0000","last_active":"2024-04-20T11:02:37.296057+0000","last_peered":"2024-04-20T11:02:37.296057+0000","last_clean":"2024-04-20T11:02:37.296057+0000","last_became_active":"2024-04-20T10:55:42.789896+0000","last_became_peered":"2024-04-20T10:55:42.789896+0000","last_unstale":"2024-04-20T11:02:37.296057+0000","last_undegraded":"2024-04-20T11:02:37.296057+0000","last_fullsized":"2024-04-20T11:02:37.296057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'472","last_scrub_stamp":"2024-04-20T11:02:37.295959+0000","last_deep_scrub":"27'472","last_deep_scrub_stamp":"2024-04-20T11:02:37.295959+0000","last_clean_scrub_stamp":"2024-04-20T11:02:37.295959+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:43:04.335630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":61071,"num_write":433,"num_write_kb":92498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"27'405","reported_seq":449,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:38.659951+0000","last_change":"2024-04-20T11:02:38.660010+0000","last_active":"2024-04-20T11:02:38.659951+0000","last_peered":"2024-04-20T11:02:38.659951+0000","last_clean":"2024-04-20T11:02:38.659951+0000","last_became_active":"2024-04-20T10:55:42.796199+0000","last_became_peered":"2024-04-20T10:55:42.796199+0000","last_unstale":"2024-04-20T11:02:38.659951+0000","last_undegraded":"2024-04-20T11:02:38.659951+0000","last_fullsized":"2024-04-20T11:02:38.659951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'405","last_scrub_stamp":"2024-04-20T11:02:38.659847+0000","last_deep_scrub":"27'405","last_deep_scrub_stamp":"2024-04-20T11:02:38.659847+0000","last_clean_scrub_stamp":"2024-04-20T11:02:38.659847+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:25:56.920259+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":21468,"num_write":380,"num_write_kb":33760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"27'460","reported_seq":501,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:40.470678+0000","last_change":"2024-04-20T11:02:40.470733+0000","last_active":"2024-04-20T11:02:40.470678+0000","last_peered":"2024-04-20T11:02:40.470678+0000","last_clean":"2024-04-20T11:02:40.470678+0000","last_became_active":"2024-04-20T10:55:42.798148+0000","last_became_peered":"2024-04-20T10:55:42.798148+0000","last_unstale":"2024-04-20T11:02:40.470678+0000","last_undegraded":"2024-04-20T11:02:40.470678+0000","last_fullsized":"2024-04-20T11:02:40.470678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'460","last_scrub_stamp":"2024-04-20T11:02:40.470570+0000","last_deep_scrub":"27'460","last_deep_scrub_stamp":"2024-04-20T11:02:40.470570+0000","last_clean_scrub_stamp":"2024-04-20T11:02:40.470570+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:21:12.045821+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8561,"num_write":404,"num_write_kb":47121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"27'432","reported_seq":496,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:39.524187+0000","last_change":"2024-04-20T11:02:39.524217+0000","last_active":"2024-04-20T11:02:39.524187+0000","last_peered":"2024-04-20T11:02:39.524187+0000","last_clean":"2024-04-20T11:02:39.524187+0000","last_became_active":"2024-04-20T10:55:42.796284+0000","last_became_peered":"2024-04-20T10:55:42.796284+0000","last_unstale":"2024-04-20T11:02:39.524187+0000","last_undegraded":"2024-04-20T11:02:39.524187+0000","last_fullsized":"2024-04-20T11:02:39.524187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'432","last_scrub_stamp":"2024-04-20T11:02:39.524088+0000","last_deep_scrub":"27'432","last_deep_scrub_stamp":"2024-04-20T11:02:39.524088+0000","last_clean_scrub_stamp":"2024-04-20T11:02:39.524088+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:26.742488+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":82311,"num_write":397,"num_write_kb":100756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"27'406","reported_seq":464,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:37.692291+0000","last_change":"2024-04-20T11:02:37.692329+0000","last_active":"2024-04-20T11:02:37.692291+0000","last_peered":"2024-04-20T11:02:37.692291+0000","last_clean":"2024-04-20T11:02:37.692291+0000","last_became_active":"2024-04-20T10:55:42.795511+0000","last_became_peered":"2024-04-20T10:55:42.795511+0000","last_unstale":"2024-04-20T11:02:37.692291+0000","last_undegraded":"2024-04-20T11:02:37.692291+0000","last_fullsized":"2024-04-20T11:02:37.692291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'406","last_scrub_stamp":"2024-04-20T11:02:37.692203+0000","last_deep_scrub":"27'406","last_deep_scrub_stamp":"2024-04-20T11:02:37.692203+0000","last_clean_scrub_stamp":"2024-04-20T11:02:37.692203+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:44:47.281541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":40362,"num_write":368,"num_write_kb":83934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"27'437","reported_seq":494,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:36.781275+0000","last_change":"2024-04-20T11:02:36.781307+0000","last_active":"2024-04-20T11:02:36.781275+0000","last_peered":"2024-04-20T11:02:36.781275+0000","last_clean":"2024-04-20T11:02:36.781275+0000","last_became_active":"2024-04-20T10:55:42.798901+0000","last_became_peered":"2024-04-20T10:55:42.798901+0000","last_unstale":"2024-04-20T11:02:36.781275+0000","last_undegraded":"2024-04-20T11:02:36.781275+0000","last_fullsized":"2024-04-20T11:02:36.781275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'437","last_scrub_stamp":"2024-04-20T11:02:36.781179+0000","last_deep_scrub":"27'437","last_deep_scrub_stamp":"2024-04-20T11:02:36.781179+0000","last_clean_scrub_stamp":"2024-04-20T11:02:36.781179+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:30.991548+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":51907,"num_write":395,"num_write_kb":76367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"27'479","reported_seq":540,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:42.303079+0000","last_change":"2024-04-20T11:02:42.303107+0000","last_active":"2024-04-20T11:02:42.303079+0000","last_peered":"2024-04-20T11:02:42.303079+0000","last_clean":"2024-04-20T11:02:42.303079+0000","last_became_active":"2024-04-20T10:55:42.796030+0000","last_became_peered":"2024-04-20T10:55:42.796030+0000","last_unstale":"2024-04-20T11:02:42.303079+0000","last_undegraded":"2024-04-20T11:02:42.303079+0000","last_fullsized":"2024-04-20T11:02:42.303079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'479","last_scrub_stamp":"2024-04-20T11:02:42.302988+0000","last_deep_scrub":"27'479","last_deep_scrub_stamp":"2024-04-20T11:02:42.302988+0000","last_clean_scrub_stamp":"2024-04-20T11:02:42.302988+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:01:17.851792+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":45044,"num_write":427,"num_write_kb":43197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"27'425","reported_seq":471,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:36.341340+0000","last_change":"2024-04-20T11:02:36.341367+0000","last_active":"2024-04-20T11:02:36.341340+0000","last_peered":"2024-04-20T11:02:36.341340+0000","last_clean":"2024-04-20T11:02:36.341340+0000","last_became_active":"2024-04-20T10:55:42.806759+0000","last_became_peered":"2024-04-20T10:55:42.806759+0000","last_unstale":"2024-04-20T11:02:36.341340+0000","last_undegraded":"2024-04-20T11:02:36.341340+0000","last_fullsized":"2024-04-20T11:02:36.341340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'425","last_scrub_stamp":"2024-04-20T11:02:36.341234+0000","last_deep_scrub":"27'425","last_deep_scrub_stamp":"2024-04-20T11:02:36.341234+0000","last_clean_scrub_stamp":"2024-04-20T11:02:36.341234+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:36:32.362755+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":30452,"num_write":380,"num_write_kb":32013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"27'454","reported_seq":499,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:43.548640+0000","last_change":"2024-04-20T11:02:43.548666+0000","last_active":"2024-04-20T11:02:43.548640+0000","last_peered":"2024-04-20T11:02:43.548640+0000","last_clean":"2024-04-20T11:02:43.548640+0000","last_became_active":"2024-04-20T10:55:42.805334+0000","last_became_peered":"2024-04-20T10:55:42.805334+0000","last_unstale":"2024-04-20T11:02:43.548640+0000","last_undegraded":"2024-04-20T11:02:43.548640+0000","last_fullsized":"2024-04-20T11:02:43.548640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'454","last_scrub_stamp":"2024-04-20T11:02:43.548518+0000","last_deep_scrub":"27'454","last_deep_scrub_stamp":"2024-04-20T11:02:43.548518+0000","last_clean_scrub_stamp":"2024-04-20T11:02:43.548518+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:38:17.980999+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":23674,"num_write":409,"num_write_kb":56716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"27'397","reported_seq":447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:33.753904+0000","last_change":"2024-04-20T11:02:33.753950+0000","last_active":"2024-04-20T11:02:33.753904+0000","last_peered":"2024-04-20T11:02:33.753904+0000","last_clean":"2024-04-20T11:02:33.753904+0000","last_became_active":"2024-04-20T10:55:42.795548+0000","last_became_peered":"2024-04-20T10:55:42.795548+0000","last_unstale":"2024-04-20T11:02:33.753904+0000","last_undegraded":"2024-04-20T11:02:33.753904+0000","last_fullsized":"2024-04-20T11:02:33.753904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'397","last_scrub_stamp":"2024-04-20T11:02:33.753810+0000","last_deep_scrub":"27'397","last_deep_scrub_stamp":"2024-04-20T11:02:33.753810+0000","last_clean_scrub_stamp":"2024-04-20T11:02:33.753810+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:08.756975+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":36784,"num_write":356,"num_write_kb":38078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"27'163","reported_seq":205,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:32.699656+0000","last_change":"2024-04-20T11:02:32.699713+0000","last_active":"2024-04-20T11:02:32.699656+0000","last_peered":"2024-04-20T11:02:32.699656+0000","last_clean":"2024-04-20T11:02:32.699656+0000","last_became_active":"2024-04-20T10:55:41.793152+0000","last_became_peered":"2024-04-20T10:55:41.793152+0000","last_unstale":"2024-04-20T11:02:32.699656+0000","last_undegraded":"2024-04-20T11:02:32.699656+0000","last_fullsized":"2024-04-20T11:02:32.699656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'163","last_scrub_stamp":"2024-04-20T11:02:32.699571+0000","last_deep_scrub":"27'163","last_deep_scrub_stamp":"2024-04-20T11:02:32.699571+0000","last_clean_scrub_stamp":"2024-04-20T11:02:32.699571+0000","objects_scrubbed":3,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:27:31.203068+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195439,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":112,"num_write_kb":4163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"27'403","reported_seq":448,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:32.788107+0000","last_change":"2024-04-20T11:02:32.788129+0000","last_active":"2024-04-20T11:02:32.788107+0000","last_peered":"2024-04-20T11:02:32.788107+0000","last_clean":"2024-04-20T11:02:32.788107+0000","last_became_active":"2024-04-20T10:55:42.795681+0000","last_became_peered":"2024-04-20T10:55:42.795681+0000","last_unstale":"2024-04-20T11:02:32.788107+0000","last_undegraded":"2024-04-20T11:02:32.788107+0000","last_fullsized":"2024-04-20T11:02:32.788107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'403","last_scrub_stamp":"2024-04-20T11:02:32.788038+0000","last_deep_scrub":"27'403","last_deep_scrub_stamp":"2024-04-20T11:02:32.788038+0000","last_clean_scrub_stamp":"2024-04-20T11:02:32.788038+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:18:38.282523+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":26712,"num_write":367,"num_write_kb":67448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"27'328","reported_seq":373,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:34.556229+0000","last_change":"2024-04-20T11:02:34.556269+0000","last_active":"2024-04-20T11:02:34.556229+0000","last_peered":"2024-04-20T11:02:34.556229+0000","last_clean":"2024-04-20T11:02:34.556229+0000","last_became_active":"2024-04-20T10:55:41.794725+0000","last_became_peered":"2024-04-20T10:55:41.794725+0000","last_unstale":"2024-04-20T11:02:34.556229+0000","last_undegraded":"2024-04-20T11:02:34.556229+0000","last_fullsized":"2024-04-20T11:02:34.556229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'328","last_scrub_stamp":"2024-04-20T11:02:34.556139+0000","last_deep_scrub":"27'328","last_deep_scrub_stamp":"2024-04-20T11:02:34.556139+0000","last_clean_scrub_stamp":"2024-04-20T11:02:34.556139+0000","objects_scrubbed":6,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:06.234577+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":260,"num_write_kb":12413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"27'98","reported_seq":141,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:33.727264+0000","last_change":"2024-04-20T11:02:33.727298+0000","last_active":"2024-04-20T11:02:33.727264+0000","last_peered":"2024-04-20T11:02:33.727264+0000","last_clean":"2024-04-20T11:02:33.727264+0000","last_became_active":"2024-04-20T10:55:41.795174+0000","last_became_peered":"2024-04-20T10:55:41.795174+0000","last_unstale":"2024-04-20T11:02:33.727264+0000","last_undegraded":"2024-04-20T11:02:33.727264+0000","last_fullsized":"2024-04-20T11:02:33.727264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'98","last_scrub_stamp":"2024-04-20T11:02:33.727180+0000","last_deep_scrub":"27'98","last_deep_scrub_stamp":"2024-04-20T11:02:33.727180+0000","last_clean_scrub_stamp":"2024-04-20T11:02:33.727180+0000","objects_scrubbed":3,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:18:21.811081+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":20,"num_write":51,"num_write_kb":4136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"27'449","reported_seq":495,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:39.689340+0000","last_change":"2024-04-20T11:02:39.689375+0000","last_active":"2024-04-20T11:02:39.689340+0000","last_peered":"2024-04-20T11:02:39.689340+0000","last_clean":"2024-04-20T11:02:39.689340+0000","last_became_active":"2024-04-20T10:55:42.797183+0000","last_became_peered":"2024-04-20T10:55:42.797183+0000","last_unstale":"2024-04-20T11:02:39.689340+0000","last_undegraded":"2024-04-20T11:02:39.689340+0000","last_fullsized":"2024-04-20T11:02:39.689340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'449","last_scrub_stamp":"2024-04-20T11:02:39.689240+0000","last_deep_scrub":"27'449","last_deep_scrub_stamp":"2024-04-20T11:02:39.689240+0000","last_clean_scrub_stamp":"2024-04-20T11:02:39.689240+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:22:15.171328+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":16783,"num_write":403,"num_write_kb":42765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"27'800","reported_seq":840,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:32.184954+0000","last_change":"2024-04-20T11:02:32.185009+0000","last_active":"2024-04-20T11:02:32.184954+0000","last_peered":"2024-04-20T11:02:32.184954+0000","last_clean":"2024-04-20T11:02:32.184954+0000","last_became_active":"2024-04-20T10:55:41.792887+0000","last_became_peered":"2024-04-20T10:55:41.792887+0000","last_unstale":"2024-04-20T11:02:32.184954+0000","last_undegraded":"2024-04-20T11:02:32.184954+0000","last_fullsized":"2024-04-20T11:02:32.184954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'800","last_scrub_stamp":"2024-04-20T11:02:32.184837+0000","last_deep_scrub":"27'800","last_deep_scrub_stamp":"2024-04-20T11:02:32.184837+0000","last_clean_scrub_stamp":"2024-04-20T11:02:32.184837+0000","objects_scrubbed":3,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:26:47.504048+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":724,"num_write_kb":8631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"27'457","reported_seq":529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:40.554361+0000","last_change":"2024-04-20T11:02:40.554394+0000","last_active":"2024-04-20T11:02:40.554361+0000","last_peered":"2024-04-20T11:02:40.554361+0000","last_clean":"2024-04-20T11:02:40.554361+0000","last_became_active":"2024-04-20T10:55:42.798630+0000","last_became_peered":"2024-04-20T10:55:42.798630+0000","last_unstale":"2024-04-20T11:02:40.554361+0000","last_undegraded":"2024-04-20T11:02:40.554361+0000","last_fullsized":"2024-04-20T11:02:40.554361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'457","last_scrub_stamp":"2024-04-20T11:02:40.554259+0000","last_deep_scrub":"27'457","last_deep_scrub_stamp":"2024-04-20T11:02:40.554259+0000","last_clean_scrub_stamp":"2024-04-20T11:02:40.554259+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:29:27.911137+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":131398,"num_write":411,"num_write_kb":92477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"27'75","reported_seq":118,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:37.592041+0000","last_change":"2024-04-20T11:02:37.592072+0000","last_active":"2024-04-20T11:02:37.592041+0000","last_peered":"2024-04-20T11:02:37.592041+0000","last_clean":"2024-04-20T11:02:37.592041+0000","last_became_active":"2024-04-20T10:55:41.799365+0000","last_became_peered":"2024-04-20T10:55:41.799365+0000","last_unstale":"2024-04-20T11:02:37.592041+0000","last_undegraded":"2024-04-20T11:02:37.592041+0000","last_fullsized":"2024-04-20T11:02:37.592041+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'75","last_scrub_stamp":"2024-04-20T11:02:37.591955+0000","last_deep_scrub":"27'75","last_deep_scrub_stamp":"2024-04-20T11:02:37.591955+0000","last_clean_scrub_stamp":"2024-04-20T11:02:37.591955+0000","objects_scrubbed":3,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:26:04.129983+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":22,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"27'455","reported_seq":517,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:45.178489+0000","last_change":"2024-04-20T11:02:45.178520+0000","last_active":"2024-04-20T11:02:45.178489+0000","last_peered":"2024-04-20T11:02:45.178489+0000","last_clean":"2024-04-20T11:02:45.178489+0000","last_became_active":"2024-04-20T10:55:42.798585+0000","last_became_peered":"2024-04-20T10:55:42.798585+0000","last_unstale":"2024-04-20T11:02:45.178489+0000","last_undegraded":"2024-04-20T11:02:45.178489+0000","last_fullsized":"2024-04-20T11:02:45.178489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'455","last_scrub_stamp":"2024-04-20T11:02:45.178379+0000","last_deep_scrub":"27'455","last_deep_scrub_stamp":"2024-04-20T11:02:45.178379+0000","last_clean_scrub_stamp":"2024-04-20T11:02:45.178379+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:50.904525+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":75132,"num_write":413,"num_write_kb":100522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"27'98","reported_seq":143,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:35.223686+0000","last_change":"2024-04-20T11:02:35.223755+0000","last_active":"2024-04-20T11:02:35.223686+0000","last_peered":"2024-04-20T11:02:35.223686+0000","last_clean":"2024-04-20T11:02:35.223686+0000","last_became_active":"2024-04-20T10:55:41.798145+0000","last_became_peered":"2024-04-20T10:55:41.798145+0000","last_unstale":"2024-04-20T11:02:35.223686+0000","last_undegraded":"2024-04-20T11:02:35.223686+0000","last_fullsized":"2024-04-20T11:02:35.223686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'98","last_scrub_stamp":"2024-04-20T11:02:35.223580+0000","last_deep_scrub":"27'98","last_deep_scrub_stamp":"2024-04-20T11:02:35.223580+0000","last_clean_scrub_stamp":"2024-04-20T11:02:35.223580+0000","objects_scrubbed":3,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:20:53.059224+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":51,"num_write_kb":4153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"27'450","reported_seq":541,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:37.764560+0000","last_change":"2024-04-20T11:02:37.764585+0000","last_active":"2024-04-20T11:02:37.764560+0000","last_peered":"2024-04-20T11:02:37.764560+0000","last_clean":"2024-04-20T11:02:37.764560+0000","last_became_active":"2024-04-20T10:55:42.802800+0000","last_became_peered":"2024-04-20T10:55:42.802800+0000","last_unstale":"2024-04-20T11:02:37.764560+0000","last_undegraded":"2024-04-20T11:02:37.764560+0000","last_fullsized":"2024-04-20T11:02:37.764560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'450","last_scrub_stamp":"2024-04-20T11:02:37.764455+0000","last_deep_scrub":"27'450","last_deep_scrub_stamp":"2024-04-20T11:02:37.764455+0000","last_clean_scrub_stamp":"2024-04-20T11:02:37.764455+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:57.035593+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":54,"num_read_kb":176347,"num_write":418,"num_write_kb":171699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"27'272","reported_seq":313,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:35.526223+0000","last_change":"2024-04-20T11:02:35.526291+0000","last_active":"2024-04-20T11:02:35.526223+0000","last_peered":"2024-04-20T11:02:35.526223+0000","last_clean":"2024-04-20T11:02:35.526223+0000","last_became_active":"2024-04-20T10:55:41.798252+0000","last_became_peered":"2024-04-20T10:55:41.798252+0000","last_unstale":"2024-04-20T11:02:35.526223+0000","last_undegraded":"2024-04-20T11:02:35.526223+0000","last_fullsized":"2024-04-20T11:02:35.526223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'272","last_scrub_stamp":"2024-04-20T11:02:35.526129+0000","last_deep_scrub":"27'272","last_deep_scrub_stamp":"2024-04-20T11:02:35.526129+0000","last_clean_scrub_stamp":"2024-04-20T11:02:35.526129+0000","objects_scrubbed":5,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:34:37.810369+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":8,"num_write":225,"num_write_kb":4349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'408","reported_seq":465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:35.761713+0000","last_change":"2024-04-20T11:02:35.761713+0000","last_active":"2024-04-20T11:02:35.761713+0000","last_peered":"2024-04-20T11:02:35.761713+0000","last_clean":"2024-04-20T11:02:35.761713+0000","last_became_active":"2024-04-20T10:55:42.797993+0000","last_became_peered":"2024-04-20T10:55:42.797993+0000","last_unstale":"2024-04-20T11:02:35.761713+0000","last_undegraded":"2024-04-20T11:02:35.761713+0000","last_fullsized":"2024-04-20T11:02:35.761713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'408","last_scrub_stamp":"2024-04-20T11:02:35.761600+0000","last_deep_scrub":"27'408","last_deep_scrub_stamp":"2024-04-20T11:02:35.761600+0000","last_clean_scrub_stamp":"2024-04-20T11:02:35.761600+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:15:08.011327+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":53530,"num_write":359,"num_write_kb":71690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"27'403","reported_seq":449,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:35.595648+0000","last_change":"2024-04-20T11:02:35.595648+0000","last_active":"2024-04-20T11:02:35.595648+0000","last_peered":"2024-04-20T11:02:35.595648+0000","last_clean":"2024-04-20T11:02:35.595648+0000","last_became_active":"2024-04-20T10:55:41.795280+0000","last_became_peered":"2024-04-20T10:55:41.795280+0000","last_unstale":"2024-04-20T11:02:35.595648+0000","last_undegraded":"2024-04-20T11:02:35.595648+0000","last_fullsized":"2024-04-20T11:02:35.595648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'403","last_scrub_stamp":"2024-04-20T11:02:35.595503+0000","last_deep_scrub":"27'403","last_deep_scrub_stamp":"2024-04-20T11:02:35.595503+0000","last_clean_scrub_stamp":"2024-04-20T11:02:35.595503+0000","objects_scrubbed":8,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:56:34.086728+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":365,"num_write_kb":12513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"27'429","reported_seq":489,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:42.169945+0000","last_change":"2024-04-20T11:02:42.169973+0000","last_active":"2024-04-20T11:02:42.169945+0000","last_peered":"2024-04-20T11:02:42.169945+0000","last_clean":"2024-04-20T11:02:42.169945+0000","last_became_active":"2024-04-20T10:55:42.801562+0000","last_became_peered":"2024-04-20T10:55:42.801562+0000","last_unstale":"2024-04-20T11:02:42.169945+0000","last_undegraded":"2024-04-20T11:02:42.169945+0000","last_fullsized":"2024-04-20T11:02:42.169945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'429","last_scrub_stamp":"2024-04-20T11:02:42.169851+0000","last_deep_scrub":"27'429","last_deep_scrub_stamp":"2024-04-20T11:02:42.169851+0000","last_clean_scrub_stamp":"2024-04-20T11:02:42.169851+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:09.030329+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":72220,"num_write":383,"num_write_kb":72034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"27'365","reported_seq":414,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:29.852379+0000","last_change":"2024-04-20T11:02:29.852379+0000","last_active":"2024-04-20T11:02:29.852379+0000","last_peered":"2024-04-20T11:02:29.852379+0000","last_clean":"2024-04-20T11:02:29.852379+0000","last_became_active":"2024-04-20T10:55:41.793114+0000","last_became_peered":"2024-04-20T10:55:41.793114+0000","last_unstale":"2024-04-20T11:02:29.852379+0000","last_undegraded":"2024-04-20T11:02:29.852379+0000","last_fullsized":"2024-04-20T11:02:29.852379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'365","last_scrub_stamp":"2024-04-20T11:02:29.852266+0000","last_deep_scrub":"27'365","last_deep_scrub_stamp":"2024-04-20T11:02:29.852266+0000","last_clean_scrub_stamp":"2024-04-20T11:02:29.852266+0000","objects_scrubbed":8,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:17:39.238573+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":299,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"27'406","reported_seq":453,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:40.721450+0000","last_change":"2024-04-20T11:02:40.721450+0000","last_active":"2024-04-20T11:02:40.721450+0000","last_peered":"2024-04-20T11:02:40.721450+0000","last_clean":"2024-04-20T11:02:40.721450+0000","last_became_active":"2024-04-20T10:55:42.804460+0000","last_became_peered":"2024-04-20T10:55:42.804460+0000","last_unstale":"2024-04-20T11:02:40.721450+0000","last_undegraded":"2024-04-20T11:02:40.721450+0000","last_fullsized":"2024-04-20T11:02:40.721450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'406","last_scrub_stamp":"2024-04-20T11:02:40.721290+0000","last_deep_scrub":"27'406","last_deep_scrub_stamp":"2024-04-20T11:02:40.721290+0000","last_clean_scrub_stamp":"2024-04-20T11:02:40.721290+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:07:32.757777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":32812,"num_write":361,"num_write_kb":30179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"27'76","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:34.166070+0000","last_change":"2024-04-20T11:02:34.166070+0000","last_active":"2024-04-20T11:02:34.166070+0000","last_peered":"2024-04-20T11:02:34.166070+0000","last_clean":"2024-04-20T11:02:34.166070+0000","last_became_active":"2024-04-20T10:55:41.795681+0000","last_became_peered":"2024-04-20T10:55:41.795681+0000","last_unstale":"2024-04-20T11:02:34.166070+0000","last_undegraded":"2024-04-20T11:02:34.166070+0000","last_fullsized":"2024-04-20T11:02:34.166070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'76","last_scrub_stamp":"2024-04-20T11:02:34.165945+0000","last_deep_scrub":"27'76","last_deep_scrub_stamp":"2024-04-20T11:02:34.165945+0000","last_clean_scrub_stamp":"2024-04-20T11:02:34.165945+0000","objects_scrubbed":3,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:20:54.374333+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":17,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"27'418","reported_seq":470,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:37.357910+0000","last_change":"2024-04-20T11:02:37.357937+0000","last_active":"2024-04-20T11:02:37.357910+0000","last_peered":"2024-04-20T11:02:37.357910+0000","last_clean":"2024-04-20T11:02:37.357910+0000","last_became_active":"2024-04-20T10:56:22.904118+0000","last_became_peered":"2024-04-20T10:56:22.904118+0000","last_unstale":"2024-04-20T11:02:37.357910+0000","last_undegraded":"2024-04-20T11:02:37.357910+0000","last_fullsized":"2024-04-20T11:02:37.357910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'418","last_scrub_stamp":"2024-04-20T11:02:37.357813+0000","last_deep_scrub":"27'418","last_deep_scrub_stamp":"2024-04-20T11:02:37.357813+0000","last_clean_scrub_stamp":"2024-04-20T11:02:37.357813+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:03:14.908345+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":63538,"num_write":372,"num_write_kb":113421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"27'81","reported_seq":126,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:33.489683+0000","last_change":"2024-04-20T11:02:33.489683+0000","last_active":"2024-04-20T11:02:33.489683+0000","last_peered":"2024-04-20T11:02:33.489683+0000","last_clean":"2024-04-20T11:02:33.489683+0000","last_became_active":"2024-04-20T10:55:41.794693+0000","last_became_peered":"2024-04-20T10:55:41.794693+0000","last_unstale":"2024-04-20T11:02:33.489683+0000","last_undegraded":"2024-04-20T11:02:33.489683+0000","last_fullsized":"2024-04-20T11:02:33.489683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'81","last_scrub_stamp":"2024-04-20T11:02:33.489540+0000","last_deep_scrub":"27'81","last_deep_scrub_stamp":"2024-04-20T11:02:33.489540+0000","last_clean_scrub_stamp":"2024-04-20T11:02:33.489540+0000","objects_scrubbed":3,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:26:30.321484+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":10,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"27'444","reported_seq":502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:41.518613+0000","last_change":"2024-04-20T11:02:41.518613+0000","last_active":"2024-04-20T11:02:41.518613+0000","last_peered":"2024-04-20T11:02:41.518613+0000","last_clean":"2024-04-20T11:02:41.518613+0000","last_became_active":"2024-04-20T10:55:42.798457+0000","last_became_peered":"2024-04-20T10:55:42.798457+0000","last_unstale":"2024-04-20T11:02:41.518613+0000","last_undegraded":"2024-04-20T11:02:41.518613+0000","last_fullsized":"2024-04-20T11:02:41.518613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'444","last_scrub_stamp":"2024-04-20T11:02:41.518481+0000","last_deep_scrub":"27'444","last_deep_scrub_stamp":"2024-04-20T11:02:41.518481+0000","last_clean_scrub_stamp":"2024-04-20T11:02:41.518481+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:04:03.676153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":68877,"num_write":411,"num_write_kb":59157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"27'371","reported_seq":417,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:31.345421+0000","last_change":"2024-04-20T11:02:31.345421+0000","last_active":"2024-04-20T11:02:31.345421+0000","last_peered":"2024-04-20T11:02:31.345421+0000","last_clean":"2024-04-20T11:02:31.345421+0000","last_became_active":"2024-04-20T10:55:41.788854+0000","last_became_peered":"2024-04-20T10:55:41.788854+0000","last_unstale":"2024-04-20T11:02:31.345421+0000","last_undegraded":"2024-04-20T11:02:31.345421+0000","last_fullsized":"2024-04-20T11:02:31.345421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'371","last_scrub_stamp":"2024-04-20T11:02:31.345298+0000","last_deep_scrub":"27'371","last_deep_scrub_stamp":"2024-04-20T11:02:31.345298+0000","last_clean_scrub_stamp":"2024-04-20T11:02:31.345298+0000","objects_scrubbed":10,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:37:39.502127+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":307,"num_write_kb":8375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"27'477","reported_seq":541,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:44.206654+0000","last_change":"2024-04-20T11:02:44.206654+0000","last_active":"2024-04-20T11:02:44.206654+0000","last_peered":"2024-04-20T11:02:44.206654+0000","last_clean":"2024-04-20T11:02:44.206654+0000","last_became_active":"2024-04-20T10:55:42.803791+0000","last_became_peered":"2024-04-20T10:55:42.803791+0000","last_unstale":"2024-04-20T11:02:44.206654+0000","last_undegraded":"2024-04-20T11:02:44.206654+0000","last_fullsized":"2024-04-20T11:02:44.206654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'477","last_scrub_stamp":"2024-04-20T11:02:44.206525+0000","last_deep_scrub":"27'477","last_deep_scrub_stamp":"2024-04-20T11:02:44.206525+0000","last_clean_scrub_stamp":"2024-04-20T11:02:44.206525+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:15:10.197391+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":59233,"num_write":434,"num_write_kb":71822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'78","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:31.840891+0000","last_change":"2024-04-20T11:02:31.840922+0000","last_active":"2024-04-20T11:02:31.840891+0000","last_peered":"2024-04-20T11:02:31.840891+0000","last_clean":"2024-04-20T11:02:31.840891+0000","last_became_active":"2024-04-20T10:55:41.795413+0000","last_became_peered":"2024-04-20T10:55:41.795413+0000","last_unstale":"2024-04-20T11:02:31.840891+0000","last_undegraded":"2024-04-20T11:02:31.840891+0000","last_fullsized":"2024-04-20T11:02:31.840891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'78","last_scrub_stamp":"2024-04-20T11:02:31.840806+0000","last_deep_scrub":"27'78","last_deep_scrub_stamp":"2024-04-20T11:02:31.840806+0000","last_clean_scrub_stamp":"2024-04-20T11:02:31.840806+0000","objects_scrubbed":3,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:58.395228+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"27'413","reported_seq":484,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:46.195594+0000","last_change":"2024-04-20T11:02:46.195618+0000","last_active":"2024-04-20T11:02:46.195594+0000","last_peered":"2024-04-20T11:02:46.195594+0000","last_clean":"2024-04-20T11:02:46.195594+0000","last_became_active":"2024-04-20T10:55:42.803817+0000","last_became_peered":"2024-04-20T10:55:42.803817+0000","last_unstale":"2024-04-20T11:02:46.195594+0000","last_undegraded":"2024-04-20T11:02:46.195594+0000","last_fullsized":"2024-04-20T11:02:46.195594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'413","last_scrub_stamp":"2024-04-20T11:02:46.195489+0000","last_deep_scrub":"27'413","last_deep_scrub_stamp":"2024-04-20T11:02:46.195489+0000","last_clean_scrub_stamp":"2024-04-20T11:02:46.195489+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:24:11.014151+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":108675,"num_write":383,"num_write_kb":121193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'265","reported_seq":309,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:32.334023+0000","last_change":"2024-04-20T11:02:32.334050+0000","last_active":"2024-04-20T11:02:32.334023+0000","last_peered":"2024-04-20T11:02:32.334023+0000","last_clean":"2024-04-20T11:02:32.334023+0000","last_became_active":"2024-04-20T10:55:41.790860+0000","last_became_peered":"2024-04-20T10:55:41.790860+0000","last_unstale":"2024-04-20T11:02:32.334023+0000","last_undegraded":"2024-04-20T11:02:32.334023+0000","last_fullsized":"2024-04-20T11:02:32.334023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'265","last_scrub_stamp":"2024-04-20T11:02:32.333946+0000","last_deep_scrub":"27'265","last_deep_scrub_stamp":"2024-04-20T11:02:32.333946+0000","last_clean_scrub_stamp":"2024-04-20T11:02:32.333946+0000","objects_scrubbed":3,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:15.147375+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":218,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"27'393","reported_seq":444,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:45.236421+0000","last_change":"2024-04-20T11:02:45.236448+0000","last_active":"2024-04-20T11:02:45.236421+0000","last_peered":"2024-04-20T11:02:45.236421+0000","last_clean":"2024-04-20T11:02:45.236421+0000","last_became_active":"2024-04-20T10:55:42.804182+0000","last_became_peered":"2024-04-20T10:55:42.804182+0000","last_unstale":"2024-04-20T11:02:45.236421+0000","last_undegraded":"2024-04-20T11:02:45.236421+0000","last_fullsized":"2024-04-20T11:02:45.236421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'393","last_scrub_stamp":"2024-04-20T11:02:45.236332+0000","last_deep_scrub":"27'393","last_deep_scrub_stamp":"2024-04-20T11:02:45.236332+0000","last_clean_scrub_stamp":"2024-04-20T11:02:45.236332+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:13:50.814704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":38645,"num_write":351,"num_write_kb":38372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"27'466","reported_seq":506,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:30.463699+0000","last_change":"2024-04-20T11:02:30.463729+0000","last_active":"2024-04-20T11:02:30.463699+0000","last_peered":"2024-04-20T11:02:30.463699+0000","last_clean":"2024-04-20T11:02:30.463699+0000","last_became_active":"2024-04-20T10:55:41.787702+0000","last_became_peered":"2024-04-20T10:55:41.787702+0000","last_unstale":"2024-04-20T11:02:30.463699+0000","last_undegraded":"2024-04-20T11:02:30.463699+0000","last_fullsized":"2024-04-20T11:02:30.463699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'466","last_scrub_stamp":"2024-04-20T11:02:30.463624+0000","last_deep_scrub":"27'466","last_deep_scrub_stamp":"2024-04-20T11:02:30.463624+0000","last_clean_scrub_stamp":"2024-04-20T11:02:30.463624+0000","objects_scrubbed":2,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:36:08.205659+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"27'466","reported_seq":516,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:41.291727+0000","last_change":"2024-04-20T11:02:41.291771+0000","last_active":"2024-04-20T11:02:41.291727+0000","last_peered":"2024-04-20T11:02:41.291727+0000","last_clean":"2024-04-20T11:02:41.291727+0000","last_became_active":"2024-04-20T10:55:42.794695+0000","last_became_peered":"2024-04-20T10:55:42.794695+0000","last_unstale":"2024-04-20T11:02:41.291727+0000","last_undegraded":"2024-04-20T11:02:41.291727+0000","last_fullsized":"2024-04-20T11:02:41.291727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'466","last_scrub_stamp":"2024-04-20T11:02:41.291614+0000","last_deep_scrub":"27'466","last_deep_scrub_stamp":"2024-04-20T11:02:41.291614+0000","last_clean_scrub_stamp":"2024-04-20T11:02:41.291614+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:04.094074+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":42813,"num_write":427,"num_write_kb":83769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"27'73","reported_seq":118,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:34.339604+0000","last_change":"2024-04-20T11:02:34.339648+0000","last_active":"2024-04-20T11:02:34.339604+0000","last_peered":"2024-04-20T11:02:34.339604+0000","last_clean":"2024-04-20T11:02:34.339604+0000","last_became_active":"2024-04-20T10:55:41.798219+0000","last_became_peered":"2024-04-20T10:55:41.798219+0000","last_unstale":"2024-04-20T11:02:34.339604+0000","last_undegraded":"2024-04-20T11:02:34.339604+0000","last_fullsized":"2024-04-20T11:02:34.339604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'73","last_scrub_stamp":"2024-04-20T11:02:34.339503+0000","last_deep_scrub":"27'73","last_deep_scrub_stamp":"2024-04-20T11:02:34.339503+0000","last_clean_scrub_stamp":"2024-04-20T11:02:34.339503+0000","objects_scrubbed":3,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:16:01.256121+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":8,"num_write":11,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"27'404","reported_seq":445,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:49.171636+0000","last_change":"2024-04-20T11:02:49.171681+0000","last_active":"2024-04-20T11:02:49.171636+0000","last_peered":"2024-04-20T11:02:49.171636+0000","last_clean":"2024-04-20T11:02:49.171636+0000","last_became_active":"2024-04-20T10:55:42.806717+0000","last_became_peered":"2024-04-20T10:55:42.806717+0000","last_unstale":"2024-04-20T11:02:49.171636+0000","last_undegraded":"2024-04-20T11:02:49.171636+0000","last_fullsized":"2024-04-20T11:02:49.171636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'404","last_scrub_stamp":"2024-04-20T11:02:49.171519+0000","last_deep_scrub":"27'404","last_deep_scrub_stamp":"2024-04-20T11:02:49.171519+0000","last_clean_scrub_stamp":"2024-04-20T11:02:49.171519+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:12:52.308662+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6180,"num_write":356,"num_write_kb":9278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":198,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:34.228545+0000","last_change":"2024-04-20T11:02:34.228610+0000","last_active":"2024-04-20T11:02:34.228545+0000","last_peered":"2024-04-20T11:02:34.228545+0000","last_clean":"2024-04-20T11:02:34.228545+0000","last_became_active":"2024-04-20T10:55:30.923098+0000","last_became_peered":"2024-04-20T10:55:30.923098+0000","last_unstale":"2024-04-20T11:02:34.228545+0000","last_undegraded":"2024-04-20T11:02:34.228545+0000","last_fullsized":"2024-04-20T11:02:34.228545+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-04-20T11:02:34.228430+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-04-20T11:02:34.228430+0000","last_clean_scrub_stamp":"2024-04-20T11:02:34.228430+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:08:58.668543+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"27'82","reported_seq":129,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:33.191357+0000","last_change":"2024-04-20T11:02:33.191388+0000","last_active":"2024-04-20T11:02:33.191357+0000","last_peered":"2024-04-20T11:02:33.191357+0000","last_clean":"2024-04-20T11:02:33.191357+0000","last_became_active":"2024-04-20T10:55:41.792377+0000","last_became_peered":"2024-04-20T10:55:41.792377+0000","last_unstale":"2024-04-20T11:02:33.191357+0000","last_undegraded":"2024-04-20T11:02:33.191357+0000","last_fullsized":"2024-04-20T11:02:33.191357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'82","last_scrub_stamp":"2024-04-20T11:02:33.191247+0000","last_deep_scrub":"27'82","last_deep_scrub_stamp":"2024-04-20T11:02:33.191247+0000","last_clean_scrub_stamp":"2024-04-20T11:02:33.191247+0000","objects_scrubbed":5,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:41:11.758063+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":18,"num_write":19,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"27'360","reported_seq":404,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:34.685286+0000","last_change":"2024-04-20T11:02:34.685313+0000","last_active":"2024-04-20T11:02:34.685286+0000","last_peered":"2024-04-20T11:02:34.685286+0000","last_clean":"2024-04-20T11:02:34.685286+0000","last_became_active":"2024-04-20T10:55:42.793351+0000","last_became_peered":"2024-04-20T10:55:42.793351+0000","last_unstale":"2024-04-20T11:02:34.685286+0000","last_undegraded":"2024-04-20T11:02:34.685286+0000","last_fullsized":"2024-04-20T11:02:34.685286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'360","last_scrub_stamp":"2024-04-20T11:02:34.685189+0000","last_deep_scrub":"27'360","last_deep_scrub_stamp":"2024-04-20T11:02:34.685189+0000","last_clean_scrub_stamp":"2024-04-20T11:02:34.685189+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:23.446763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":17282,"num_write":327,"num_write_kb":42097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"27'125","reported_seq":167,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:36.190661+0000","last_change":"2024-04-20T11:02:36.190694+0000","last_active":"2024-04-20T11:02:36.190661+0000","last_peered":"2024-04-20T11:02:36.190661+0000","last_clean":"2024-04-20T11:02:36.190661+0000","last_became_active":"2024-04-20T10:55:41.792253+0000","last_became_peered":"2024-04-20T10:55:41.792253+0000","last_unstale":"2024-04-20T11:02:36.190661+0000","last_undegraded":"2024-04-20T11:02:36.190661+0000","last_fullsized":"2024-04-20T11:02:36.190661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'125","last_scrub_stamp":"2024-04-20T11:02:36.190566+0000","last_deep_scrub":"27'125","last_deep_scrub_stamp":"2024-04-20T11:02:36.190566+0000","last_clean_scrub_stamp":"2024-04-20T11:02:36.190566+0000","objects_scrubbed":3,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:05:44.866070+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":64,"num_write_kb":8241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"27'385","reported_seq":435,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:31.809331+0000","last_change":"2024-04-20T11:02:31.809357+0000","last_active":"2024-04-20T11:02:31.809331+0000","last_peered":"2024-04-20T11:02:31.809331+0000","last_clean":"2024-04-20T11:02:31.809331+0000","last_became_active":"2024-04-20T10:55:42.791659+0000","last_became_peered":"2024-04-20T10:55:42.791659+0000","last_unstale":"2024-04-20T11:02:31.809331+0000","last_undegraded":"2024-04-20T11:02:31.809331+0000","last_fullsized":"2024-04-20T11:02:31.809331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'385","last_scrub_stamp":"2024-04-20T11:02:31.809251+0000","last_deep_scrub":"27'385","last_deep_scrub_stamp":"2024-04-20T11:02:31.809251+0000","last_clean_scrub_stamp":"2024-04-20T11:02:31.809251+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:17:17.574022+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":18713,"num_write":347,"num_write_kb":51198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"27'329","reported_seq":623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:33.344513+0000","last_change":"2024-04-20T11:02:33.344540+0000","last_active":"2024-04-20T11:02:33.344513+0000","last_peered":"2024-04-20T11:02:33.344513+0000","last_clean":"2024-04-20T11:02:33.344513+0000","last_became_active":"2024-04-20T10:55:41.791601+0000","last_became_peered":"2024-04-20T10:55:41.791601+0000","last_unstale":"2024-04-20T11:02:33.344513+0000","last_undegraded":"2024-04-20T11:02:33.344513+0000","last_fullsized":"2024-04-20T11:02:33.344513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'329","last_scrub_stamp":"2024-04-20T11:02:33.344435+0000","last_deep_scrub":"27'329","last_deep_scrub_stamp":"2024-04-20T11:02:33.344435+0000","last_clean_scrub_stamp":"2024-04-20T11:02:33.344435+0000","objects_scrubbed":4,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:13:45.156222+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462167,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":258,"num_read_kb":1597,"num_write":262,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"27'418","reported_seq":474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:38.475304+0000","last_change":"2024-04-20T11:02:38.475347+0000","last_active":"2024-04-20T11:02:38.475304+0000","last_peered":"2024-04-20T11:02:38.475304+0000","last_clean":"2024-04-20T11:02:38.475304+0000","last_became_active":"2024-04-20T10:55:42.797323+0000","last_became_peered":"2024-04-20T10:55:42.797323+0000","last_unstale":"2024-04-20T11:02:38.475304+0000","last_undegraded":"2024-04-20T11:02:38.475304+0000","last_fullsized":"2024-04-20T11:02:38.475304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'418","last_scrub_stamp":"2024-04-20T11:02:38.475207+0000","last_deep_scrub":"27'418","last_deep_scrub_stamp":"2024-04-20T11:02:38.475207+0000","last_clean_scrub_stamp":"2024-04-20T11:02:38.475207+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:44:35.147690+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":47086,"num_write":383,"num_write_kb":51294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"27'803","reported_seq":851,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:36.275270+0000","last_change":"2024-04-20T11:02:36.275309+0000","last_active":"2024-04-20T11:02:36.275270+0000","last_peered":"2024-04-20T11:02:36.275270+0000","last_clean":"2024-04-20T11:02:36.275270+0000","last_became_active":"2024-04-20T10:55:41.798602+0000","last_became_peered":"2024-04-20T10:55:41.798602+0000","last_unstale":"2024-04-20T11:02:36.275270+0000","last_undegraded":"2024-04-20T11:02:36.275270+0000","last_fullsized":"2024-04-20T11:02:36.275270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'803","last_scrub_stamp":"2024-04-20T11:02:36.275173+0000","last_deep_scrub":"27'803","last_deep_scrub_stamp":"2024-04-20T11:02:36.275173+0000","last_clean_scrub_stamp":"2024-04-20T11:02:36.275173+0000","objects_scrubbed":4,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:41.580036+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":14,"num_write":731,"num_write_kb":8599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"27'428","reported_seq":487,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:32.786805+0000","last_change":"2024-04-20T11:02:32.786832+0000","last_active":"2024-04-20T11:02:32.786805+0000","last_peered":"2024-04-20T11:02:32.786805+0000","last_clean":"2024-04-20T11:02:32.786805+0000","last_became_active":"2024-04-20T10:55:42.793651+0000","last_became_peered":"2024-04-20T10:55:42.793651+0000","last_unstale":"2024-04-20T11:02:32.786805+0000","last_undegraded":"2024-04-20T11:02:32.786805+0000","last_fullsized":"2024-04-20T11:02:32.786805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'428","last_scrub_stamp":"2024-04-20T11:02:32.786718+0000","last_deep_scrub":"27'428","last_deep_scrub_stamp":"2024-04-20T11:02:32.786718+0000","last_clean_scrub_stamp":"2024-04-20T11:02:32.786718+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:17:22.186117+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":65250,"num_write":382,"num_write_kb":83821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"27'75","reported_seq":115,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:33.427093+0000","last_change":"2024-04-20T11:02:33.427113+0000","last_active":"2024-04-20T11:02:33.427093+0000","last_peered":"2024-04-20T11:02:33.427093+0000","last_clean":"2024-04-20T11:02:33.427093+0000","last_became_active":"2024-04-20T10:55:41.797098+0000","last_became_peered":"2024-04-20T10:55:41.797098+0000","last_unstale":"2024-04-20T11:02:33.427093+0000","last_undegraded":"2024-04-20T11:02:33.427093+0000","last_fullsized":"2024-04-20T11:02:33.427093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'75","last_scrub_stamp":"2024-04-20T11:02:33.427025+0000","last_deep_scrub":"27'75","last_deep_scrub_stamp":"2024-04-20T11:02:33.427025+0000","last_clean_scrub_stamp":"2024-04-20T11:02:33.427025+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:48.157745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"27'426","reported_seq":468,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:40.262082+0000","last_change":"2024-04-20T11:02:40.262133+0000","last_active":"2024-04-20T11:02:40.262082+0000","last_peered":"2024-04-20T11:02:40.262082+0000","last_clean":"2024-04-20T11:02:40.262082+0000","last_became_active":"2024-04-20T10:55:42.795433+0000","last_became_peered":"2024-04-20T10:55:42.795433+0000","last_unstale":"2024-04-20T11:02:40.262082+0000","last_undegraded":"2024-04-20T11:02:40.262082+0000","last_fullsized":"2024-04-20T11:02:40.262082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'426","last_scrub_stamp":"2024-04-20T11:02:40.261938+0000","last_deep_scrub":"27'426","last_deep_scrub_stamp":"2024-04-20T11:02:40.261938+0000","last_clean_scrub_stamp":"2024-04-20T11:02:40.261938+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:34:58.918948+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":375,"num_write_kb":62820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"27'89","reported_seq":134,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:37.174324+0000","last_change":"2024-04-20T11:02:37.174357+0000","last_active":"2024-04-20T11:02:37.174324+0000","last_peered":"2024-04-20T11:02:37.174324+0000","last_clean":"2024-04-20T11:02:37.174324+0000","last_became_active":"2024-04-20T10:55:41.797029+0000","last_became_peered":"2024-04-20T10:55:41.797029+0000","last_unstale":"2024-04-20T11:02:37.174324+0000","last_undegraded":"2024-04-20T11:02:37.174324+0000","last_fullsized":"2024-04-20T11:02:37.174324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'89","last_scrub_stamp":"2024-04-20T11:02:37.174229+0000","last_deep_scrub":"27'89","last_deep_scrub_stamp":"2024-04-20T11:02:37.174229+0000","last_clean_scrub_stamp":"2024-04-20T11:02:37.174229+0000","objects_scrubbed":4,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:17:28.987596+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194416,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":29,"num_write_kb":4110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"27'391","reported_seq":439,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:37.492684+0000","last_change":"2024-04-20T11:02:37.492709+0000","last_active":"2024-04-20T11:02:37.492684+0000","last_peered":"2024-04-20T11:02:37.492684+0000","last_clean":"2024-04-20T11:02:37.492684+0000","last_became_active":"2024-04-20T10:55:42.796654+0000","last_became_peered":"2024-04-20T10:55:42.796654+0000","last_unstale":"2024-04-20T11:02:37.492684+0000","last_undegraded":"2024-04-20T11:02:37.492684+0000","last_fullsized":"2024-04-20T11:02:37.492684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'391","last_scrub_stamp":"2024-04-20T11:02:37.492579+0000","last_deep_scrub":"27'391","last_deep_scrub_stamp":"2024-04-20T11:02:37.492579+0000","last_clean_scrub_stamp":"2024-04-20T11:02:37.492579+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:29:30.788675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":25761,"num_write":337,"num_write_kb":50347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"27'349","reported_seq":395,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:34.476708+0000","last_change":"2024-04-20T11:02:34.476742+0000","last_active":"2024-04-20T11:02:34.476708+0000","last_peered":"2024-04-20T11:02:34.476708+0000","last_clean":"2024-04-20T11:02:34.476708+0000","last_became_active":"2024-04-20T10:55:41.798269+0000","last_became_peered":"2024-04-20T10:55:41.798269+0000","last_unstale":"2024-04-20T11:02:34.476708+0000","last_undegraded":"2024-04-20T11:02:34.476708+0000","last_fullsized":"2024-04-20T11:02:34.476708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'349","last_scrub_stamp":"2024-04-20T11:02:34.476614+0000","last_deep_scrub":"27'349","last_deep_scrub_stamp":"2024-04-20T11:02:34.476614+0000","last_clean_scrub_stamp":"2024-04-20T11:02:34.476614+0000","objects_scrubbed":4,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:31:20.275116+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":669,"num_write_kb":1875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"27'345","reported_seq":390,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:42.678917+0000","last_change":"2024-04-20T11:02:42.678971+0000","last_active":"2024-04-20T11:02:42.678917+0000","last_peered":"2024-04-20T11:02:42.678917+0000","last_clean":"2024-04-20T11:02:42.678917+0000","last_became_active":"2024-04-20T10:55:42.800038+0000","last_became_peered":"2024-04-20T10:55:42.800038+0000","last_unstale":"2024-04-20T11:02:42.678917+0000","last_undegraded":"2024-04-20T11:02:42.678917+0000","last_fullsized":"2024-04-20T11:02:42.678917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'345","last_scrub_stamp":"2024-04-20T11:02:42.678809+0000","last_deep_scrub":"27'345","last_deep_scrub_stamp":"2024-04-20T11:02:42.678809+0000","last_clean_scrub_stamp":"2024-04-20T11:02:42.678809+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:56.873259+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15872,"num_write":302,"num_write_kb":13258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"27'215","reported_seq":262,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:36.621701+0000","last_change":"2024-04-20T11:02:36.621738+0000","last_active":"2024-04-20T11:02:36.621701+0000","last_peered":"2024-04-20T11:02:36.621701+0000","last_clean":"2024-04-20T11:02:36.621701+0000","last_became_active":"2024-04-20T10:55:41.795586+0000","last_became_peered":"2024-04-20T10:55:41.795586+0000","last_unstale":"2024-04-20T11:02:36.621701+0000","last_undegraded":"2024-04-20T11:02:36.621701+0000","last_fullsized":"2024-04-20T11:02:36.621701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'215","last_scrub_stamp":"2024-04-20T11:02:36.621602+0000","last_deep_scrub":"27'215","last_deep_scrub_stamp":"2024-04-20T11:02:36.621602+0000","last_clean_scrub_stamp":"2024-04-20T11:02:36.621602+0000","objects_scrubbed":5,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:44:07.495357+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3862944,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":149,"num_write_kb":3853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'458","reported_seq":537,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:34.778929+0000","last_change":"2024-04-20T11:02:34.778954+0000","last_active":"2024-04-20T11:02:34.778929+0000","last_peered":"2024-04-20T11:02:34.778929+0000","last_clean":"2024-04-20T11:02:34.778929+0000","last_became_active":"2024-04-20T10:55:42.799291+0000","last_became_peered":"2024-04-20T10:55:42.799291+0000","last_unstale":"2024-04-20T11:02:34.778929+0000","last_undegraded":"2024-04-20T11:02:34.778929+0000","last_fullsized":"2024-04-20T11:02:34.778929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'458","last_scrub_stamp":"2024-04-20T11:02:34.778820+0000","last_deep_scrub":"27'458","last_deep_scrub_stamp":"2024-04-20T11:02:34.778820+0000","last_clean_scrub_stamp":"2024-04-20T11:02:34.778820+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:00:48.996860+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":106048,"num_write":421,"num_write_kb":76099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"27'561","reported_seq":603,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:38.173381+0000","last_change":"2024-04-20T11:02:38.173456+0000","last_active":"2024-04-20T11:02:38.173381+0000","last_peered":"2024-04-20T11:02:38.173381+0000","last_clean":"2024-04-20T11:02:38.173381+0000","last_became_active":"2024-04-20T10:55:41.797658+0000","last_became_peered":"2024-04-20T10:55:41.797658+0000","last_unstale":"2024-04-20T11:02:38.173381+0000","last_undegraded":"2024-04-20T11:02:38.173381+0000","last_fullsized":"2024-04-20T11:02:38.173381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'561","last_scrub_stamp":"2024-04-20T11:02:38.173273+0000","last_deep_scrub":"27'561","last_deep_scrub_stamp":"2024-04-20T11:02:38.173273+0000","last_clean_scrub_stamp":"2024-04-20T11:02:38.173273+0000","objects_scrubbed":2,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:12:22.802140+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":504,"num_write_kb":4393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"27'461","reported_seq":515,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:34.797004+0000","last_change":"2024-04-20T11:02:34.797078+0000","last_active":"2024-04-20T11:02:34.797004+0000","last_peered":"2024-04-20T11:02:34.797004+0000","last_clean":"2024-04-20T11:02:34.797004+0000","last_became_active":"2024-04-20T10:55:42.796938+0000","last_became_peered":"2024-04-20T10:55:42.796938+0000","last_unstale":"2024-04-20T11:02:34.797004+0000","last_undegraded":"2024-04-20T11:02:34.797004+0000","last_fullsized":"2024-04-20T11:02:34.797004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'461","last_scrub_stamp":"2024-04-20T11:02:34.796902+0000","last_deep_scrub":"27'461","last_deep_scrub_stamp":"2024-04-20T11:02:34.796902+0000","last_clean_scrub_stamp":"2024-04-20T11:02:34.796902+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:03:29.948707+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":55402,"num_write":430,"num_write_kb":80382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"27'251","reported_seq":300,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:31.422910+0000","last_change":"2024-04-20T11:02:31.422972+0000","last_active":"2024-04-20T11:02:31.422910+0000","last_peered":"2024-04-20T11:02:31.422910+0000","last_clean":"2024-04-20T11:02:31.422910+0000","last_became_active":"2024-04-20T10:55:41.790223+0000","last_became_peered":"2024-04-20T10:55:41.790223+0000","last_unstale":"2024-04-20T11:02:31.422910+0000","last_undegraded":"2024-04-20T11:02:31.422910+0000","last_fullsized":"2024-04-20T11:02:31.422910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'251","last_scrub_stamp":"2024-04-20T11:02:31.422814+0000","last_deep_scrub":"27'251","last_deep_scrub_stamp":"2024-04-20T11:02:31.422814+0000","last_clean_scrub_stamp":"2024-04-20T11:02:31.422814+0000","objects_scrubbed":5,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:48:14.507767+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":197,"num_write_kb":4290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'410","reported_seq":460,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:36.663849+0000","last_change":"2024-04-20T11:02:36.663880+0000","last_active":"2024-04-20T11:02:36.663849+0000","last_peered":"2024-04-20T11:02:36.663849+0000","last_clean":"2024-04-20T11:02:36.663849+0000","last_became_active":"2024-04-20T10:55:42.794944+0000","last_became_peered":"2024-04-20T10:55:42.794944+0000","last_unstale":"2024-04-20T11:02:36.663849+0000","last_undegraded":"2024-04-20T11:02:36.663849+0000","last_fullsized":"2024-04-20T11:02:36.663849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'410","last_scrub_stamp":"2024-04-20T11:02:36.663747+0000","last_deep_scrub":"27'410","last_deep_scrub_stamp":"2024-04-20T11:02:36.663747+0000","last_clean_scrub_stamp":"2024-04-20T11:02:36.663747+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:11:56.729354+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":45668,"num_write":367,"num_write_kb":46153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'232","reported_seq":274,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:32.462459+0000","last_change":"2024-04-20T11:02:32.462487+0000","last_active":"2024-04-20T11:02:32.462459+0000","last_peered":"2024-04-20T11:02:32.462459+0000","last_clean":"2024-04-20T11:02:32.462459+0000","last_became_active":"2024-04-20T10:55:41.794975+0000","last_became_peered":"2024-04-20T10:55:41.794975+0000","last_unstale":"2024-04-20T11:02:32.462459+0000","last_undegraded":"2024-04-20T11:02:32.462459+0000","last_fullsized":"2024-04-20T11:02:32.462459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'232","last_scrub_stamp":"2024-04-20T11:02:32.462377+0000","last_deep_scrub":"27'232","last_deep_scrub_stamp":"2024-04-20T11:02:32.462377+0000","last_clean_scrub_stamp":"2024-04-20T11:02:32.462377+0000","objects_scrubbed":3,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:20.758722+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":165,"num_write_kb":4183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"27'472","reported_seq":545,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:40.692062+0000","last_change":"2024-04-20T11:02:40.692089+0000","last_active":"2024-04-20T11:02:40.692062+0000","last_peered":"2024-04-20T11:02:40.692062+0000","last_clean":"2024-04-20T11:02:40.692062+0000","last_became_active":"2024-04-20T10:55:42.797359+0000","last_became_peered":"2024-04-20T10:55:42.797359+0000","last_unstale":"2024-04-20T11:02:40.692062+0000","last_undegraded":"2024-04-20T11:02:40.692062+0000","last_fullsized":"2024-04-20T11:02:40.692062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'472","last_scrub_stamp":"2024-04-20T11:02:40.691973+0000","last_deep_scrub":"27'472","last_deep_scrub_stamp":"2024-04-20T11:02:40.691973+0000","last_clean_scrub_stamp":"2024-04-20T11:02:40.691973+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:51:54.140950+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":121848,"num_write":422,"num_write_kb":125716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"27'144","reported_seq":186,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:35.243585+0000","last_change":"2024-04-20T11:02:35.243632+0000","last_active":"2024-04-20T11:02:35.243585+0000","last_peered":"2024-04-20T11:02:35.243585+0000","last_clean":"2024-04-20T11:02:35.243585+0000","last_became_active":"2024-04-20T10:55:41.789743+0000","last_became_peered":"2024-04-20T10:55:41.789743+0000","last_unstale":"2024-04-20T11:02:35.243585+0000","last_undegraded":"2024-04-20T11:02:35.243585+0000","last_fullsized":"2024-04-20T11:02:35.243585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'144","last_scrub_stamp":"2024-04-20T11:02:35.243488+0000","last_deep_scrub":"27'144","last_deep_scrub_stamp":"2024-04-20T11:02:35.243488+0000","last_clean_scrub_stamp":"2024-04-20T11:02:35.243488+0000","objects_scrubbed":4,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:28.886042+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":89,"num_write_kb":8247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"27'440","reported_seq":496,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:39.513454+0000","last_change":"2024-04-20T11:02:39.513498+0000","last_active":"2024-04-20T11:02:39.513454+0000","last_peered":"2024-04-20T11:02:39.513454+0000","last_clean":"2024-04-20T11:02:39.513454+0000","last_became_active":"2024-04-20T10:55:42.797978+0000","last_became_peered":"2024-04-20T10:55:42.797978+0000","last_unstale":"2024-04-20T11:02:39.513454+0000","last_undegraded":"2024-04-20T11:02:39.513454+0000","last_fullsized":"2024-04-20T11:02:39.513454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'440","last_scrub_stamp":"2024-04-20T11:02:39.513354+0000","last_deep_scrub":"27'440","last_deep_scrub_stamp":"2024-04-20T11:02:39.513354+0000","last_clean_scrub_stamp":"2024-04-20T11:02:39.513354+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:20:38.013613+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":46429,"num_write":408,"num_write_kb":79692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"27'393","reported_seq":431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:38.353663+0000","last_change":"2024-04-20T11:02:38.353687+0000","last_active":"2024-04-20T11:02:38.353663+0000","last_peered":"2024-04-20T11:02:38.353663+0000","last_clean":"2024-04-20T11:02:38.353663+0000","last_became_active":"2024-04-20T10:56:22.904460+0000","last_became_peered":"2024-04-20T10:56:22.904460+0000","last_unstale":"2024-04-20T11:02:38.353663+0000","last_undegraded":"2024-04-20T11:02:38.353663+0000","last_fullsized":"2024-04-20T11:02:38.353663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'393","last_scrub_stamp":"2024-04-20T11:02:38.353572+0000","last_deep_scrub":"27'393","last_deep_scrub_stamp":"2024-04-20T11:02:38.353572+0000","last_clean_scrub_stamp":"2024-04-20T11:02:38.353572+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:18:28.415661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":15562,"num_write":337,"num_write_kb":42622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"27'268","reported_seq":313,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:40.164060+0000","last_change":"2024-04-20T11:02:40.164099+0000","last_active":"2024-04-20T11:02:40.164060+0000","last_peered":"2024-04-20T11:02:40.164060+0000","last_clean":"2024-04-20T11:02:40.164060+0000","last_became_active":"2024-04-20T10:55:41.798798+0000","last_became_peered":"2024-04-20T10:55:41.798798+0000","last_unstale":"2024-04-20T11:02:40.164060+0000","last_undegraded":"2024-04-20T11:02:40.164060+0000","last_fullsized":"2024-04-20T11:02:40.164060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'268","last_scrub_stamp":"2024-04-20T11:02:40.163963+0000","last_deep_scrub":"27'268","last_deep_scrub_stamp":"2024-04-20T11:02:40.163963+0000","last_clean_scrub_stamp":"2024-04-20T11:02:40.163963+0000","objects_scrubbed":4,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:33:13.283807+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":212,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'389","reported_seq":439,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:38.251770+0000","last_change":"2024-04-20T11:02:38.251814+0000","last_active":"2024-04-20T11:02:38.251770+0000","last_peered":"2024-04-20T11:02:38.251770+0000","last_clean":"2024-04-20T11:02:38.251770+0000","last_became_active":"2024-04-20T10:55:42.791496+0000","last_became_peered":"2024-04-20T10:55:42.791496+0000","last_unstale":"2024-04-20T11:02:38.251770+0000","last_undegraded":"2024-04-20T11:02:38.251770+0000","last_fullsized":"2024-04-20T11:02:38.251770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'389","last_scrub_stamp":"2024-04-20T11:02:38.251647+0000","last_deep_scrub":"27'389","last_deep_scrub_stamp":"2024-04-20T11:02:38.251647+0000","last_clean_scrub_stamp":"2024-04-20T11:02:38.251647+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:08.592392+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":31020,"num_write":340,"num_write_kb":46796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"27'229","reported_seq":282,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:39.186795+0000","last_change":"2024-04-20T11:02:39.186837+0000","last_active":"2024-04-20T11:02:39.186795+0000","last_peered":"2024-04-20T11:02:39.186795+0000","last_clean":"2024-04-20T11:02:39.186795+0000","last_became_active":"2024-04-20T10:55:41.798089+0000","last_became_peered":"2024-04-20T10:55:41.798089+0000","last_unstale":"2024-04-20T11:02:39.186795+0000","last_undegraded":"2024-04-20T11:02:39.186795+0000","last_fullsized":"2024-04-20T11:02:39.186795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'229","last_scrub_stamp":"2024-04-20T11:02:39.186715+0000","last_deep_scrub":"27'229","last_deep_scrub_stamp":"2024-04-20T11:02:39.186715+0000","last_clean_scrub_stamp":"2024-04-20T11:02:39.186715+0000","objects_scrubbed":10,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:59.066862+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":39,"num_read_kb":43,"num_write":158,"num_write_kb":12367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"27'383","reported_seq":445,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:38.740290+0000","last_change":"2024-04-20T11:02:38.740317+0000","last_active":"2024-04-20T11:02:38.740290+0000","last_peered":"2024-04-20T11:02:38.740290+0000","last_clean":"2024-04-20T11:02:38.740290+0000","last_became_active":"2024-04-20T10:55:42.803144+0000","last_became_peered":"2024-04-20T10:55:42.803144+0000","last_unstale":"2024-04-20T11:02:38.740290+0000","last_undegraded":"2024-04-20T11:02:38.740290+0000","last_fullsized":"2024-04-20T11:02:38.740290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'383","last_scrub_stamp":"2024-04-20T11:02:38.740192+0000","last_deep_scrub":"27'383","last_deep_scrub_stamp":"2024-04-20T11:02:38.740192+0000","last_clean_scrub_stamp":"2024-04-20T11:02:38.740192+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:55:50.461128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":72309,"num_write":351,"num_write_kb":75679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"27'89","reported_seq":129,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:31.722274+0000","last_change":"2024-04-20T11:02:31.722302+0000","last_active":"2024-04-20T11:02:31.722274+0000","last_peered":"2024-04-20T11:02:31.722274+0000","last_clean":"2024-04-20T11:02:31.722274+0000","last_became_active":"2024-04-20T10:55:41.792458+0000","last_became_peered":"2024-04-20T10:55:41.792458+0000","last_unstale":"2024-04-20T11:02:31.722274+0000","last_undegraded":"2024-04-20T11:02:31.722274+0000","last_fullsized":"2024-04-20T11:02:31.722274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'89","last_scrub_stamp":"2024-04-20T11:02:31.722197+0000","last_deep_scrub":"27'89","last_deep_scrub_stamp":"2024-04-20T11:02:31.722197+0000","last_clean_scrub_stamp":"2024-04-20T11:02:31.722197+0000","objects_scrubbed":1,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:35:26.368902+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":4150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'434","reported_seq":488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:42.600213+0000","last_change":"2024-04-20T11:02:42.600240+0000","last_active":"2024-04-20T11:02:42.600213+0000","last_peered":"2024-04-20T11:02:42.600213+0000","last_clean":"2024-04-20T11:02:42.600213+0000","last_became_active":"2024-04-20T10:56:22.912718+0000","last_became_peered":"2024-04-20T10:56:22.912718+0000","last_unstale":"2024-04-20T11:02:42.600213+0000","last_undegraded":"2024-04-20T11:02:42.600213+0000","last_fullsized":"2024-04-20T11:02:42.600213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'434","last_scrub_stamp":"2024-04-20T11:02:42.600115+0000","last_deep_scrub":"27'434","last_deep_scrub_stamp":"2024-04-20T11:02:42.600115+0000","last_clean_scrub_stamp":"2024-04-20T11:02:42.600115+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:05.317977+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":67508,"num_write":400,"num_write_kb":96285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1f","version":"27'77","reported_seq":125,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T11:02:30.816937+0000","last_change":"2024-04-20T11:02:30.816977+0000","last_active":"2024-04-20T11:02:30.816937+0000","last_peered":"2024-04-20T11:02:30.816937+0000","last_clean":"2024-04-20T11:02:30.816937+0000","last_became_active":"2024-04-20T10:55:41.793793+0000","last_became_peered":"2024-04-20T10:55:41.793793+0000","last_unstale":"2024-04-20T11:02:30.816937+0000","last_undegraded":"2024-04-20T11:02:30.816937+0000","last_fullsized":"2024-04-20T11:02:30.816937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'77","last_scrub_stamp":"2024-04-20T11:02:30.816866+0000","last_deep_scrub":"27'77","last_deep_scrub_stamp":"2024-04-20T11:02:30.816866+0000","last_clean_scrub_stamp":"2024-04-20T11:02:30.816866+0000","objects_scrubbed":7,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:32:22.410594+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":25,"num_write_kb":4104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1110,"num_read_kb":3290197,"num_write":24364,"num_write_kb":4196200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27062,"ondisk_log_size":27062,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":152133224,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":619,"num_read_kb":1973,"num_write":6427,"num_write_kb":154893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":304513024,"data_stored":304266448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7930,"ondisk_log_size":7930,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115468,"kb_used_data":57924,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94256372,"statfs":{"total":96636764160,"available":96518524928,"internally_reserved":0,"allocated":59314176,"data_stored":59096824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80316,"kb_used_data":49140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94291524,"statfs":{"total":96636764160,"available":96554520576,"internally_reserved":0,"allocated":50319360,"data_stored":50116697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86548,"kb_used_data":29004,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94285292,"statfs":{"total":96636764160,"available":96548139008,"internally_reserved":0,"allocated":29700096,"data_stored":29479029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102908,"kb_used_data":45364,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94268932,"statfs":{"total":96636764160,"available":96531386368,"internally_reserved":0,"allocated":46452736,"data_stored":46253643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74004,"kb_used_data":34508,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94297836,"statfs":{"total":96636764160,"available":96560984064,"internally_reserved":0,"allocated":35336192,"data_stored":35133465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41048,"kb_used_data":14032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330792,"statfs":{"total":96636764160,"available":96594731008,"internally_reserved":0,"allocated":14368768,"data_stored":14161398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94656,"kb_used_data":41272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94277184,"statfs":{"total":96636764160,"available":96539836416,"internally_reserved":0,"allocated":42262528,"data_stored":42059541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64900,"kb_used_data":29564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94306940,"statfs":{"total":96636764160,"available":96570306560,"internally_reserved":0,"allocated":30273536,"data_stored":30066339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":29388800,"data_stored":29361002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41971712,"data_stored":41944572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14077952,"data_stored":14046429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":35045376,"data_stored":35018496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":46161920,"data_stored":46138674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":29409280,"data_stored":29364060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":50028544,"data_stored":50001728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":58429440,"data_stored":58391487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T11:02:53.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-20T11:02:53.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.759+0000 7fda4f0a5640 1 Processor -- start 2024-04-20T11:02:53.758 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.759+0000 7fda4f0a5640 1 -- start start 2024-04-20T11:02:53.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.759+0000 7fda4f0a5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 0x7fda480c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.759+0000 7fda4f0a5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda480c62e0 0x7fda4815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.759+0000 7fda4f0a5640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fda4815c400 0x7fda480cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.759+0000 7fda4f0a5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7fda48155e00 con 0x7fda480c62e0 2024-04-20T11:02:53.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.759+0000 7fda4f0a5640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fda4815ba30 con 0x7fda4815c400 2024-04-20T11:02:53.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.759+0000 7fda4f0a5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fda4815f210 con 0x7fda48155a20 2024-04-20T11:02:53.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda47fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda480c62e0 0x7fda4815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.762 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda47fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda480c62e0 0x7fda4815bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42354/0 (socket says 172.21.15.27:42354) 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda47fff640 1 -- 172.21.15.27:0/635201305 learned_addr learned my addr 172.21.15.27:0/635201305 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda4ce41640 1 --2- 172.21.15.27:0/635201305 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 0x7fda480c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda4d642640 1 --2- 172.21.15.27:0/635201305 >> v2:172.21.15.137:3300/0 conn(0x7fda4815c400 0x7fda480cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda47fff640 1 -- 172.21.15.27:0/635201305 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 msgr2=0x7fda480c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda47fff640 1 --2- 172.21.15.27:0/635201305 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 0x7fda480c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda47fff640 1 -- 172.21.15.27:0/635201305 >> v2:172.21.15.137:3300/0 conn(0x7fda4815c400 msgr2=0x7fda480cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda47fff640 1 --2- 172.21.15.27:0/635201305 >> v2:172.21.15.137:3300/0 conn(0x7fda4815c400 0x7fda480cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda47fff640 1 -- 172.21.15.27:0/635201305 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4815f4f0 con 0x7fda480c62e0 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda4d642640 1 --2- 172.21.15.27:0/635201305 >> v2:172.21.15.137:3300/0 conn(0x7fda4815c400 0x7fda480cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.760+0000 7fda4ce41640 1 --2- 172.21.15.27:0/635201305 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 0x7fda480c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda47fff640 1 --2- 172.21.15.27:0/635201305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 0x7fda4815bec0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fda480cce30 tx=0x7fda3c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4901ec8796990727 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda477fe640 1 -- 172.21.15.27:0/635201305 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c034070 con 0x7fda480c62e0 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda477fe640 1 -- 172.21.15.27:0/635201305 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3c030860 con 0x7fda480c62e0 2024-04-20T11:02:53.763 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda477fe640 1 -- 172.21.15.27:0/635201305 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c030c30 con 0x7fda480c62e0 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda4f0a5640 1 -- 172.21.15.27:0/635201305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 msgr2=0x7fda4815bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/635201305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 0x7fda4815bec0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fda480cce30 tx=0x7fda3c02b9a0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda4f0a5640 1 -- 172.21.15.27:0/635201305 shutdown_connections 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/635201305 >> v2:172.21.15.137:3300/0 conn(0x7fda4815c400 0x7fda480cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/635201305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 0x7fda4815bec0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/635201305 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 0x7fda480c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda4f0a5640 1 -- 172.21.15.27:0/635201305 >> 172.21.15.27:0/635201305 conn(0x7fda480678a0 msgr2=0x7fda48067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda4f0a5640 1 -- 172.21.15.27:0/635201305 shutdown_connections 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.761+0000 7fda4f0a5640 1 -- 172.21.15.27:0/635201305 wait complete. 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.763+0000 7fda4f0a5640 1 Processor -- start 2024-04-20T11:02:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.763+0000 7fda4f0a5640 1 -- start start 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.763+0000 7fda4f0a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 0x7fda48152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.763+0000 7fda4f0a5640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 0x7fda480cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.763+0000 7fda4f0a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda4815c400 0x7fda48153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4f0a5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda480c67c0 con 0x7fda480c62e0 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4f0a5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fda4815fab0 con 0x7fda4815c400 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4f0a5640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fda4815b650 con 0x7fda48155a20 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4ce41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 0x7fda48152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4ce41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 0x7fda48152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42362/0 (socket says 172.21.15.27:42362) 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4ce41640 1 -- 172.21.15.27:0/295885327 learned_addr learned my addr 172.21.15.27:0/295885327 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda47fff640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 0x7fda480cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4d642640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda4815c400 0x7fda48153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4ce41640 1 -- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 msgr2=0x7fda480cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4ce41640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 0x7fda480cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.767 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4ce41640 1 -- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda4815c400 msgr2=0x7fda48153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4ce41640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda4815c400 0x7fda48153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4ce41640 1 -- 172.21.15.27:0/295885327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda480cf1e0 con 0x7fda480c62e0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4ce41640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 0x7fda48152bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fda4000d100 tx=0x7fda4000d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda457fa640 1 -- 172.21.15.27:0/295885327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4001a070 con 0x7fda480c62e0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda457fa640 1 -- 172.21.15.27:0/295885327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda40003b90 con 0x7fda480c62e0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda480cec20 con 0x7fda480c62e0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda457fa640 1 -- 172.21.15.27:0/295885327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda40004030 con 0x7fda480c62e0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.764+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda481523f0 con 0x7fda480c62e0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.765+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4804fcf0 con 0x7fda480c62e0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.766+0000 7fda457fa640 1 -- 172.21.15.27:0/295885327 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda4001d070 con 0x7fda480c62e0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.766+0000 7fda457fa640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fda3406c8f0 0x7fda3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.766+0000 7fda457fa640 1 -- 172.21.15.27:0/295885327 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7fda4008bfd0 con 0x7fda480c62e0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.766+0000 7fda47fff640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fda3406c8f0 0x7fda3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:53.768 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.767+0000 7fda47fff640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fda3406c8f0 0x7fda3406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fda3c036000 tx=0x7fda3c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:53.770 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.771+0000 7fda457fa640 1 -- 172.21.15.27:0/295885327 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fda40010070 con 0x7fda480c62e0 2024-04-20T11:02:53.959 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.960+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fda48151720 con 0x7fda480c62e0 2024-04-20T11:02:53.962 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.963+0000 7fda457fa640 1 -- 172.21.15.27:0/295885327 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7fda40059b10 con 0x7fda480c62e0 2024-04-20T11:02:53.963 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.964+0000 7fda457fa640 1 -- 172.21.15.27:0/295885327 <== mon.0 v2:172.21.15.27:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fda4005de10 con 0x7fda480c62e0 2024-04-20T11:02:53.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.965+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fda3406c8f0 msgr2=0x7fda3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.965+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fda3406c8f0 0x7fda3406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fda3c036000 tx=0x7fda3c03b040 comp rx=0 tx=0).stop 2024-04-20T11:02:53.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.965+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 msgr2=0x7fda48152bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:53.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.965+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 0x7fda48152bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fda4000d100 tx=0x7fda4000d5c0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.966+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 shutdown_connections 2024-04-20T11:02:53.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.966+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fda3406c8f0 0x7fda3406edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.966+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda4815c400 0x7fda48153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.966 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.966+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fda48155a20 0x7fda480cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.966+0000 7fda4f0a5640 1 --2- 172.21.15.27:0/295885327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda480c62e0 0x7fda48152bf0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.966+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 >> 172.21.15.27:0/295885327 conn(0x7fda480678a0 msgr2=0x7fda480b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.966+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 shutdown_connections 2024-04-20T11:02:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:53.966+0000 7fda4f0a5640 1 -- 172.21.15.27:0/295885327 wait complete. 2024-04-20T11:02:53.976 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T11:02:53.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:54.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.078+0000 7f956458f640 1 Processor -- start 2024-04-20T11:02:54.078 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.078+0000 7f956458f640 1 -- start start 2024-04-20T11:02:54.079 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f956458f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c155910 0x7f955c0c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f956458f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f955c15eaf0 0x7f955c06d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f956458f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f955c15f9b0 con 0x7f955c0c2910 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f956458f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f955c155cf0 con 0x7f955c15eaf0 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f956458f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f955c0c7a70 con 0x7f955c155910 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f9561b2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f955c15eaf0 0x7f955c06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f9561b2a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f955c15eaf0 0x7f955c06d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42368/0 (socket says 172.21.15.27:42368) 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f9561b2a640 1 -- 172.21.15.27:0/459406037 learned_addr learned my addr 172.21.15.27:0/459406037 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f956232b640 1 --2- 172.21.15.27:0/459406037 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c155910 0x7f955c0c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f9561b2a640 1 -- 172.21.15.27:0/459406037 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c155910 msgr2=0x7f955c0c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f9561b2a640 1 --2- 172.21.15.27:0/459406037 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c155910 0x7f955c0c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f9561b2a640 1 -- 172.21.15.27:0/459406037 >> v1:172.21.15.137:6789/0 conn(0x7f955c0c2910 legacy=0x7f955c15b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.079+0000 7f9561b2a640 1 -- 172.21.15.27:0/459406037 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c15f110 con 0x7f955c15eaf0 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.080+0000 7f9561b2a640 1 --2- 172.21.15.27:0/459406037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c15eaf0 0x7f955c06d200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f954c009870 tx=0x7f954c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d958b92d3a753f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.080+0000 7f9561329640 1 -- 172.21.15.27:0/459406037 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c034070 con 0x7f955c15eaf0 2024-04-20T11:02:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.080+0000 7f9561329640 1 -- 172.21.15.27:0/459406037 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f954c030840 con 0x7f955c15eaf0 2024-04-20T11:02:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.080+0000 7f9561329640 1 -- 172.21.15.27:0/459406037 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c030c30 con 0x7f955c15eaf0 2024-04-20T11:02:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.081+0000 7f956458f640 1 -- 172.21.15.27:0/459406037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c15eaf0 msgr2=0x7f955c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.081+0000 7f956458f640 1 --2- 172.21.15.27:0/459406037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c15eaf0 0x7f955c06d200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f954c009870 tx=0x7f954c02b980 comp rx=0 tx=0).stop 2024-04-20T11:02:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.081+0000 7f956458f640 1 -- 172.21.15.27:0/459406037 shutdown_connections 2024-04-20T11:02:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.081+0000 7f956458f640 1 --2- 172.21.15.27:0/459406037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c15eaf0 0x7f955c06d200 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.081+0000 7f956458f640 1 --2- 172.21.15.27:0/459406037 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c155910 0x7f955c0c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.081+0000 7f956458f640 1 -- 172.21.15.27:0/459406037 >> 172.21.15.27:0/459406037 conn(0x7f955c0ccee0 msgr2=0x7f955c0cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.081+0000 7f956458f640 1 -- 172.21.15.27:0/459406037 shutdown_connections 2024-04-20T11:02:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.081+0000 7f956458f640 1 -- 172.21.15.27:0/459406037 wait complete. 2024-04-20T11:02:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.082+0000 7f956458f640 1 Processor -- start 2024-04-20T11:02:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.082+0000 7f956458f640 1 -- start start 2024-04-20T11:02:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956458f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c0c2910 0x7f955c154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956458f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f955c155910 0x7f955c14f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956458f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c15eaf0 0x7f955c14f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956458f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f955c06ca00 con 0x7f955c0c2910 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956458f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f955c15f6d0 con 0x7f955c155910 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956458f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f955c06cce0 con 0x7f955c15eaf0 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956232b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c0c2910 0x7f955c154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956232b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c0c2910 0x7f955c154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42372/0 (socket says 172.21.15.27:42372) 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956232b640 1 -- 172.21.15.27:0/727808824 learned_addr learned my addr 172.21.15.27:0/727808824 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f9561b2a640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f955c155910 0x7f955c14f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f9562b2c640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c15eaf0 0x7f955c14f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956232b640 1 -- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c15eaf0 msgr2=0x7f955c14f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956232b640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c15eaf0 0x7f955c14f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956232b640 1 -- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f955c155910 msgr2=0x7f955c14f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.085 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956232b640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f955c155910 0x7f955c14f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956232b640 1 -- 172.21.15.27:0/727808824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c1f1620 con 0x7f955c0c2910 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f9561b2a640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f955c155910 0x7f955c14f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.083+0000 7f956232b640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c0c2910 0x7f955c154140 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f955800e8a0 tx=0x7f955800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.084+0000 7f9552ffd640 1 -- 172.21.15.27:0/727808824 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558017070 con 0x7f955c0c2910 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.084+0000 7f9552ffd640 1 -- 172.21.15.27:0/727808824 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f955800c940 con 0x7f955c0c2910 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.084+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c1f1900 con 0x7f955c0c2910 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.084+0000 7f9552ffd640 1 -- 172.21.15.27:0/727808824 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955800cd50 con 0x7f955c0c2910 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.084+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c1f1ec0 con 0x7f955c0c2910 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.085+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c04fcf0 con 0x7f955c0c2910 2024-04-20T11:02:54.086 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.085+0000 7f9552ffd640 1 -- 172.21.15.27:0/727808824 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f955801a070 con 0x7f955c0c2910 2024-04-20T11:02:54.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.085+0000 7f9552ffd640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f954006c8f0 0x7f954006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.086+0000 7f9552ffd640 1 -- 172.21.15.27:0/727808824 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f955808c0c0 con 0x7f955c0c2910 2024-04-20T11:02:54.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.086+0000 7f9561b2a640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f954006c8f0 0x7f954006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.087 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.086+0000 7f9561b2a640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f954006c8f0 0x7f954006edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f954c036000 tx=0x7f954c03b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:54.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.090+0000 7f9552ffd640 1 -- 172.21.15.27:0/727808824 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f955805a240 con 0x7f955c0c2910 2024-04-20T11:02:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.312+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f955c14ee90 con 0x7f955c0c2910 2024-04-20T11:02:54.312 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.313+0000 7f9552ffd640 1 -- 172.21.15.27:0/727808824 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+6939 (secure 0 0 0) 0x7f9558059b40 con 0x7f955c0c2910 2024-04-20T11:02:54.313 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:54.313 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6836/1455214274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1455214274},{"type":"v1","addr":"172.21.15.27:6836","nonce":1455214274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:12.937424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5406,5029,4894,4293]},"id":1}]} 2024-04-20T11:02:54.313 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 47 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f954006c8f0 msgr2=0x7f954006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f954006c8f0 0x7f954006edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f954c036000 tx=0x7f954c03b040 comp rx=0 tx=0).stop 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c0c2910 msgr2=0x7f955c154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c0c2910 0x7f955c154140 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f955800e8a0 tx=0x7f955800ed60 comp rx=0 tx=0).stop 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 shutdown_connections 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f954006c8f0 0x7f954006edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f955c15eaf0 0x7f955c14f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f955c155910 0x7f955c14f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 --2- 172.21.15.27:0/727808824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f955c0c2910 0x7f955c154140 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.315 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 >> 172.21.15.27:0/727808824 conn(0x7f955c0ccee0 msgr2=0x7f955c15e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:54.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 shutdown_connections 2024-04-20T11:02:54.316 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.315+0000 7f956458f640 1 -- 172.21.15.27:0/727808824 wait complete. 2024-04-20T11:02:54.326 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T11:02:54.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.426+0000 7f267fcb2640 1 Processor -- start 2024-04-20T11:02:54.426 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.427+0000 7f267fcb2640 1 -- start start 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.427+0000 7f267fcb2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2678155950 0x7f26780cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.427+0000 7f267fcb2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f26780b5c00 0x7f26780d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.427+0000 7f267fcb2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267815b9c0 con 0x7f26780cc330 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.427+0000 7f267fcb2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f267815bca0 con 0x7f26780b5c00 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.427+0000 7f267fcb2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f267815bf80 con 0x7f2678155950 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.427+0000 7f267e24f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f26780b5c00 0x7f26780d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.428+0000 7f267e24f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f26780b5c00 0x7f26780d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42386/0 (socket says 172.21.15.27:42386) 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.428+0000 7f267d24d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f26780cc330 0x7f26780b54f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:48854/0 (socket says 172.21.15.27:48854) 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.428+0000 7f267e24f640 1 -- 172.21.15.27:0/3187936690 learned_addr learned my addr 172.21.15.27:0/3187936690 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.428+0000 7f267da4e640 1 --2- 172.21.15.27:0/3187936690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2678155950 0x7f26780cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.428+0000 7f267e24f640 1 -- 172.21.15.27:0/3187936690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2678155950 msgr2=0x7f26780cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.428+0000 7f267e24f640 1 --2- 172.21.15.27:0/3187936690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2678155950 0x7f26780cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.428+0000 7f267e24f640 1 -- 172.21.15.27:0/3187936690 >> v1:172.21.15.27:6789/0 conn(0x7f26780cc330 legacy=0x7f26780b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.428+0000 7f267ca4c640 1 -- 172.21.15.27:0/3187936690 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1255945642 0 0) 0x7f267815b9c0 con 0x7f26780cc330 2024-04-20T11:02:54.429 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.428+0000 7f267e24f640 1 -- 172.21.15.27:0/3187936690 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267815c260 con 0x7f26780b5c00 2024-04-20T11:02:54.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267e24f640 1 --2- 172.21.15.27:0/3187936690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26780b5c00 0x7f26780d0c90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2668009e50 tx=0x7f266802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ff2007b724d4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:54.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267ca4c640 1 -- 172.21.15.27:0/3187936690 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266802cb90 con 0x7f26780b5c00 2024-04-20T11:02:54.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267ca4c640 1 -- 172.21.15.27:0/3187936690 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f266802cd30 con 0x7f26780b5c00 2024-04-20T11:02:54.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267ca4c640 1 -- 172.21.15.27:0/3187936690 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668034630 con 0x7f26780b5c00 2024-04-20T11:02:54.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267fcb2640 1 -- 172.21.15.27:0/3187936690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26780b5c00 msgr2=0x7f26780d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267fcb2640 1 --2- 172.21.15.27:0/3187936690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26780b5c00 0x7f26780d0c90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2668009e50 tx=0x7f266802bff0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267fcb2640 1 -- 172.21.15.27:0/3187936690 shutdown_connections 2024-04-20T11:02:54.430 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267fcb2640 1 --2- 172.21.15.27:0/3187936690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26780b5c00 0x7f26780d0c90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267fcb2640 1 --2- 172.21.15.27:0/3187936690 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2678155950 0x7f26780cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267fcb2640 1 -- 172.21.15.27:0/3187936690 >> 172.21.15.27:0/3187936690 conn(0x7f26780676c0 msgr2=0x7f2678067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:54.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267fcb2640 1 -- 172.21.15.27:0/3187936690 shutdown_connections 2024-04-20T11:02:54.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.429+0000 7f267fcb2640 1 -- 172.21.15.27:0/3187936690 wait complete. 2024-04-20T11:02:54.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267fcb2640 1 Processor -- start 2024-04-20T11:02:54.431 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267fcb2640 1 -- start start 2024-04-20T11:02:54.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267fcb2640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f26780b5c00 0x7f26780c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267fcb2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26780cc330 0x7f2678159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267fcb2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2678155950 0x7f267814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267fcb2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f267815fd10 con 0x7f2678155950 2024-04-20T11:02:54.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267fcb2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f267815fb90 con 0x7f26780cc330 2024-04-20T11:02:54.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267fcb2640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f2678155d30 con 0x7f26780b5c00 2024-04-20T11:02:54.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267e24f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2678155950 0x7f267814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267e24f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2678155950 0x7f267814dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42392/0 (socket says 172.21.15.27:42392) 2024-04-20T11:02:54.433 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.431+0000 7f267e24f640 1 -- 172.21.15.27:0/1540935261 learned_addr learned my addr 172.21.15.27:0/1540935261 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267da4e640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f26780b5c00 0x7f26780c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267d24d640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26780cc330 0x7f2678159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267e24f640 1 -- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f26780b5c00 msgr2=0x7f26780c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267e24f640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f26780b5c00 0x7f26780c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267e24f640 1 -- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26780cc330 msgr2=0x7f2678159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267e24f640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26780cc330 0x7f2678159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267e24f640 1 -- 172.21.15.27:0/1540935261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26781f1f00 con 0x7f2678155950 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267e24f640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2678155950 0x7f267814dce0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2668036040 tx=0x7f2668004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f26767fc640 1 -- 172.21.15.27:0/1540935261 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668043020 con 0x7f2678155950 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f26767fc640 1 -- 172.21.15.27:0/1540935261 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f266803e450 con 0x7f2678155950 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f26767fc640 1 -- 172.21.15.27:0/1540935261 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668009510 con 0x7f2678155950 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267815b140 con 0x7f2678155950 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.432+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2678159670 con 0x7f2678155950 2024-04-20T11:02:54.434 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.433+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267804fcf0 con 0x7f2678155950 2024-04-20T11:02:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.434+0000 7f26767fc640 1 -- 172.21.15.27:0/1540935261 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2668046020 con 0x7f2678155950 2024-04-20T11:02:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.434+0000 7f26767fc640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f266406c920 0x7f266406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.434+0000 7f26767fc640 1 -- 172.21.15.27:0/1540935261 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f26680ae500 con 0x7f2678155950 2024-04-20T11:02:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.434+0000 7f267da4e640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f266406c920 0x7f266406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.435 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.435+0000 7f267da4e640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f266406c920 0x7f266406ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2670006fd0 tx=0x7f2670008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:54.438 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.438+0000 7f26767fc640 1 -- 172.21.15.27:0/1540935261 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f266804d030 con 0x7f2678155950 2024-04-20T11:02:54.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.663+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26780cd0a0 con 0x7f2678155950 2024-04-20T11:02:54.663 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.664+0000 7f26767fc640 1 -- 172.21.15.27:0/1540935261 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+6939 (secure 0 0 0) 0x7f266807c710 con 0x7f2678155950 2024-04-20T11:02:54.664 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:54.664 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6836/1455214274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1455214274},{"type":"v1","addr":"172.21.15.27:6836","nonce":1455214274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:55:42.150319+0000","modified":"2024-04-20T11:02:12.937424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":5406,"mds_2":5143,"mds_3":5029,"mds_4":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.137:6839/2929802235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6837","nonce":2929802235},{"type":"v1","addr":"172.21.15.137:6839","nonce":2929802235}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4894":{"gid":4894,"name":"b","rank":4,"incarnation":24,"state":"up:active","state_seq":17,"addr":"172.21.15.137:6838/2896327439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2896327439},{"type":"v1","addr":"172.21.15.137:6838","nonce":2896327439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5029":{"gid":5029,"name":"f","rank":3,"incarnation":30,"state":"up:active","state_seq":22,"addr":"172.21.15.137:6835/4235742413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4235742413},{"type":"v1","addr":"172.21.15.137:6835","nonce":4235742413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5143":{"gid":5143,"name":"c","rank":2,"incarnation":36,"state":"up:active","state_seq":20,"addr":"172.21.15.27:6835/2020174232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2020174232},{"type":"v1","addr":"172.21.15.27:6835","nonce":2020174232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5406":{"gid":5406,"name":"e","rank":1,"incarnation":42,"state":"up:active","state_seq":19,"addr":"172.21.15.27:6837/3526024786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3526024786},{"type":"v1","addr":"172.21.15.27:6837","nonce":3526024786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[5143,5406,5029,4894,4293]},"id":1}]} 2024-04-20T11:02:54.664 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 47 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f266406c920 msgr2=0x7f266406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f266406c920 0x7f266406ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2670006fd0 tx=0x7f2670008040 comp rx=0 tx=0).stop 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2678155950 msgr2=0x7f267814dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2678155950 0x7f267814dce0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2668036040 tx=0x7f2668004a40 comp rx=0 tx=0).stop 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 shutdown_connections 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f266406c920 0x7f266406ede0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2678155950 0x7f267814dce0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26780cc330 0x7f2678159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 --2- 172.21.15.27:0/1540935261 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f26780b5c00 0x7f26780c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.666 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 >> 172.21.15.27:0/1540935261 conn(0x7f26780676c0 msgr2=0x7f26780cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:54.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 shutdown_connections 2024-04-20T11:02:54.667 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.666+0000 7f267fcb2640 1 -- 172.21.15.27:0/1540935261 wait complete. 2024-04-20T11:02:54.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T11:02:54.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.778+0000 7f43b925c640 1 Processor -- start 2024-04-20T11:02:54.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.778+0000 7f43b925c640 1 -- start start 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.778+0000 7f43b925c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b4152640 0x7f43b40719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.778+0000 7f43b925c640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f43b40cd6f0 0x7f43b4158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.778+0000 7f43b925c640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43b415ba80 con 0x7f43b4152260 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.778+0000 7f43b925c640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f43b415bd60 con 0x7f43b40cd6f0 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.778+0000 7f43b925c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f43b415c040 con 0x7f43b4152640 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.779+0000 7f43b259c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b4152640 0x7f43b40719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.779+0000 7f43b359e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f43b40cd6f0 0x7f43b4158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.779+0000 7f43b259c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b4152640 0x7f43b40719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.27:34064/0 (socket says 172.21.15.27:34064) 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.779+0000 7f43b259c640 1 -- 172.21.15.27:0/1847951801 learned_addr learned my addr 172.21.15.27:0/1847951801 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.779+0000 7f43b259c640 1 -- 172.21.15.27:0/1847951801 >> v2:172.21.15.137:3300/0 conn(0x7f43b40cd6f0 msgr2=0x7f43b4158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.779+0000 7f43b259c640 1 --2- 172.21.15.27:0/1847951801 >> v2:172.21.15.137:3300/0 conn(0x7f43b40cd6f0 0x7f43b4158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.779+0000 7f43b259c640 1 -- 172.21.15.27:0/1847951801 >> v1:172.21.15.137:6789/0 conn(0x7f43b4152260 legacy=0x7f43b40c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.779+0000 7f43b259c640 1 -- 172.21.15.27:0/1847951801 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b415c320 con 0x7f43b4152640 2024-04-20T11:02:54.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.779+0000 7f43b359e640 1 --2- 172.21.15.27:0/1847951801 >> v2:172.21.15.137:3300/0 conn(0x7f43b40cd6f0 0x7f43b4158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:02:54.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.780+0000 7f43b259c640 1 --2- 172.21.15.27:0/1847951801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b4152640 0x7f43b40719c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f439c009870 tx=0x7f439c02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cb2f2837dd51c41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:54.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.780+0000 7f43b1d9b640 1 -- 172.21.15.27:0/1847951801 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c02f840 con 0x7f43b4152640 2024-04-20T11:02:54.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.780+0000 7f43b1d9b640 1 -- 172.21.15.27:0/1847951801 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f439c02f9e0 con 0x7f43b4152640 2024-04-20T11:02:54.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.780+0000 7f43b1d9b640 1 -- 172.21.15.27:0/1847951801 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c02fdd0 con 0x7f43b4152640 2024-04-20T11:02:54.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.780+0000 7f43b925c640 1 -- 172.21.15.27:0/1847951801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b4152640 msgr2=0x7f43b40719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.780+0000 7f43b925c640 1 --2- 172.21.15.27:0/1847951801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b4152640 0x7f43b40719c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f439c009870 tx=0x7f439c02ba20 comp rx=0 tx=0).stop 2024-04-20T11:02:54.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.781+0000 7f43b925c640 1 -- 172.21.15.27:0/1847951801 shutdown_connections 2024-04-20T11:02:54.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.781+0000 7f43b925c640 1 --2- 172.21.15.27:0/1847951801 >> v2:172.21.15.137:3300/0 conn(0x7f43b40cd6f0 0x7f43b4158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.781+0000 7f43b925c640 1 --2- 172.21.15.27:0/1847951801 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b4152640 0x7f43b40719c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.781+0000 7f43b925c640 1 -- 172.21.15.27:0/1847951801 >> 172.21.15.27:0/1847951801 conn(0x7f43b40ccbd0 msgr2=0x7f43b40ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:54.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.781+0000 7f43b925c640 1 -- 172.21.15.27:0/1847951801 shutdown_connections 2024-04-20T11:02:54.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.781+0000 7f43b925c640 1 -- 172.21.15.27:0/1847951801 wait complete. 2024-04-20T11:02:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.782+0000 7f43b925c640 1 Processor -- start 2024-04-20T11:02:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.782+0000 7f43b925c640 1 -- start start 2024-04-20T11:02:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b925c640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b40cd6f0 0x7f43b41f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b925c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43b4152260 0x7f43b413e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b925c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b4152640 0x7f43b413f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b925c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43b415c600 con 0x7f43b4152260 2024-04-20T11:02:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b925c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43b4074260 con 0x7f43b4152640 2024-04-20T11:02:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b925c640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f43b415b720 con 0x7f43b40cd6f0 2024-04-20T11:02:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43b4152260 0x7f43b413e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43b4152260 0x7f43b413e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57114/0 (socket says 172.21.15.27:57114) 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b259c640 1 -- 172.21.15.27:0/991847529 learned_addr learned my addr 172.21.15.27:0/991847529 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b359e640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b4152640 0x7f43b413f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b2d9d640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b40cd6f0 0x7f43b41f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b259c640 1 -- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b40cd6f0 msgr2=0x7f43b41f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b259c640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b40cd6f0 0x7f43b41f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b259c640 1 -- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b4152640 msgr2=0x7f43b413f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b259c640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b4152640 0x7f43b413f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b259c640 1 -- 172.21.15.27:0/991847529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b414ce60 con 0x7f43b4152260 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b359e640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b4152640 0x7f43b413f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.783+0000 7f43b259c640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43b4152260 0x7f43b413e1e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f439c000c00 tx=0x7f439c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.784+0000 7f43ab7fe640 1 -- 172.21.15.27:0/991847529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c037bc0 con 0x7f43b4152260 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.784+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b41f2a90 con 0x7f43b4152260 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.784+0000 7f43ab7fe640 1 -- 172.21.15.27:0/991847529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f439c046070 con 0x7f43b4152260 2024-04-20T11:02:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.784+0000 7f43ab7fe640 1 -- 172.21.15.27:0/991847529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c041400 con 0x7f43b4152260 2024-04-20T11:02:54.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.784+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b41f3050 con 0x7f43b4152260 2024-04-20T11:02:54.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.785+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b404fcf0 con 0x7f43b4152260 2024-04-20T11:02:54.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.785+0000 7f43ab7fe640 1 -- 172.21.15.27:0/991847529 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f439c040070 con 0x7f43b4152260 2024-04-20T11:02:54.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.785+0000 7f43ab7fe640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f439406c8f0 0x7f439406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:54.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.786+0000 7f43ab7fe640 1 -- 172.21.15.27:0/991847529 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f439c0ae630 con 0x7f43b4152260 2024-04-20T11:02:54.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.786+0000 7f43b2d9d640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f439406c8f0 0x7f439406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:54.787 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.786+0000 7f43b2d9d640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f439406c8f0 0x7f439406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001640 tx=0x7f43a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:54.789 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.790+0000 7f43ab7fe640 1 -- 172.21.15.27:0/991847529 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f43b414ce60 con 0x7f43b4152260 2024-04-20T11:02:54.977 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.978+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43b414ede0 con 0x7f43b4152260 2024-04-20T11:02:54.978 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.979+0000 7f43ab7fe640 1 -- 172.21.15.27:0/991847529 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+17470 (secure 0 0 0) 0x7f439c033030 con 0x7f43b4152260 2024-04-20T11:02:54.979 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-04-20T11:02:54.979 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":27,"fsid":"056b9857-269e-4bc1-b62f-8882e32cd3e3","created":"2024-04-20T10:55:17.518188+0000","modified":"2024-04-20T11:01:52.359149+0000","last_up_change":"2024-04-20T10:55:28.771109+0000","last_in_change":"2024-04-20T10:55:19.707863+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:55:29.717620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:55:39.832461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:55:41.103975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4a4dfaa3-f7b0-4a8e-bed9-018df61c508b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6809","nonce":2513808316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6811","nonce":2513808316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6815","nonce":2513808316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2513808316},{"type":"v1","addr":"172.21.15.27:6813","nonce":2513808316}]},"public_addr":"172.21.15.27:6809/2513808316","cluster_addr":"172.21.15.27:6811/2513808316","heartbeat_back_addr":"172.21.15.27:6815/2513808316","heartbeat_front_addr":"172.21.15.27:6813/2513808316","state":["exists","up"]},{"osd":1,"uuid":"f540862a-9f5e-411a-9f8a-5096226ef3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6801","nonce":600824511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6803","nonce":600824511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6807","nonce":600824511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":600824511},{"type":"v1","addr":"172.21.15.27:6805","nonce":600824511}]},"public_addr":"172.21.15.27:6801/600824511","cluster_addr":"172.21.15.27:6803/600824511","heartbeat_back_addr":"172.21.15.27:6807/600824511","heartbeat_front_addr":"172.21.15.27:6805/600824511","state":["exists","up"]},{"osd":2,"uuid":"0059bcea-d2c5-47f3-9100-8d862a129dd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6817","nonce":3544986990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6819","nonce":3544986990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6827","nonce":3544986990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":3544986990},{"type":"v1","addr":"172.21.15.27:6823","nonce":3544986990}]},"public_addr":"172.21.15.27:6817/3544986990","cluster_addr":"172.21.15.27:6819/3544986990","heartbeat_back_addr":"172.21.15.27:6827/3544986990","heartbeat_front_addr":"172.21.15.27:6823/3544986990","state":["exists","up"]},{"osd":3,"uuid":"d33806ff-f093-4414-8a74-f48775c0e13d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6822","nonce":574866103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6826","nonce":574866103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6831","nonce":574866103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":574866103},{"type":"v1","addr":"172.21.15.27:6829","nonce":574866103}]},"public_addr":"172.21.15.27:6822/574866103","cluster_addr":"172.21.15.27:6826/574866103","heartbeat_back_addr":"172.21.15.27:6831/574866103","heartbeat_front_addr":"172.21.15.27:6829/574866103","state":["exists","up"]},{"osd":4,"uuid":"ace9e9ac-5375-4880-b660-097a18353f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6801","nonce":3079865776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6803","nonce":3079865776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6807","nonce":3079865776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3079865776},{"type":"v1","addr":"172.21.15.137:6805","nonce":3079865776}]},"public_addr":"172.21.15.137:6801/3079865776","cluster_addr":"172.21.15.137:6803/3079865776","heartbeat_back_addr":"172.21.15.137:6807/3079865776","heartbeat_front_addr":"172.21.15.137:6805/3079865776","state":["exists","up"]},{"osd":5,"uuid":"aad00479-b2ec-4539-8260-774ffc288dc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6825","nonce":673751184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6827","nonce":673751184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6831","nonce":673751184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":673751184},{"type":"v1","addr":"172.21.15.137:6829","nonce":673751184}]},"public_addr":"172.21.15.137:6825/673751184","cluster_addr":"172.21.15.137:6827/673751184","heartbeat_back_addr":"172.21.15.137:6831/673751184","heartbeat_front_addr":"172.21.15.137:6829/673751184","state":["exists","up"]},{"osd":6,"uuid":"a220d407-a929-41f9-9f73-4044163f0f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6809","nonce":1373372098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6811","nonce":1373372098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6815","nonce":1373372098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1373372098},{"type":"v1","addr":"172.21.15.137:6813","nonce":1373372098}]},"public_addr":"172.21.15.137:6809/1373372098","cluster_addr":"172.21.15.137:6811/1373372098","heartbeat_back_addr":"172.21.15.137:6815/1373372098","heartbeat_front_addr":"172.21.15.137:6813/1373372098","state":["exists","up"]},{"osd":7,"uuid":"14cc12e6-57c0-42bf-8b76-aca9b5edb063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6817","nonce":460941503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6819","nonce":460941503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6823","nonce":460941503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":460941503},{"type":"v1","addr":"172.21.15.137:6821","nonce":460941503}]},"public_addr":"172.21.15.137:6817/460941503","cluster_addr":"172.21.15.137:6819/460941503","heartbeat_back_addr":"172.21.15.137:6823/460941503","heartbeat_front_addr":"172.21.15.137:6821/460941503","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.862957+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.873796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.863104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.833680+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.814414+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.827041+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.810937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:55:24.895941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1e","mappings":[{"from":3,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6836/4195543922":"2024-04-21T11:01:52.359070+0000","172.21.15.27:6833/4195543922":"2024-04-21T11:01:52.359070+0000","172.21.15.27:6837/1680990583":"2024-04-21T11:00:42.348974+0000","172.21.15.27:6834/1680990583":"2024-04-21T11:00:42.348974+0000","172.21.15.27:6832/859496751":"2024-04-21T10:59:37.339287+0000","172.21.15.137:6835/1164978945":"2024-04-21T10:58:22.329607+0000","172.21.15.137:6834/1164978945":"2024-04-21T10:58:22.329607+0000","172.21.15.27:6835/859496751":"2024-04-21T10:59:37.339287+0000","172.21.15.137:6838/1990627502":"2024-04-21T10:57:27.321673+0000","172.21.15.137:6836/1990627502":"2024-04-21T10:57:27.321673+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T11:02:54.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f439406c8f0 msgr2=0x7f439406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f439406c8f0 0x7f439406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001640 tx=0x7f43a4009040 comp rx=0 tx=0).stop 2024-04-20T11:02:54.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43b4152260 msgr2=0x7f43b413e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:54.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43b4152260 0x7f43b413e1e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f439c000c00 tx=0x7f439c004730 comp rx=0 tx=0).stop 2024-04-20T11:02:54.981 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 shutdown_connections 2024-04-20T11:02:54.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f439406c8f0 0x7f439406edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b4152640 0x7f43b413f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43b4152260 0x7f43b413e1e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 --2- 172.21.15.27:0/991847529 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43b40cd6f0 0x7f43b41f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:54.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 >> 172.21.15.27:0/991847529 conn(0x7f43b40ccbd0 msgr2=0x7f43b40ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:54.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 shutdown_connections 2024-04-20T11:02:54.982 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:54.981+0000 7f43b925c640 1 -- 172.21.15.27:0/991847529 wait complete. 2024-04-20T11:02:54.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T11:02:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.095+0000 7f552c7a4640 1 Processor -- start 2024-04-20T11:02:55.095 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.095+0000 7f552c7a4640 1 -- start start 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f552c7a4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f552415ee50 0x7f552415c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f552c7a4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f552415f230 0x7f552406cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f552c7a4640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f55240ccbd0 0x7f5524155410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f552c7a4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f552415f9a0 con 0x7f552415f230 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f552c7a4640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7f55240cb870 con 0x7f55240ccbd0 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f552c7a4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f552406d490 con 0x7f552415ee50 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f5529d3f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f552415f230 0x7f552406cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f5529d3f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f552415f230 0x7f552406cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57128/0 (socket says 172.21.15.27:57128) 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f5529d3f640 1 -- 172.21.15.27:0/3133486166 learned_addr learned my addr 172.21.15.27:0/3133486166 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f552a540640 1 --2- 172.21.15.27:0/3133486166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f552415ee50 0x7f552415c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f552ad41640 1 --2- 172.21.15.27:0/3133486166 >> v2:172.21.15.137:3300/0 conn(0x7f55240ccbd0 0x7f5524155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.097 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f5529d3f640 1 -- 172.21.15.27:0/3133486166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f552415ee50 msgr2=0x7f552415c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.096+0000 7f5529d3f640 1 --2- 172.21.15.27:0/3133486166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f552415ee50 0x7f552415c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.097+0000 7f5529d3f640 1 -- 172.21.15.27:0/3133486166 >> v2:172.21.15.137:3300/0 conn(0x7f55240ccbd0 msgr2=0x7f5524155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.097+0000 7f5529d3f640 1 --2- 172.21.15.27:0/3133486166 >> v2:172.21.15.137:3300/0 conn(0x7f55240ccbd0 0x7f5524155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.097+0000 7f5529d3f640 1 -- 172.21.15.27:0/3133486166 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55240cbb30 con 0x7f552415f230 2024-04-20T11:02:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.097+0000 7f5529d3f640 1 --2- 172.21.15.27:0/3133486166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415f230 0x7f552406cf50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f552408ee60 tx=0x7f551802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f66fd97c79f3d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552953e640 1 -- 172.21.15.27:0/3133486166 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518034070 con 0x7f552415f230 2024-04-20T11:02:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552953e640 1 -- 172.21.15.27:0/3133486166 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5518030860 con 0x7f552415f230 2024-04-20T11:02:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552953e640 1 -- 172.21.15.27:0/3133486166 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518030c50 con 0x7f552415f230 2024-04-20T11:02:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552c7a4640 1 -- 172.21.15.27:0/3133486166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415f230 msgr2=0x7f552406cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552c7a4640 1 --2- 172.21.15.27:0/3133486166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415f230 0x7f552406cf50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f552408ee60 tx=0x7f551802b9a0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552c7a4640 1 -- 172.21.15.27:0/3133486166 shutdown_connections 2024-04-20T11:02:55.099 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552c7a4640 1 --2- 172.21.15.27:0/3133486166 >> v2:172.21.15.137:3300/0 conn(0x7f55240ccbd0 0x7f5524155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552c7a4640 1 --2- 172.21.15.27:0/3133486166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415f230 0x7f552406cf50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552c7a4640 1 --2- 172.21.15.27:0/3133486166 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f552415ee50 0x7f552415c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552c7a4640 1 -- 172.21.15.27:0/3133486166 >> 172.21.15.27:0/3133486166 conn(0x7f552406e010 msgr2=0x7f55240d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552c7a4640 1 -- 172.21.15.27:0/3133486166 shutdown_connections 2024-04-20T11:02:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.098+0000 7f552c7a4640 1 -- 172.21.15.27:0/3133486166 wait complete. 2024-04-20T11:02:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.099+0000 7f552c7a4640 1 Processor -- start 2024-04-20T11:02:55.100 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f552c7a4640 1 -- start start 2024-04-20T11:02:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f552c7a4640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55240ccbd0 0x7f552414da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f552c7a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415ee50 0x7f552414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f552c7a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f552415f230 0x7f55241543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f552c7a4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55240cc3d0 con 0x7f552415ee50 2024-04-20T11:02:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f552c7a4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55240cc0f0 con 0x7f552415f230 2024-04-20T11:02:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f552c7a4640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f55240cc6b0 con 0x7f55240ccbd0 2024-04-20T11:02:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f5529d3f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415ee50 0x7f552414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.102 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f5529d3f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415ee50 0x7f552414ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57138/0 (socket says 172.21.15.27:57138) 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f5529d3f640 1 -- 172.21.15.27:0/1546251674 learned_addr learned my addr 172.21.15.27:0/1546251674 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f552a540640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55240ccbd0 0x7f552414da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.100+0000 7f552ad41640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f552415f230 0x7f55241543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f5529d3f640 1 -- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55240ccbd0 msgr2=0x7f552414da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f5529d3f640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55240ccbd0 0x7f552414da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f5529d3f640 1 -- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f552415f230 msgr2=0x7f55241543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f5529d3f640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f552415f230 0x7f55241543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f5529d3f640 1 -- 172.21.15.27:0/1546251674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55241f5560 con 0x7f552415ee50 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f5529d3f640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415ee50 0x7f552414ed40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5518000c00 tx=0x7f5518004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f5522ffd640 1 -- 172.21.15.27:0/1546251674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518048070 con 0x7f552415ee50 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f5522ffd640 1 -- 172.21.15.27:0/1546251674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5518030660 con 0x7f552415ee50 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f5522ffd640 1 -- 172.21.15.27:0/1546251674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518034040 con 0x7f552415ee50 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55240cf040 con 0x7f552415ee50 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.101+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55240cf600 con 0x7f552415ee50 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.102+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55240688e0 con 0x7f552415ee50 2024-04-20T11:02:55.103 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.103+0000 7f5522ffd640 1 -- 172.21.15.27:0/1546251674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f551804b070 con 0x7f552415ee50 2024-04-20T11:02:55.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.103+0000 7f5522ffd640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f551006c8f0 0x7f551006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.103+0000 7f5522ffd640 1 -- 172.21.15.27:0/1546251674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6255+0+0 (secure 0 0 0) 0x7f55180adec0 con 0x7f552415ee50 2024-04-20T11:02:55.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.103+0000 7f552a540640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f551006c8f0 0x7f551006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.104 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.104+0000 7f552a540640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f551006c8f0 0x7f551006edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f551c003b80 tx=0x7f551c0098d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:55.107 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.108+0000 7f5522ffd640 1 -- 172.21.15.27:0/1546251674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5518030830 con 0x7f552415ee50 2024-04-20T11:02:55.329 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.329+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5524159ba0 con 0x7f552415ee50 2024-04-20T11:02:55.331 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.332+0000 7f5522ffd640 1 -- 172.21.15.27:0/1546251674 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f5524159ba0 con 0x7f552415ee50 2024-04-20T11:02:55.331 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T11:02:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.335+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f551006c8f0 msgr2=0x7f551006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.335+0000 7f552c7a4640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f551006c8f0 0x7f551006edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f551c003b80 tx=0x7f551c0098d0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.335+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415ee50 msgr2=0x7f552414ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.335+0000 7f552c7a4640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415ee50 0x7f552414ed40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5518000c00 tx=0x7f5518004730 comp rx=0 tx=0).stop 2024-04-20T11:02:55.335 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.336+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 shutdown_connections 2024-04-20T11:02:55.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.336+0000 7f552c7a4640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f551006c8f0 0x7f551006edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.336+0000 7f552c7a4640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f552415f230 0x7f55241543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.336+0000 7f552c7a4640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f552415ee50 0x7f552414ed40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.336+0000 7f552c7a4640 1 --2- 172.21.15.27:0/1546251674 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f55240ccbd0 0x7f552414da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.336+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 >> 172.21.15.27:0/1546251674 conn(0x7f552406e010 msgr2=0x7f55240cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:55.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.336+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 shutdown_connections 2024-04-20T11:02:55.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.336+0000 7f552c7a4640 1 -- 172.21.15.27:0/1546251674 wait complete. 2024-04-20T11:02:55.348 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T11:02:55.365 INFO:tasks.ceph.mds.e.smithi027.stdout:starting mds.e at 2024-04-20T11:02:55.366 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2024-04-20T11:02:55.379 INFO:tasks.ceph.mds.f.smithi137.stdout:starting mds.f at 2024-04-20T11:02:55.380 INFO:tasks.ceph.mds.d.smithi137.stdout:starting mds.d at 2024-04-20T11:02:55.381 INFO:tasks.ceph.mds.b.smithi137.stdout:starting mds.b at 2024-04-20T11:02:55.454 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.455+0000 7f637338f640 1 Processor -- start 2024-04-20T11:02:55.455 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.455+0000 7f637338f640 1 -- start start 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637338f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f636c06fcd0 0x7f636c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637338f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636c0c9970 con 0x7f636c151f40 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637338f640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636c15b750 con 0x7f636c152320 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637338f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f636c070210 con 0x7f636c06fcd0 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637192c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f636c06fcd0 0x7f636c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637112b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f636c151f40 0x7f636c06f5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36936/0 (socket says 172.21.15.27:36936) 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637112b640 1 -- 172.21.15.27:0/13624468 learned_addr learned my addr 172.21.15.27:0/13624468 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637192c640 1 -- 172.21.15.27:0/13624468 >> v1:172.21.15.137:6789/0 conn(0x7f636c152320 legacy=0x7f636c070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637192c640 1 -- 172.21.15.27:0/13624468 >> v1:172.21.15.27:6789/0 conn(0x7f636c151f40 legacy=0x7f636c06f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.456 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.456+0000 7f637192c640 1 -- 172.21.15.27:0/13624468 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c15ba30 con 0x7f636c06fcd0 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.457+0000 7f637192c640 1 --2- 172.21.15.27:0/13624468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c06fcd0 0x7f636c0711d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6364009d90 tx=0x7f636402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c94f54f7c9c9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.457+0000 7f6363fff640 1 -- 172.21.15.27:0/13624468 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364039070 con 0x7f636c06fcd0 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.457+0000 7f6363fff640 1 -- 172.21.15.27:0/13624468 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f636402cd80 con 0x7f636c06fcd0 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.457+0000 7f6363fff640 1 -- 172.21.15.27:0/13624468 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364034680 con 0x7f636c06fcd0 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.457+0000 7f637338f640 1 -- 172.21.15.27:0/13624468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c06fcd0 msgr2=0x7f636c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.457+0000 7f637338f640 1 --2- 172.21.15.27:0/13624468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c06fcd0 0x7f636c0711d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6364009d90 tx=0x7f636402bfd0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.457+0000 7f637338f640 1 -- 172.21.15.27:0/13624468 shutdown_connections 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.458+0000 7f637338f640 1 --2- 172.21.15.27:0/13624468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c06fcd0 0x7f636c0711d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.458+0000 7f637338f640 1 -- 172.21.15.27:0/13624468 >> 172.21.15.27:0/13624468 conn(0x7f636c0cee90 msgr2=0x7f636c0cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.458+0000 7f637338f640 1 -- 172.21.15.27:0/13624468 shutdown_connections 2024-04-20T11:02:55.458 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.458+0000 7f637338f640 1 -- 172.21.15.27:0/13624468 wait complete. 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.459+0000 7f637338f640 1 Processor -- start 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.459+0000 7f637338f640 1 -- start start 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.459+0000 7f637338f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f636c06fcd0 0x7f636c0ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637338f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c151f40 0x7f636c1f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637338f640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f636c152320 0x7f636c149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637338f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f636c15c2d0 con 0x7f636c151f40 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637338f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f636c15bff0 con 0x7f636c06fcd0 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637338f640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f636c06f0a0 con 0x7f636c152320 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637092a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c151f40 0x7f636c1f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637092a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c151f40 0x7f636c1f24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57192/0 (socket says 172.21.15.27:57192) 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637092a640 1 -- 172.21.15.27:0/46525831 learned_addr learned my addr 172.21.15.27:0/46525831 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637112b640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f636c06fcd0 0x7f636c0ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637192c640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f636c152320 0x7f636c149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637092a640 1 -- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f636c152320 msgr2=0x7f636c149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637092a640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f636c152320 0x7f636c149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637092a640 1 -- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f636c06fcd0 msgr2=0x7f636c0ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637092a640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f636c06fcd0 0x7f636c0ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637092a640 1 -- 172.21.15.27:0/46525831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c15e240 con 0x7f636c151f40 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637092a640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c151f40 0x7f636c1f24c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c470 tx=0x7f635c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f6361ffb640 1 -- 172.21.15.27:0/46525831 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c0052d0 con 0x7f636c151f40 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f6361ffb640 1 -- 172.21.15.27:0/46525831 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f635c005490 con 0x7f636c151f40 2024-04-20T11:02:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c1f5d00 con 0x7f636c151f40 2024-04-20T11:02:55.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.460+0000 7f6361ffb640 1 -- 172.21.15.27:0/46525831 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c007280 con 0x7f636c151f40 2024-04-20T11:02:55.463 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.461+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c1f1ce0 con 0x7f636c151f40 2024-04-20T11:02:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.462+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c14f320 con 0x7f636c151f40 2024-04-20T11:02:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.462+0000 7f6361ffb640 1 -- 172.21.15.27:0/46525831 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f635c007420 con 0x7f636c151f40 2024-04-20T11:02:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.462+0000 7f6361ffb640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f635006c8f0 0x7f635006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.462+0000 7f6361ffb640 1 -- 172.21.15.27:0/46525831 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6685+0+0 (secure 0 0 0) 0x7f635c08b450 con 0x7f636c151f40 2024-04-20T11:02:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.463+0000 7f637112b640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f635006c8f0 0x7f635006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:55.464 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.463+0000 7f637112b640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f635006c8f0 0x7f635006edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6368005880 tx=0x7f6368009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:55.466 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.467+0000 7f6361ffb640 1 -- 172.21.15.27:0/46525831 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f635c059420 con 0x7f636c151f40 2024-04-20T11:02:55.693 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:55.693+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f636c04fcf0 con 0x7f636c151f40 2024-04-20T11:02:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.336+0000 7f6361ffb640 1 -- 172.21.15.27:0/46525831 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f635c058d20 con 0x7f636c151f40 2024-04-20T11:02:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f635006c8f0 msgr2=0x7f635006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f635006c8f0 0x7f635006edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6368005880 tx=0x7f6368009040 comp rx=0 tx=0).stop 2024-04-20T11:02:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c151f40 msgr2=0x7f636c1f24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c151f40 0x7f636c1f24c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c470 tx=0x7f635c00c930 comp rx=0 tx=0).stop 2024-04-20T11:02:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 shutdown_connections 2024-04-20T11:02:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f635006c8f0 0x7f635006edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f636c152320 0x7f636c149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f636c151f40 0x7f636c1f24c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 --2- 172.21.15.27:0/46525831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f636c06fcd0 0x7f636c0ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 >> 172.21.15.27:0/46525831 conn(0x7f636c0cee90 msgr2=0x7f636c14dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 shutdown_connections 2024-04-20T11:02:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.340+0000 7f637338f640 1 -- 172.21.15.27:0/46525831 wait complete. 2024-04-20T11:02:56.354 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T11:02:56.504 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.505+0000 7fcad657e640 1 Processor -- start 2024-04-20T11:02:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.505+0000 7fcad657e640 1 -- start start 2024-04-20T11:02:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.505+0000 7fcad657e640 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcad00c6160 0x7fcad015c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcad657e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad00c5d50 con 0x7fcad00cbdf0 2024-04-20T11:02:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcad657e640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad00ce060 con 0x7fcad00c66b0 2024-04-20T11:02:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcad657e640 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 -- 0x7fcad00ce5e0 con 0x7fcad00c6160 2024-04-20T11:02:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcacffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcad00cbdf0 0x7fcad00cd640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36952/0 (socket says 172.21.15.27:36952) 2024-04-20T11:02:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcacffff640 1 -- 172.21.15.27:0/676667980 learned_addr learned my addr 172.21.15.27:0/676667980 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcacf7fe640 1 --2- 172.21.15.27:0/676667980 >> v2:172.21.15.137:3300/0 conn(0x7fcad00c6160 0x7fcad015c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:56.509 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3103489031 0 0) 0x7fcad00c5d50 con 0x7fcad00cbdf0 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcabc003520 con 0x7fcad00cbdf0 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1343255673 0 0) 0x7fcabc003520 con 0x7fcad00cbdf0 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 >> v2:172.21.15.137:3300/0 conn(0x7fcad00c6160 msgr2=0x7fcad015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 --2- 172.21.15.27:0/676667980 >> v2:172.21.15.137:3300/0 conn(0x7fcad00c6160 0x7fcad015c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 >> v1:172.21.15.137:6789/0 conn(0x7fcad00c66b0 legacy=0x7fcad0165fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad00ce300 con 0x7fcad00cbdf0 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 454993359 0 0) 0x7fcac0001670 con 0x7fcad00cbdf0 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcacf7fe640 1 --2- 172.21.15.27:0/676667980 >> v2:172.21.15.137:3300/0 conn(0x7fcad00c6160 0x7fcad015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 >> v1:172.21.15.27:6789/0 conn(0x7fcad00cbdf0 legacy=0x7fcad00cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcabc0039b0 0x7fcabc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0043b0 0x7fcabc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcabc005080 0x7fcabc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcad00ce5e0 con 0x7fcabc0039b0 2024-04-20T11:02:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc00bfe0 con 0x7fcabc005080 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fcabc00c1e0 con 0x7fcabc0043b0 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.506+0000 7fcacffff640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcabc0039b0 0x7fcabc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.507+0000 7fcad4b1b640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcabc005080 0x7fcabc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.507+0000 7fcacf7fe640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0043b0 0x7fcabc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.507+0000 7fcacffff640 1 -- 172.21.15.27:0/676667980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0043b0 msgr2=0x7fcabc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.507+0000 7fcacffff640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0043b0 0x7fcabc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.507+0000 7fcacffff640 1 -- 172.21.15.27:0/676667980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcabc005080 msgr2=0x7fcabc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.507+0000 7fcacffff640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcabc005080 0x7fcabc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.507+0000 7fcad4b1b640 1 -- 172.21.15.27:0/676667980 reap_dead start 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.507+0000 7fcacffff640 1 -- 172.21.15.27:0/676667980 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc003520 con 0x7fcabc0039b0 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.507+0000 7fcacf7fe640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0043b0 0x7fcabc004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcacffff640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcabc0039b0 0x7fcabc003d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcac000c240 tx=0x7fcabc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344604b7f0a83e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0007d30 con 0x7fcabc0039b0 2024-04-20T11:02:56.511 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fcac0030070 con 0x7fcabc0039b0 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcaceffd640 1 -- 172.21.15.27:0/676667980 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac002cdc0 con 0x7fcabc0039b0 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcad657e640 1 -- 172.21.15.27:0/676667980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcabc0039b0 msgr2=0x7fcabc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcad657e640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcabc0039b0 0x7fcabc003d90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcac000c240 tx=0x7fcabc005830 comp rx=0 tx=0).stop 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcad657e640 1 -- 172.21.15.27:0/676667980 shutdown_connections 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcad657e640 1 --2- 172.21.15.27:0/676667980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcabc0039b0 0x7fcabc003d90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcad657e640 1 -- 172.21.15.27:0/676667980 >> 172.21.15.27:0/676667980 conn(0x7fcad00676c0 msgr2=0x7fcad0067ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcad657e640 1 -- 172.21.15.27:0/676667980 shutdown_connections 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.508+0000 7fcad657e640 1 -- 172.21.15.27:0/676667980 wait complete. 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.509+0000 7fcad657e640 1 Processor -- start 2024-04-20T11:02:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcad657e640 1 -- start start 2024-04-20T11:02:56.514 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcad657e640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0039b0 0x7fcad01577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcad657e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad01550a0 0x7fcad015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcad657e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcad015eac0 0x7fcad0152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcad657e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcad015baa0 con 0x7fcad01550a0 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcad657e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcad015be10 con 0x7fcad015eac0 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcad657e640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7fcad00c6ac0 con 0x7fcabc0039b0 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcacf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad01550a0 0x7fcad015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcacf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad01550a0 0x7fcad015ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57200/0 (socket says 172.21.15.27:57200) 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcacf7fe640 1 -- 172.21.15.27:0/1150517933 learned_addr learned my addr 172.21.15.27:0/1150517933 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.510+0000 7fcad4b1b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcad015eac0 0x7fcad0152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcacffff640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0039b0 0x7fcad01577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcacf7fe640 1 -- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0039b0 msgr2=0x7fcad01577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcacf7fe640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0039b0 0x7fcad01577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcacf7fe640 1 -- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcad015eac0 msgr2=0x7fcad0152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcacf7fe640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcad015eac0 0x7fcad0152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcacf7fe640 1 -- 172.21.15.27:0/1150517933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad01f5830 con 0x7fcad01550a0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcacf7fe640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad01550a0 0x7fcad015ca50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcab8011830 tx=0x7fcab8011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcaccff9640 1 -- 172.21.15.27:0/1150517933 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab800f9c0 con 0x7fcad01550a0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcaccff9640 1 -- 172.21.15.27:0/1150517933 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fcab800fb60 con 0x7fcad01550a0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad01580a0 con 0x7fcad01550a0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcaccff9640 1 -- 172.21.15.27:0/1150517933 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8015460 con 0x7fcad01550a0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.511+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad01f4d50 con 0x7fcad01550a0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.512+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad004fcf0 con 0x7fcad01550a0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.513+0000 7fcaccff9640 1 -- 172.21.15.27:0/1150517933 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcab8015600 con 0x7fcad01550a0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.513+0000 7fcaccff9640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcab006c920 0x7fcab006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.513+0000 7fcaccff9640 1 -- 172.21.15.27:0/1150517933 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6685+0+0 (secure 0 0 0) 0x7fcab808c820 con 0x7fcad01550a0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.513+0000 7fcacffff640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcab006c920 0x7fcab006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.514+0000 7fcacffff640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcab006c920 0x7fcab006ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcac0034000 tx=0x7fcac0009030 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:56.517 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.517+0000 7fcaccff9640 1 -- 172.21.15.27:0/1150517933 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcab8019030 con 0x7fcad01550a0 2024-04-20T11:02:56.702 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:56.702+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcad00cc8e0 con 0x7fcad01550a0 2024-04-20T11:02:57.338 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.338+0000 7fcaccff9640 1 -- 172.21.15.27:0/1150517933 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7fcab805a1a0 con 0x7fcad01550a0 2024-04-20T11:02:57.338 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2024-04-20T11:02:57.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.341+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcab006c920 msgr2=0x7fcab006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:57.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.341+0000 7fcad657e640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcab006c920 0x7fcab006ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcac0034000 tx=0x7fcac0009030 comp rx=0 tx=0).stop 2024-04-20T11:02:57.341 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.341+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad01550a0 msgr2=0x7fcad015ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:57.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.341+0000 7fcad657e640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad01550a0 0x7fcad015ca50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcab8011830 tx=0x7fcab8011cf0 comp rx=0 tx=0).stop 2024-04-20T11:02:57.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.342+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 shutdown_connections 2024-04-20T11:02:57.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.342+0000 7fcad657e640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7fcab006c920 0x7fcab006ede0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:57.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.342+0000 7fcad657e640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcad015eac0 0x7fcad0152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:57.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.342+0000 7fcad657e640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcad01550a0 0x7fcad015ca50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:57.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.342+0000 7fcad657e640 1 --2- 172.21.15.27:0/1150517933 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcabc0039b0 0x7fcad01577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:57.342 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.342+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 >> 172.21.15.27:0/1150517933 conn(0x7fcad00676c0 msgr2=0x7fcad00cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:57.344 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.345+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 shutdown_connections 2024-04-20T11:02:57.346 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.345+0000 7fcad657e640 1 -- 172.21.15.27:0/1150517933 wait complete. 2024-04-20T11:02:57.365 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T11:02:57.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.484+0000 7f4338ffb640 1 Processor -- start 2024-04-20T11:02:57.484 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.484+0000 7f4338ffb640 1 -- start start 2024-04-20T11:02:57.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.485+0000 7f4338ffb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43340cd6e0 0x7f433415b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:57.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.485+0000 7f4338ffb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43340ce710 con 0x7f43340cb940 2024-04-20T11:02:57.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.485+0000 7f4338ffb640 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43340c60a0 con 0x7f4334155610 2024-04-20T11:02:57.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.485+0000 7f4338ffb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 -- 0x7f43340c6380 con 0x7f43340cd6e0 2024-04-20T11:02:57.485 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.485+0000 7f4331d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43340cd6e0 0x7f433415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:57.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.485+0000 7f4332d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f43340cb940 0x7f4334165d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36966/0 (socket says 172.21.15.27:36966) 2024-04-20T11:02:57.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.485+0000 7f4332d9d640 1 -- 172.21.15.27:0/3836224154 learned_addr learned my addr 172.21.15.27:0/3836224154 (peer_addr_for_me v1:172.21.15.27:0/0) 2024-04-20T11:02:57.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.486+0000 7f4331d9b640 1 -- 172.21.15.27:0/3836224154 >> v1:172.21.15.137:6789/0 conn(0x7f4334155610 legacy=0x7f433406da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T11:02:57.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.486+0000 7f4331d9b640 1 -- 172.21.15.27:0/3836224154 >> v1:172.21.15.27:6789/0 conn(0x7f43340cb940 legacy=0x7f4334165d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:57.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.486+0000 7f4331d9b640 1 -- 172.21.15.27:0/3836224154 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43340c6660 con 0x7f43340cd6e0 2024-04-20T11:02:57.486 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.486+0000 7f433159a640 1 -- 172.21.15.27:0/3836224154 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1798149959 0 0) 0x7f43340ce710 con 0x7f43340cb940 2024-04-20T11:02:57.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.486+0000 7f4331d9b640 1 --2- 172.21.15.27:0/3836224154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43340cd6e0 0x7f433415b410 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f431c009790 tx=0x7f431c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8142f1df9f21993d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:57.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f433159a640 1 -- 172.21.15.27:0/3836224154 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c034070 con 0x7f43340cd6e0 2024-04-20T11:02:57.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f433159a640 1 -- 172.21.15.27:0/3836224154 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f431c030610 con 0x7f43340cd6e0 2024-04-20T11:02:57.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f433159a640 1 -- 172.21.15.27:0/3836224154 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c030a00 con 0x7f43340cd6e0 2024-04-20T11:02:57.487 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f4338ffb640 1 -- 172.21.15.27:0/3836224154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43340cd6e0 msgr2=0x7f433415b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:57.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f4338ffb640 1 --2- 172.21.15.27:0/3836224154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43340cd6e0 0x7f433415b410 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f431c009790 tx=0x7f431c02b960 comp rx=0 tx=0).stop 2024-04-20T11:02:57.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f4338ffb640 1 -- 172.21.15.27:0/3836224154 shutdown_connections 2024-04-20T11:02:57.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f4338ffb640 1 --2- 172.21.15.27:0/3836224154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43340cd6e0 0x7f433415b410 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:57.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f4338ffb640 1 -- 172.21.15.27:0/3836224154 >> 172.21.15.27:0/3836224154 conn(0x7f433406c310 msgr2=0x7f433406cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:57.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f4338ffb640 1 -- 172.21.15.27:0/3836224154 shutdown_connections 2024-04-20T11:02:57.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.487+0000 7f4338ffb640 1 -- 172.21.15.27:0/3836224154 wait complete. 2024-04-20T11:02:57.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4338ffb640 1 Processor -- start 2024-04-20T11:02:57.488 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4338ffb640 1 -- start start 2024-04-20T11:02:57.490 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4338ffb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43340cb940 0x7f43341f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4338ffb640 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43340cd6e0 0x7f43340cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4338ffb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4334155610 0x7f43340cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4338ffb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f433406e2e0 con 0x7f4334155610 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4338ffb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f433406e160 con 0x7f43340cb940 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4338ffb640 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 -- 0x7f43341559f0 con 0x7f43340cd6e0 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4332d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4334155610 0x7f43340cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.489+0000 7f4332d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4334155610 0x7f43340cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57220/0 (socket says 172.21.15.27:57220) 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4332d9d640 1 -- 172.21.15.27:0/1205723889 learned_addr learned my addr 172.21.15.27:0/1205723889 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f433259c640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43340cb940 0x7f43341f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4331d9b640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43340cd6e0 0x7f43340cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4332d9d640 1 -- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43340cd6e0 msgr2=0x7f43340cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4332d9d640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43340cd6e0 0x7f43340cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:57.491 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4332d9d640 1 -- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43340cb940 msgr2=0x7f43341f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4332d9d640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43340cb940 0x7f43341f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4332d9d640 1 -- 172.21.15.27:0/1205723889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43340d18b0 con 0x7f4334155610 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4331d9b640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43340cd6e0 0x7f43340cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f433259c640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43340cb940 0x7f43341f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4332d9d640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4334155610 0x7f43340cfef0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f432400c410 tx=0x7f432400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f432affd640 1 -- 172.21.15.27:0/1205723889 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240074e0 con 0x7f4334155610 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f432affd640 1 -- 172.21.15.27:0/1205723889 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f43240047e0 con 0x7f4334155610 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43340d1b90 con 0x7f4334155610 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f432affd640 1 -- 172.21.15.27:0/1205723889 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324004bf0 con 0x7f4334155610 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.490+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4334148380 con 0x7f4334155610 2024-04-20T11:02:57.492 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.491+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433404fcf0 con 0x7f4334155610 2024-04-20T11:02:57.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.492+0000 7f432affd640 1 -- 172.21.15.27:0/1205723889 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4324011070 con 0x7f4334155610 2024-04-20T11:02:57.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.492+0000 7f432affd640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f431406c8f0 0x7f431406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T11:02:57.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.492+0000 7f432affd640 1 -- 172.21.15.27:0/1205723889 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6247+0+0 (secure 0 0 0) 0x7f432408b930 con 0x7f4334155610 2024-04-20T11:02:57.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.492+0000 7f433259c640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f431406c8f0 0x7f431406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T11:02:57.493 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.493+0000 7f433259c640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f431406c8f0 0x7f431406edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f432c0045b0 tx=0x7f432c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T11:02:57.496 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.497+0000 7f432affd640 1 -- 172.21.15.27:0/1205723889 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f432401a2b0 con 0x7f4334155610 2024-04-20T11:02:57.680 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:57.681+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f433414ad00 con 0x7f4334155610 2024-04-20T11:02:58.350 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.350+0000 7f432affd640 1 -- 172.21.15.27:0/1205723889 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f4324059b40 con 0x7f4334155610 2024-04-20T11:02:58.350 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2024-04-20T11:02:58.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f431406c8f0 msgr2=0x7f431406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:58.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f431406c8f0 0x7f431406edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f432c0045b0 tx=0x7f432c009040 comp rx=0 tx=0).stop 2024-04-20T11:02:58.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4334155610 msgr2=0x7f43340cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T11:02:58.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4334155610 0x7f43340cfef0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f432400c410 tx=0x7f432400c8d0 comp rx=0 tx=0).stop 2024-04-20T11:02:58.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 shutdown_connections 2024-04-20T11:02:58.354 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:6832/2036904965,v1:172.21.15.137:6833/2036904965] conn(0x7f431406c8f0 0x7f431406edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:58.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4334155610 0x7f43340cfef0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:58.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f43340cd6e0 0x7f43340cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:58.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 --2- 172.21.15.27:0/1205723889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43340cb940 0x7f43341f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T11:02:58.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.354+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 >> 172.21.15.27:0/1205723889 conn(0x7f433406c310 msgr2=0x7f43341e75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T11:02:58.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.355+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 shutdown_connections 2024-04-20T11:02:58.355 INFO:teuthology.orchestra.run.smithi027.stderr:2024-04-20T11:02:58.355+0000 7f4338ffb640 1 -- 172.21.15.27:0/1205723889 wait complete. 2024-04-20T11:02:58.372 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-20T11:02:58.373 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T11:02:58.373 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.383 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T11:02:58.384 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T11:02:58.384 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.434 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T11:02:58.434 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-20T11:02:58.434 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.484 INFO:tasks.ceph.mds.e:Stopped 2024-04-20T11:02:58.485 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T11:02:58.485 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.496 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T11:02:58.496 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T11:02:58.496 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.547 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T11:02:58.547 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-20T11:02:58.547 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.599 INFO:tasks.ceph.mds.f:Stopped 2024-04-20T11:02:58.599 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-20T11:02:58.599 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-20T11:02:58.599 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.633 INFO:tasks.ceph.osd.0:Stopped 2024-04-20T11:02:58.633 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-20T11:02:58.633 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.707 INFO:tasks.ceph.osd.1:Stopped 2024-04-20T11:02:58.707 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-20T11:02:58.707 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.779 INFO:tasks.ceph.osd.2:Stopped 2024-04-20T11:02:58.779 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-20T11:02:58.779 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.854 INFO:tasks.ceph.osd.3:Stopped 2024-04-20T11:02:58.854 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-20T11:02:58.854 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.885 INFO:tasks.ceph.osd.4:Stopped 2024-04-20T11:02:58.885 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-20T11:02:58.885 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:58.968 INFO:tasks.ceph.osd.5:Stopped 2024-04-20T11:02:58.968 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-20T11:02:58.968 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:59.003 INFO:tasks.ceph.osd.6:Stopped 2024-04-20T11:02:59.003 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-20T11:02:59.004 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:59.080 INFO:tasks.ceph.osd.7:Stopped 2024-04-20T11:02:59.080 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-20T11:02:59.080 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-20T11:02:59.080 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:59.105 INFO:tasks.ceph.mgr.y:Stopped 2024-04-20T11:02:59.105 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-20T11:02:59.105 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:59.149 INFO:tasks.ceph.mgr.x:Stopped 2024-04-20T11:02:59.149 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-20T11:02:59.149 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-20T11:02:59.149 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:59.161 INFO:tasks.ceph.mon.a:Stopped 2024-04-20T11:02:59.161 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-20T11:02:59.161 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:59.202 INFO:tasks.ceph.mon.b:Stopped 2024-04-20T11:02:59.202 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T11:02:59.202 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T11:02:59.255 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T11:02:59.255 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-20T11:02:59.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v 'Replacing daemon mds' | head -n 1 2024-04-20T11:02:59.299 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T11:02:59.299 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-20T11:02:59.607 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T11:02:59.607 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-20T11:02:59.663 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T11:02:59.663 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-20T11:02:59.703 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T11:02:59.704 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-20T11:02:59.787 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T11:02:59.787 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-20T11:03:00.017 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T11:03:00.017 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-20T11:03:00.096 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T11:03:00.096 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-20T11:03:00.160 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T11:03:00.160 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-20T11:03:00.212 INFO:tasks.ceph:Archiving mon data... 2024-04-20T11:03:00.213 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/data/mon.a.tgz 2024-04-20T11:03:00.213 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-04-20T11:03:00.229 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.SzgZuJFgPS 2024-04-20T11:03:00.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.SzgZuJFgPS 2024-04-20T11:03:00.373 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.SzgZuJFgPS 2024-04-20T11:03:00.456 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.SzgZuJFgPS is 410KB 2024-04-20T11:03:00.552 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.SzgZuJFgPS 2024-04-20T11:03:00.567 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/data/mon.b.tgz 2024-04-20T11:03:00.567 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-04-20T11:03:00.585 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.BfnnXjxtPV 2024-04-20T11:03:00.586 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.BfnnXjxtPV 2024-04-20T11:03:00.766 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.BfnnXjxtPV 2024-04-20T11:03:00.849 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.BfnnXjxtPV is 533KB 2024-04-20T11:03:00.955 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.BfnnXjxtPV 2024-04-20T11:03:00.969 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/data/mon.c.tgz 2024-04-20T11:03:00.970 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-04-20T11:03:01.032 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.F7Zspqi764 2024-04-20T11:03:01.032 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.F7Zspqi764 2024-04-20T11:03:01.196 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.F7Zspqi764 2024-04-20T11:03:01.277 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.F7Zspqi764 is 533KB 2024-04-20T11:03:01.374 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.F7Zspqi764 2024-04-20T11:03:01.389 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-20T11:03:01.389 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T11:03:01.391 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T11:03:01.475 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T11:03:01.506 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T11:03:01.506 INFO:tasks.ceph:Archiving crash dumps... 2024-04-20T11:03:01.508 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/remote/smithi027/crash 2024-04-20T11:03:01.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T11:03:01.542 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/remote/smithi137/crash 2024-04-20T11:03:01.543 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T11:03:01.569 INFO:tasks.ceph:Shutting down logrotate 2024-04-20T11:03:01.570 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:03:01.587 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:03:01.659 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T11:03:01.684 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T11:03:01.707 INFO:tasks.ceph:Compressing logs... 2024-04-20T11:03:01.707 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T11:03:01.727 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23634.log 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23463.log 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.23634.log: /var/log/ceph/ceph-client.admin.23463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23714.log 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.23634.log.gz 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.23463.log.gz 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23674.log 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.23714.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.23674.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.23714.log.gz 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.23674.log.gz 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-20T11:03:01.791 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23754.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23594.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.23754.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.23754.log.gz 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.23594.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr: 88.5%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.23594.log.gz 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23543.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23503.log 2024-04-20T11:03:01.792 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.23543.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.23543.log.gz 2024-04-20T11:03:01.797 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-20T11:03:01.797 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.23503.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.23503.log.gz 2024-04-20T11:03:01.812 INFO:teuthology.orchestra.run.smithi137.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-20T11:03:01.812 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-20T11:03:01.812 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-20T11:03:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36800.log 2024-04-20T11:03:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36940.log 2024-04-20T11:03:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37473.log 2024-04-20T11:03:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36800.log: /var/log/ceph/ceph-client.admin.36940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37260.log 2024-04-20T11:03:01.839 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36800.log.gz 2024-04-20T11:03:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37473.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36940.log.gz 2024-04-20T11:03:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37218.log 2024-04-20T11:03:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37260.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37473.log.gz 2024-04-20T11:03:01.839 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36612.log 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37260.log.gz 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38555.log 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37218.log.gz 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34231.log 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38555.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36612.log.gz 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33873.log 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34231.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38555.log.gz 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33873.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34231.log.gz 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36214.log 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40182.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33873.log.gz 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35840.log 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39578.log 2024-04-20T11:03:01.840 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35840.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36214.log.gz 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37515.log 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39578.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35840.log.gz 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33663.log 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37515.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39578.log.gz 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31313.log 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33663.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37515.log.gz 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30983.log 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31313.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33663.log.gz 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23751.log 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31313.log.gz 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40224.log 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30983.log: /var/log/ceph/ceph-client.admin.23751.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30983.log.gz 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40224.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.23751.log.gz 2024-04-20T11:03:01.841 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23854.log 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40224.log.gz 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30986.log 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.23854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35276.log 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.23854.log.gz 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30102.log 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35276.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30986.log.gz 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33957.log 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35276.log.gz 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32260.log 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30102.log.gz 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34655.log 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39536.log 2024-04-20T11:03:01.842 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33957.log.gz 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34655.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32260.log.gz 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39536.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.34655.log.gz 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36486.log 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35696.log 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39536.log.gz 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40308.log 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36486.log.gz 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32005.log 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31749.log 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35696.log.gz 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35654.log 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40308.log.gz 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31749.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32005.log.gz 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30978.log 2024-04-20T11:03:01.843 INFO:teuthology.orchestra.run.smithi027.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31749.log.gz 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35132.log 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32404.log 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35654.log.gz 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35132.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30978.log.gz 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38639.log 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35132.log.gz 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38639.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38639.log.gz 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36570.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.32404.log.gz 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31312.log 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31921.log 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.31312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40082.log 2024-04-20T11:03:01.844 INFO:teuthology.orchestra.run.smithi027.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31312.log.gz 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31837.log 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.31921.log.gz 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38935.log 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31837.log: gzip -5 88.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.40082.log.gz 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.37698.log 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31837.log.gz 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33915.log 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37698.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38935.log.gz 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31879.log 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33915.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37698.log.gz 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39284.log 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33915.log.gz 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33453.log 2024-04-20T11:03:01.845 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39284.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31879.log.gz 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35528.log 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33453.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39284.log.gz 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34087.log 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33453.log.gz 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32602.log 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34087.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35528.log.gz 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31321.log 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32602.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34087.log.gz 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30762.log 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31321.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.32602.log.gz 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31321.log.gz 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30460.log 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37066.log 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30762.log.gz 2024-04-20T11:03:01.846 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.30460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36046.log 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30460.log.gz 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34129.log 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37066.log.gz 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29790.log 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36046.log: /var/log/ceph/ceph-client.admin.34129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30544.log 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36046.log.gz 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34129.log.gz 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.29790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37938.log 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38471.log 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29790.log.gz 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30544.log: /var/log/ceph/ceph-client.admin.37938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39452.log 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30544.log.gz 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37938.log.gz/var/log/ceph/ceph-client.admin.38471.log: 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37389.log 2024-04-20T11:03:01.847 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38471.log.gz 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40428.log 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39452.log: /var/log/ceph/ceph-client.admin.37389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36862.log 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39452.log.gz 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40428.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37389.log.gz 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39103.log 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40428.log.gz 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37347.log 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36862.log: /var/log/ceph/ceph-client.admin.39103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31322.log 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36862.log.gz 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37347.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39103.log.gz 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37614.log 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37347.log.gz 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39704.log 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31322.log: /var/log/ceph/ceph-client.admin.37614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31653.log 2024-04-20T11:03:01.848 INFO:teuthology.orchestra.run.smithi027.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31322.log.gz 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39704.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37614.log.gz 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33537.log 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31653.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39704.log.gz 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33537.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31653.log.gz -5 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33327.log 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.33537.log.gz --verbose 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr: -- /var/log/ceph/ceph-client.admin.35570.log 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.33327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31560.log 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35570.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33327.log.gz 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34613.log 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38799.log 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr: 88.8%/var/log/ceph/ceph-client.admin.31560.log: -- replaced with /var/log/ceph/ceph-client.admin.35570.log.gz 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34613.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.31560.log.gz -5 2024-04-20T11:03:01.849 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30982.log 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34445.log 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38799.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34613.log.gz 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39019.log 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38799.log.gz 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30982.log.gz 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34445.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39998.log 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34445.log.gz 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39019.log.gz 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40040.log 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39998.log.gz 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38513.log 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:gzip/var/log/ceph/ceph-client.admin.40040.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35216.log 2024-04-20T11:03:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38513.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40040.log.gz 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40514.log 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38513.log.gz 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32215.log 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35216.log: /var/log/ceph/ceph-client.admin.40514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39830.log 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32215.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40514.log.gz 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35216.log.gz 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32215.log.gz 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37557.log 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39830.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39914.log 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39830.log.gz 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37557.log.gz/var/log/ceph/ceph-osd.0.log: 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35318.log 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31311.log 2024-04-20T11:03:01.851 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35318.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39914.log.gz 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35318.log.gz 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36982.log 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38258.log 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36982.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31311.log.gz 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36982.log.gz 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31518.log 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39494.log 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31518.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38258.log.gz 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33243.log 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31518.log.gz 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz/var/log/ceph/ceph-client.admin.39494.log: 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30985.log 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr: 87.9%/var/log/ceph/ceph-client.admin.33243.log: -- replaced with /var/log/ceph/ceph-client.admin.39494.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34361.log 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.852 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30985.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33243.log.gz 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32089.log 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35444.log 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30985.log.gz 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32089.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34361.log.gz 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32089.log.gz 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36384.log 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35444.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.17458.log 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36384.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35444.log.gz 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37431.log 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36384.log.gz 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph.tmp-client.admin.17458.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17458.log.gz 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33495.log 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38022.log 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37431.log.gz 2024-04-20T11:03:01.853 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33747.log 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33495.log.gz 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37854.log 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38022.log.gz 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34757.log 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37854.log: /var/log/ceph/ceph-client.admin.33747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29594.log 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34757.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33747.log.gz 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37854.log.gz 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38597.log 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34757.log.gz 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31318.log 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.29594.log: /var/log/ceph/ceph-client.admin.38597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32047.log 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31318.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29594.log.gz 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38597.log.gz 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38082.log 2024-04-20T11:03:01.854 INFO:teuthology.orchestra.run.smithi027.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31318.log.gz 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5/var/log/ceph/ceph-client.admin.32047.log: --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38082.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32047.log.gz 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35048.log 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38082.log.gz 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36300.log 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.35048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36088.log 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36300.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35048.log.gz 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37302.log 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36300.log.gz/var/log/ceph/ceph-client.admin.36088.log: 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36088.log.gz 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23487.log 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36258.log 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37302.log.gz 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.23487.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.23487.log.gz 2024-04-20T11:03:01.855 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32362.log 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34273.log 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36258.log.gz 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32362.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32362.log.gz 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38699.log 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35924.log 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34273.log: /var/log/ceph/ceph-client.admin.38699.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34273.log.gz 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30804.log 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38699.log.gz 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35924.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35924.log.gz 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30979.log 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31602.log 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30804.log: /var/log/ceph/ceph-client.admin.30979.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30804.log.gz 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39662.log 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30979.log.gz 2024-04-20T11:03:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31602.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31602.log.gz 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35612.log 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34319.log 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39662.log: /var/log/ceph/ceph-client.admin.35612.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39662.log.gz 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.35612.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.31316.log 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34319.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34319.log.gz 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38977.log 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37150.log 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31316.log: /var/log/ceph/ceph-client.admin.38977.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31316.log.gz 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38977.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37024.log 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37150.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37150.log.gz 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33369.log 2024-04-20T11:03:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36426.log 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34487.log 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37024.log.gz 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36426.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33369.log.gz 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35174.log 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38300.log 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36426.log.gz 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34487.log.gz 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29976.log 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35174.log: /var/log/ceph/ceph-client.admin.38300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34799.log 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr: 88.5%/var/log/ceph/ceph-client.admin.29976.log: -- replaced with /var/log/ceph/ceph-client.admin.35174.log.gz 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38300.log.gz 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29976.log.gz 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31963.log 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30188.log 2024-04-20T11:03:01.858 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34799.log.gz 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33108.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31963.log.gz 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36004.log 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30188.log.gz 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.33108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.33108.log.gz 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40142.log 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32668.log 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0%/var/log/ceph/ceph-client.admin.40142.log: -- replaced with /var/log/ceph/ceph-client.admin.36004.log.gz 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32536.log 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40142.log.gz 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30720.log 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32470.log 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30720.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.32668.log.gz 2024-04-20T11:03:01.859 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30720.log.gz 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:gzip 91.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.32536.log.gz -- 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr: /var/log/ceph/ceph-client.admin.30586.log 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35360.log 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32470.log: /var/log/ceph/ceph-client.admin.30586.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35360.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30586.log.gz 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35360.log.gz 90.8% -- replaced with /var/log/ceph/ceph-client.admin.32470.log.gz 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38166.log 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36528.log 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30250.log 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38166.log: /var/log/ceph/ceph-client.admin.36528.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38166.log.gz 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.36528.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.29675.log 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30250.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30250.log.gz 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36696.log 2024-04-20T11:03:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33579.log 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.29675.log: /var/log/ceph/ceph-client.admin.36696.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29675.log.gz 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36696.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30018.log 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33579.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33579.log.gz 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38124.log 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37108.log 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30018.log: /var/log/ceph/ceph-client.admin.38124.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30018.log.gz 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38124.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36738.log 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37108.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37108.log.gz 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31695.log 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29932.log 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36738.log: /var/log/ceph/ceph-client.admin.31695.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36738.log.gz 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38208.log 2024-04-20T11:03:01.861 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31695.log.gz 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.29932.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29932.log.gz 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37759.log 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33831.log 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38208.log: /var/log/ceph/ceph-client.admin.37759.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38208.log.gz 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34529.log 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37759.log.gz 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33831.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33831.log.gz 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33705.log 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35486.log 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34529.log: /var/log/ceph/ceph-client.admin.33705.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34529.log.gz 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37980.log 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33705.log.gz 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35486.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35486.log.gz 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39746.log 2024-04-20T11:03:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29890.log 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37980.log: /var/log/ceph/ceph-client.admin.39746.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37980.log.gz 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36342.log 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39746.log.gz 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.29890.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29890.log.gz 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31317.log 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32131.log 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36342.log: /var/log/ceph/ceph-client.admin.31317.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36342.log.gz 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32173.log 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31317.log.gz 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32131.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32131.log.gz 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30334.log 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33285.log 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32173.log: /var/log/ceph/ceph-client.admin.30334.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32173.log.gz 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30334.log.gz 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34571.log 2024-04-20T11:03:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33285.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33285.log.gz 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30984.log 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35090.log 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34571.log: /var/log/ceph/ceph-client.admin.30984.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34571.log.gz 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30984.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36172.log 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35090.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35090.log.gz 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39061.log 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39242.log 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36172.log: /var/log/ceph/ceph-client.admin.39061.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36172.log.gz 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30376.log 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39061.log.gz 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39242.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39242.log.gz 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35402.log 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30977.log 2024-04-20T11:03:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30376.log: /var/log/ceph/ceph-client.admin.35402.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30376.log.gz 2024-04-20T11:03:01.865 INFO:teuthology.orchestra.run.smithi027.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35402.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34841.log 2024-04-20T11:03:01.865 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.866 INFO:teuthology.orchestra.run.smithi137.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-20T11:03:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30977.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30977.log.gz 2024-04-20T11:03:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39788.log 2024-04-20T11:03:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39410.log 2024-04-20T11:03:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34841.log: /var/log/ceph/ceph-client.admin.39788.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34841.log.gz 2024-04-20T11:03:01.883 INFO:teuthology.orchestra.run.smithi027.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39788.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33411.log 2024-04-20T11:03:01.883 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39410.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39410.log.gz 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40350.log 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30292.log 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33411.log: /var/log/ceph/ceph-client.admin.40350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37656.log 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30292.log: 89.1% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33411.log.gz 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40350.log.gz 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30292.log.gz 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33621.log 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30628.log 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34403.log 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33621.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37656.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33621.log.gz 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30628.log: 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30628.log.gz 2024-04-20T11:03:01.884 INFO:teuthology.orchestra.run.smithi027.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40472.log 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30678.log 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34403.log.gz 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34883.log 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39368.log 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40472.log.gz 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30678.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.34883.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30678.log.gz 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39368.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34883.log.gz 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39368.log.gz 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37896.log 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31791.log 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.37896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35882.log 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37896.log.gz 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.31791.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31791.log.gz 2024-04-20T11:03:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40266.log 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.35882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38429.log 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35882.log.gz 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.40266.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40266.log.gz 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38387.log 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38342.log 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38429.log: /var/log/ceph/ceph-client.admin.38387.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38429.log.gz 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38387.log.gz 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32302.log 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.38342.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38342.log.gz 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38861.log 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30418.log 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.32302.log: /var/log/ceph/ceph-client.admin.38861.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32302.log.gz 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38861.log.gz 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36654.log 2024-04-20T11:03:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30418.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30418.log.gz 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34697.log 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33789.log 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36654.log: /var/log/ceph/ceph-client.admin.34697.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36654.log.gz 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34697.log.gz 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39620.log 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.33789.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33789.log.gz 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30502.log 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36130.log 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39620.log: /var/log/ceph/ceph-client.admin.30502.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39620.log.gz 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30502.log.gz 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30144.log 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.36130.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36130.log.gz 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35780.log 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39872.log 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30144.log: /var/log/ceph/ceph-client.admin.35780.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30144.log.gz 2024-04-20T11:03:01.887 INFO:teuthology.orchestra.run.smithi027.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35780.log.gz 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35738.log 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34189.log 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39872.log: /var/log/ceph/ceph-client.admin.35738.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39872.log.gz 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35738.log.gz 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39956.log 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30060.log 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39956.log: /var/log/ceph/ceph-client.admin.34189.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34189.log.gz 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39956.log.gz 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32770.log 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39326.log 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.30060.log: /var/log/ceph/ceph-client.admin.32770.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30060.log.gz 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32770.log.gz 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.admin.39326.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39326.log.gz 2024-04-20T11:03:01.888 INFO:teuthology.orchestra.run.smithi027.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-20T11:03:01.944 INFO:teuthology.orchestra.run.smithi137.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-20T11:03:02.058 INFO:teuthology.orchestra.run.smithi137.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-20T11:03:02.272 INFO:teuthology.orchestra.run.smithi027.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-20T11:03:02.751 INFO:teuthology.orchestra.run.smithi027.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-20T11:03:02.949 INFO:teuthology.orchestra.run.smithi137.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-20T11:03:03.129 INFO:teuthology.orchestra.run.smithi137.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-20T11:03:03.323 INFO:teuthology.orchestra.run.smithi027.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-20T11:03:03.351 INFO:teuthology.orchestra.run.smithi027.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-20T11:03:03.386 INFO:teuthology.orchestra.run.smithi027.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-20T11:03:03.388 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-04-20T11:03:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m1.649s 2024-04-20T11:03:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m6.222s 2024-04-20T11:03:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.337s 2024-04-20T11:03:03.926 INFO:teuthology.orchestra.run.smithi137.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-20T11:03:04.055 INFO:teuthology.orchestra.run.smithi137.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-20T11:03:46.059 INFO:teuthology.orchestra.run.smithi137.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-20T11:03:46.061 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-04-20T11:03:46.062 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m44.300s 2024-04-20T11:03:46.062 INFO:teuthology.orchestra.run.smithi137.stderr:user 0m50.129s 2024-04-20T11:03:46.062 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m1.581s 2024-04-20T11:03:46.062 INFO:tasks.ceph:Archiving logs... 2024-04-20T11:03:46.062 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/remote/smithi027/log 2024-04-20T11:03:46.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T11:03:47.138 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/remote/smithi137/log 2024-04-20T11:03:47.139 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T11:03:55.114 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T11:03:55.125 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T11:03:55.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T11:03:55.128 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T11:03:55.217 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T11:03:55.226 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T11:03:55.226 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:03:55.229 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T11:03:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:03:55.247 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-04-20T11:03:55.249 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:03:55.249 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-04-20T11:03:55.249 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 4 +541us[ +541us] +/- 73ms 2024-04-20T11:03:55.249 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 -17us[ -19us] +/- 78ms 2024-04-20T11:03:55.249 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -648us[ -648us] +/- 90ms 2024-04-20T11:03:55.249 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:03:55.251 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T11:03:55.251 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-04-20T11:03:55.251 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 2 +533us[ +533us] +/- 73ms 2024-04-20T11:03:55.251 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -27us[ -33us] +/- 78ms 2024-04-20T11:03:55.251 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -638us[ -638us] +/- 90ms 2024-04-20T11:03:55.251 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T11:03:55.252 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T11:03:55.261 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T11:03:55.262 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T11:03:55.274 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T11:03:55.349 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T11:03:55.400 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:03:55.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:03:55.429 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T11:03:55.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:03:55.457 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T11:03:55.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T11:03:55.507 INFO:teuthology.task.internal:Duration was 796.916283 seconds 2024-04-20T11:03:55.508 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T11:03:55.516 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T11:03:55.517 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T11:03:55.519 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T11:03:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:03:55.558 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T11:03:55.833 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T11:03:55.833 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-04-20T11:03:55.833 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T11:03:55.896 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-04-20T11:03:55.896 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T11:03:55.920 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T11:03:55.920 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T11:03:55.939 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T11:03:55.986 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T11:03:55.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T11:03:56.009 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T11:03:56.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T11:03:56.101 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T11:03:56.102 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T11:03:56.127 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T11:03:56.151 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T11:03:56.163 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:03:56.169 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:03:56.190 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-04-20T11:03:56.213 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-04-20T11:03:56.232 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:03:56.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:03:56.257 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T11:03:56.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:03:56.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T11:03:56.296 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T11:03:56.297 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/remote/smithi027 2024-04-20T11:03:56.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T11:03:56.330 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664497/remote/smithi137 2024-04-20T11:03:56.331 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T11:03:56.360 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T11:03:56.360 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T11:03:56.367 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T11:03:56.410 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T11:03:56.423 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T11:03:56.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T11:03:56.433 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T11:03:56.433 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T11:03:56.436 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T11:03:56.448 INFO:teuthology.orchestra.run.smithi027.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 11:03 /home/ubuntu/cephtest 2024-04-20T11:03:56.464 INFO:teuthology.orchestra.run.smithi137.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 11:03 /home/ubuntu/cephtest 2024-04-20T11:03:56.466 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T11:03:56.476 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T11:03:56.499 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} duration: 796.9162831306458 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-20T11:03:56.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T11:03:56.580 INFO:teuthology.run:pass